2013.05 Activity Report

From IMarine Wiki

Jump to: navigation, search

Contents

This Activity Report captures the activities performed in May 2013.

This report is organized in two parts:

Progress and Achievements at WP level

WP1: Administrative and Financial Management

WP Leader: ERCIM - Participants: CNR

  • SB teleconference call participation and minutes taking
  • Subcontract with Edward finalised and sent to VUB
  • Periodic Report template completed and sent to all partners for contribution
  • Monitor and submit of the due deliverables and milestone achievement statements


N/A


  • Subcontract with Edward finalised and 1st payment to be made
  • Periodic Report template completion

WP2: Project Management

  • Organization and chairing of the monthly Steering Board conference call (16.05) and revision of the minutes;
  • Organization and chairing of the Monthly PEB (21.05) and preparation of the minutes;
  • Liaisons with different initiatives for the organization of joint dissemination events (e.g. Biodiversity Informatics Conference, Blue Hackathon, EUDAT Conference) and for the establishment of different forms of collaborations (JPI Oceans; RDA);
  • Finalization of the Programme of the iMarine Workshop on 14th & 15th May 2013 in Brussels;
  • Preparation of the presentations and chairing of the iMarine Workshop;
  • First round of revision of the "Executive Summary" and "Record of the Workshop" documents produced by Trust-IT;
  • Initial steps in the preparation of a White Paper reporting the iMarine view of actions and activities. Preparatory call on 20.05;
  • Day-by-day technical and scientific activities supervision and monitoring;
  • Planning for the shut-down of the BCSW and transfer of its content to the iMarine shared workspace;
  • Liaising with NEAFC on drafting of subcontract;
  • Participation of the TD in a series of WP specific calls dedicated to the coordination of different activities: (22.05, 24.05, 29.05, 30.05 - related to WP8) and (30.05 -related to WP10).


Nothing to report.


  • Monthly Steering and PEB meetings;
  • Liaisons with different initiatives for the organization of joint dissemination events and for the establishment of different forms of collaborations;
  • Finalization of the Programme of the iMarine Workshop on 14th & 15th May 2013 in Brussels and preparation of the presentations;
  • Day-by-day technical and scientific activities supervision and monitoring;
  • Planning for the shut-down of the BCSW and transfer of its content to the iMarine shared workspace.

WP3: Governance and Policy Development

WP Leader: FAO - Participants: ERCIM, CNR, NKUA, FORTH, TERRADUE, FIN, UNESCO, CRIA, IRD

In Task 3.1

  • Attend the Brussels iMarine meeting
  • Dispatch of Board report
  • Established contact for AustralFish; a local adaptation for applifish
  • Meeting with Conservation international on geomorphology map
  • Contact French Marine Fisheries Directorate DPMA to collect VALID requirements
  • Promote the iMarine infrastructure for providing pool with the Helix Nebula Science Cloud
  • Promote opportunities for iMarine infrastructure to define MoE with FP7 GEOWOW (UNESCO IOC)
  • Promote the iMarine infrastructure towards activities in Mediteran area (via ACRI-ST)

In Task 3.2

  • Launching activities with DPMA on Valid (2.5 WD),
  • Analyze the DG-Mare socio economic indicators (1 WD),
  • Continue policy work related to codelists (1 wd) and
  • Commence wiki policy pages review (3.5).

In Task 3.3.

  • FORTH; TLO-Related activities https://issue.imarine.research-infrastructures.eu/ticket/224
  • Cotrix (FAO HQ 8 days, Bangkok 9 days); Continuation of model development, UI development stopped mid-May (developer left), plugins developed for SDMX.
  • FIGIS geospatial infrastructure (1 day):
    • cleaning Geoserver / GeoWebCache (mainly agree on convention names for GS/GWC layers) and other GS layer management
    • discus FAO switching to Postgresql/Postgis
    • update the Java routine procedure to publish species distribution metadata (still waiting for FLOD inputs to batch update the GN metadata & add missing species)
    • prepare summary for FI e-bulletin (data sharing through GeoNetwork)
  • SPREAD (1 day)
    • implement a standalone Java application (basic area-weighted reallocation) - sent to Terradue for deployment in iMarine WPS-Hadoop (waiting for this deployment to do next checkpoint with T2 and ask CNR for integration in TimeSeries environment)
    • communication with Terradue
  • Geospatial Cluster (2 days)
    • communication regarding the GOLD GIS layers (with Marc mainly)
    • discussions with CNR and FAO colleagues about pushing towards the use of remote data sources (WFS) for data shared by data providers
    • test the GeoExplorer application / report bugs
  • FLOD started implementation of input/output mechanism to the FAO Virtual repository and Cotrix;


The beneficiary should report here major issues faced in the reporting period and the identified corrective actions, if any.


  • produced MS16
  • produced collaboration key-points for GeoMorhology
  • produced collaboration proposal for NEAFC
  • produced development proposal for AustralFish

WP4: Communication, Dissemination and Training

WP Leader: TRUST-IT - Participants: CNR, NKUA, FAO

EuroMarine (http://www.euromarineconsortium.eu/) is organising a workshop to define a "Web Environment" that will provide services to the European Marine scientific community as an effective mean to reach integration among individuals, institutions and consortia across the fields of ‘omics, biodiversity and ecology (see the attached programme). Services will address social networking, common/customisable directories (white, yellow and blue pages), news and events posting, scholarly publications (data and papers), education and outreach. Participants will include representatives of past and present EU projects/initiatives/infrastrutures, and web service providers, such as Google, Facebook, Twitter, Skype, LinkedIn, Thomson Reuters, Elsevier and Wikipedia.

  • Training videos have been prepared by various partners.
  • May newsletter issue


The major issues faced in the reporting period and the identified corrective actions should be reported here


  • Delivery of training videos.
  • iMarine participation to Eurmarine

WP5: iMarine Data e-Infrastructure Deployment and Operation

WP Leader: CERN - Participants: CNR, NKUA, FAO, FIN

for T5.1:

CERN completed the Ecosystem VOVirtual Organization; move to the Pre-prod environment ( access has been removed from portal.d4science.org) and the 3 VREs dismissed ( TryIt has been kept as pre/prod VRE).

The installation of the 2.14 in pre-production have been completed followed by the production upgrade to 2.14.

By the end of the month the pre-prod infra has started to be upgraded to gCube 2.15.

In parallel the deployment of the Enabling layer on the secure infrastructure has been completed.

Deliverable D5.3 was edited together with CNR ( sent to official review beginning of June)

CNR has also upgraded 2 nodes in the EGI infrastructure to EMI 3.

for T5.2:

CNR installed new nodes for mongoDB development infrastructure and 1 new VM to test different postgis version and geoserver/geonetwork;

NKUA scheduled the creation of a Virtual Machine for migration of portal.d4science.org, which will be performed later in June.

for T5.3:

Nagios server was configured by CERN to monitor the new service deployed on the production infra:

  • Thredds
  • WPS
  • data.d4science.org tomcat


FIN is still not contributing to the WP.

CNR restarted all xen dom0 and VM restarted and reconfigured with minimum downtime, to fix CVE-2013-0871 kernel bug and the kernel was updated on the eticsbuild host to fix VM instability;


  • Infrastructure upgrade to gCube 2.14 ( 10/05)

WP6: Virtual Research Environments Deployment and Operation

WP Leader: CNR - Participants: ERCIM, NKUA, FAO, FIN, UNESCO, CRIA, IRD

For Virtual Research Environments Operation:

  • the previously deployed Virtual Research Environments have been maintained in operation;
  • the request for a new VRE supporting Taxon Name Disambiguation is still under analysis;

For resources and tools provision:

  • the development of a Virtual Repository (pluggable client API for standards-based Data Import and Publication) started. More information at the next TCom;
  • myOceans and World Ocean Atlas indexed by iMarine GeoNetwork;
  • Reinforced the OAI-PMH TM plugin (e.g. bug fix, collection cardinality);
  • Development and release of the FIGIS TM plugin (for accessing FishFinder material);

For common interfaces and tools:

  • Social facilities have been reinforced, in particular the share updates part;
  • Document workflow has been mavenised and revised;
  • Report management has been revised;
  • Geo-related components have been revised, e.g. enhanced metadata visualisation;
  • Workspace facilities reinforced, e.g. bug fixing;
  • IRBootstrapper has been revised for creating forward index on TM collections;
  • Forward index portlet revised and mavenised;
  • FullText index mavenised;
  • Search portlet mavenised and revised;
  • UserManagement revised, bug fixing;
  • Searc results presentation has been revised, e.g. Visualisers for OAI_DC objects added to ResultsPortlet, ASL revised to deal with snippet and title properly;


No activity has been reported by ERCIM, FIN, UNESCO, CRIA and IRD.


  • Existing VREs have been maintained;
  • MyOcean and WoA have been indexed by the iMarine SDI
  • Release of a new version of the OAI-PMH TM Plugin;
  • Release of FIGIS TM Plugin;
  • Release of Results portlet 5.1.0, Search portlet 3.0.0, FullText index portlet 2.1.0, Forward index portlet 2.0.0, IRBoostrapper portlet 2.2.1, UsersManagement portlet 1.8.1 in gCube 2.15;

WP7: Enabling-technology Integration and Distribution

WP Leader: E-IIS - Participants: CNR, NKUA

During reporting period, integration of gCube release 2.14.0 took place (released in production on April the 20th). In total for this release 28 integration builds of gCube has been executed and 63 integration issues have been identified and managed through Trac system. Distribution site and News have been upgraded by NKUA for this release.

A major upgrade to the testing infrastructure has been performed by E-IIS:

  • all gHNs and services upgraded to the versions released in gCube 2.15.0
  • all nodes have been upgraded with java 7 jvm
  • testing portal has been reinstalled and updated to gCube 2.14.0 ( https://grids10.eng.it/)

E-IIS updated ETICS instance improving performance both in portal and during builds.

BTRT tool received two bug fixes for archived builds.


Nothing to report


  • gCube 2.14.0

WP8: iMarine Data e-Infrastructure Enabling Technology Development

WP Leader: CNR - Participants: NKUA, E-IIS, FAO

T8.1:

  • softwaregateway implementation has been modified in order to ensure that service version in a software resource profile is the default one (1.0.0).
  • Fixed a bug in deployer service that produced duplicate packages in the installed packages list.
  • Analyzed problem affecting the resource manager related to a wrong targeting of the recipient GHN for a webapp deployment. This problem has been verified to not occurr with the latest version of the resource manager.

Components released in gCube 2.15:

  • org.gcube.vre-management.softwaregateway
  • org.gcube.vre-management.softwaregateway-stubs

T8.2:

  • Data model for resource accounting has been defined.
  • Requirements for resource accounting portlets have been defined and passed to the developers team.
  • Accounting data model library has been developed, it will be supplied as a java library.
  • Development of the common accounting library to be used by accounting clients and server components isbeing developed.
  • Development on the accounting portlet has started and ongoing.
  • An analysis of the effort and technological requirements for integrating the login portal to the id-federation infrastructure has been performed.
  • SOA3 components have been deployed on the dev infrastructure. A new secure VOVirtual Organization; has been deployed on the development infrastructure.

T8.3:

  • Started development of workflow adaptor which handles the execution of workflows of jobs over a variety of processing infrastructures by first translating into a PE2ng plan.
  • Introduced another JDL JobType in order to execute DAGs of jobs capable to be submitted to either PE2NG or GRID.
  • First results of PE2ng's evaluation in the UMD infrastructure have been collected. It was noted that the implementation of the AquaMaps algorithm has to be refactored in order for the evaluation procedure to continue.
  • Some addition to operators have been made, in order to support separation of an input file into records or aggregation of records to a file.
  • Started implementation of the new workflow adaptor, which exploits the new operators.
  • WorkflowGridAdaptor has been refactored to correctly retrieve the output files of a jdl job, possibly consisting of more than one sub-tasks.

Components released in gCube 2.15:

  • org.gcube.execution.RRGCubeBridge.1-4-3
  • org.gcube.execution.WorkflowEngine.1-3-2
  • org.gcube.execution.WorkflowSearchAdaptor.1-3-0


none


none

WP9: Data Management Facilities Development

WP Leader: CNR - Participants: NKUA, CERN, TERRADUE, FAO

In T9.1:

A new instance of the SDMX Fusion Registry was installed in the development infrastructure. This instance will be used in order to test sdmx client libraries. The Species Product Discovery was enhanced and bug fixed.

A new MongoDB sharded cluster was deployed in the development environment, and bugs were fixed regarding an issue with remote connections. A new version of the storage-manager library was released.

In T9.2:

The Data Transfer Agent was enhanced with the implementation of Async Tree Transfers. The scheduling of the Tree Transfers from the Agent Scheduler service was implemented too. The scheduler portlet was enhanced to manage the novel usage of the Tree sources.

Furthermore, implementation activities were conducted on WPS-Hadoop. The IRD Tuna Atlas indicators were added as new legacy application and the FAO SPREAD stand-alone application was released. Finally, legacy application files were mavenized. Effort was also spent also in supporting the search for environmental parameters processing tools that could be useful to produce, for example, new NetCDF files.

In T9.3:

The development of the cube-manager and cube-manager-api components was completed. Few corrective actions were taken with respect to the original implementation. E.g. the tabular-query module for the tabular data system was introduced to create materialized views of datasets along with codelist levels.

The development of tabular-data-manager service was started adopting the jax-ws library, for import and export facilities. Finally, a new version of sdmx registry client libraries was implemented, which solves some bugs and usability issues.


No deviations to report.


  • Release of a new version of the storage-manager library.
  • Release of sdmx-registry-client(2.0.0), sdmx-registry-client-gcube(2.0.0) and sdmxsource-utils(1.0.0)
  • Release of a new wrapper library for sdmxsource beans (sdmxsource-deps 1.2.0)

WP10: Data Consumption Facilities Development

WP Leader: NKUA - Participants: CNR, FORTH, TERRADUE, FAO

In T10.1, the integration of Couchbase with Forward Index has finished and was released. Also, the data fusion operator implementation has been completed and released. The investigation on the design of Federated Search is ongoing. Lastly, a problem with the presentable fields of Open Search DataSource and a dependency problem caused at FullTextIndexNode have been fixed.

In T10.2, gCube Data Transformation Service is now able to produce fulltext and forward rowsets out of FIGIS-TM plugin collections, in order to be used for indexing purposes.

In T10.3, effort was placed in maps comparison functionality that allows discrepancy analysis among AquaMaps distributions at different time instants. This activity also included effort in layers information retrieval from GeoNetwork, which required polygonal representation of WFS features. Time Series analysis features of the Spectral Analysis facilities were enhanced. Lastly, algorithms for trends extraction from obis were added to Trendylyzer and a database schema for computations storage was created.

Furthermore, Interactive Layers was extended to show advanced visualisations within the layer. It now supports visualizations of timeseries filtered by time and geographic area for both WMS and WFS data.

In T10.4, xSearch-portlet design changed to more scalable, which allows the analysis of more hits as they are requested by the user. Moreover RDF properties of linked data were exploited in X-Search. Finally, a related paper that acknowledges the support of iMarine has been accepted for publication.


none


  • The following components have been released in gCube 2.15.0
    • Forward and Fulltext Index related
    • OpenSearch DataSource
    • Search related
  • gDTS support Fulltext and Forward rowsets creation out of FIGIS-TM plugin collections
  • A related paper that acknowledges the support of iMarine that has been accepted for publication is the following:
    • P. Fafalios and Y. Tzitzikas. X-ENS: Semantic Enrichment of Web Search Results at Real-Time. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval (demo paper), SIGIR 2013, Dublin, Ireland.

WP11: Data e-Infrastructures Integration and Interoperability Facilities Development

WP Leader: NKUA - Participants: CNR, CERN, E-IIS, FORTH, TERRADUE, FAO


During May, in T11.2, effort was spent on providing support for the integration of the clients based on fws stack, with common-security library. Also, the implementation of the OWS Context Java library began.

In T11.3, effort was spent on changing way the preseantable fields are picked on the query construction ASL Search Library. Also, Resource Registry integration with the FeatherWeight Stack has been started. Finally, a Client Library for ForwardTextIndex has been developed and released.


None


The core OWS Context library with the support for the object model Java XML binding.

The following components have been released in gCube 2.15.0:

  • ASL Search library 4.0.1
  • ASL Content library 5.0.1
  • ASL OAI_ORE 1.0.1
  • forwardindexnode-client-library 1.0.0

Progress and Achievements by WP and Beneficiary

WP1: Administrative and Financial Management Monthly Activity By Task and Beneficiary

WP2: Project Management Monthly Activity By Task and Beneficiary

WP3: Governance and Policy Development Monthly Activity By Task and Beneficiary

WP4: Communication, Dissemination and Training Monthly Activity By Task and Beneficiary

WP5: iMarine Data e-Infrastructure Deployment and Operation Monthly Activity By Task and Beneficiary

WP6: Virtual Research Environments Deployment and Operation Monthly Activity By Task and Beneficiary

WP7: Enabling-technology Integration and Distribution Monthly Activity By Task and Beneficiary

WP8: iMarine Data e-Infrastructure Enabling Technology Development Monthly Activity By Task and Beneficiary

WP9: Data Management Facilities Development Monthly Activity By Task and Beneficiary

WP10: Data Consumption Facilities Development Monthly Activity By Task and Beneficiary

WP11: Data e-Infrastructures Integration and Interoperability Facilities Development Monthly Activity By Task and Beneficiary

Personal tools