2012.04 Activity Report

From IMarine Wiki

Jump to: navigation, search

Contents

This Activity Report captures the activities performed in April 2012.

This report is organized in two parts:

Progress and Achievements at WP level

WP1: Administrative and Financial Management

WP Leader: ERCIM - Participants: CNR

  • Help produce the draft of Management Report
  • Collecting effort report for M1-M5 (3 missing partners: UNESCO, CRIA, IRD)
  • Monitoring and submission of the due deliverables and milestone achievement statements
    • Submission: D11.1 - Integration and Interoperability Framework Definition
    • 20 milestones achieved up to date
  • Re-imbursement of advisors for Advisory Council meeting (FAO, Rome, March 19)
  • Attend SB and PEB call


None


The WP leader should report here a bullet list highlighting the main achievements of the reporting period

WP2: Project Management

WP Leader: CNR - Participants: NKUA, FAO

  • Monitoring and supervision of scientific activities and deliverables in the period;
  • Organization of monthly SB (4 April 2012) teleconference and review of the corresponding minutes prepared by ERCIM;
  • Organization of the monthly PEB on 2 April 2012 and preparation of the meeting minutes ;
  • Monitoring and replanning of the technical activities;
  • Representation of the project at Marine Board Forum (18 April, 2012, Brussels);
  • Preparation of the D2.1 Management Report.


No major deviation.


  • Monitoring and supervision of scientific activities and deliverables in the period;
  • Organization of monthly SB (4 April 2012) teleconference;
  • Monthly PEB on 2 April 2012;
  • Monitoring and replanning of the technical activities;
  • Establishment of liaisons with the Marine Board Forum community.

WP3: Governance and Policy Development

WP Leader: FAO - Participants: ERCIM, CNR, NKUA, FORTH, TERRADUE, FIN, UNESCO, CRIA, IRD

FAO

  • produced the draft report of the first iMarine Board meeting.
  • produced a first draft to outline the policy and governance model
  • continued development of SPREAD, where the aim is to consume TS objects with georeferences from an external JAVA client
  • initiated testing is SPREAD can use WPS/Hadoop to calculate new intersections
  • initiated work on the Codelist Manager; after the requirements collection, now existing tools (Vocbench) are evaluated

OBIS led the definition of the products and services expected in the biodiversity cluster.

IRD was instrumental in defining the contents of the semantics cluster, and continued adding resources to it's OpenSDMX repository.

Terradue led the definition of the products and services that can be developed in the geospatial cluster.

With FORTH, IRD and FAO work initiated on the development of a top-level ontology in the marine ecology domain. Other partners, such as OBIS, NKUA, and CNR, will be asked later to participate.

  • FORTH produced an outline for the production of such an ontology
  • IRD produced an intial set of requirements as expectations
  • OBIS reviewed the opportunity, and mentioned their interest given their involvement in DwC and TDWG

Reporting was scant, and not all partners are covered.


D3.1 is still in delay. Upon circulation of the Board Meeting report, it can be closed, as it describes the operations of this Board. It awaits completion of the cluster descriptions.


For the Board, the draft meeting report is completed, including rules and regulations, cluster descriptions, and a framework for EA-Cop policy and governance development are included.

An initiative to develop a top-level ontology for marine ecology was announced.

A next version of OpenSDMX is made available. The activity can now continue with the collaboration with other WP's; WP6 and WP9.

A OS project for CL management, that will enrich the existing D4S functionality, was announced.

WP4: Communication, Dissemination and Training

WP Leader: TRUST-IT - Participants: CNR, NKUA, FAO

The WP leader should report here a summary of the activities performed in the reporting period


The major issues faced in the reporting period and the identified corrective actions should be reported here


The WP leader should report here a bullet list highlighting the main achievements of the reporting period

WP5: iMarine Data e-Infrastructure Deployment and Operation

WP Leader: CERN - Participants: CNR, NKUA, FAO, FIN

CERN has coordinated and performed the infrastructure upgrade to gCube 2.8.1 wich comprised the upgrade of all nodes of the infra and the portal. In addition the D4Science.org portal (https://portal.d4science.org) has continued to be configured in order to give access to Ecosystem and gCubeApps VOs and their VREs.

CERN was also involved in the activity of the Maven integration in Etics, together with CNR, FAO and E-IIS. Several phone conferences haven been held with members of WP8.

The Accounting components have been extended to gather and show information regarding Aquamaps Service usage and WAR. In addition the Messaging Consumer service now integrates apache commons dpcp and pools libraries in order to enhance DB interactions. The components has been released in the gCube 2.9.0 release.

During the month the Dev Apache messaging broker has been moved to use the Kaha DB for persistent messages instead of the Mysql DB. The test is needed to understand the usage of Kaha DB in production. During the month the monitoring of the production infrastructure has continued, in particular the Service Availability test for the Information System Services has been re-enabled.


None


Infrastructure upgrade to gCube 2.8.1

Release of messaging components for gCube 2.9.0

WP6: Virtual Research Environments Deployment and Operation

WP Leader: CNR - Participants: ERCIM, NKUA, FAO, FIN, UNESCO, CRIA, IRD

The virtual research environments defined and deployed in the previous period have been maintained. Moreover, FAO collected feedback on their current scope, data-management facilities, and re-use in the wider EA-CoP 'ecosystem'.

Re Resources and Tool provision, during the reporting period the activity focused on the Species Discovery Service and on OpenSDMX. In particular, the Species Discovery Service GUI has been enhanced to properly deal with hundreds of results to display (paging has been added) and the feature for exporting discovered occurrence points in DarwinCore format has been added. A new version of OpenSDMX has been released (it supports compression of datasets, includes an enhanced version of the client capable to deal with compression and streaming).

A number of portlets has been revised/enhanced during the reporting period:

  • Resources Management Portlet;
  • Aquamaps (bug fixing);
  • Workspace portlet;
  • Report management and Template management (Integration with new Workspace);
  • TimeSeries (Bug fixing);
  • User Management portlet (released in 2.8.1);
  • Search (bug fixing);


D6.2 Applications and Tools Development Plan is not complete yet. Delay was due to the dependency among this deliverable and WP3 activities. A version has been produced and circulated by FAO on May 6th. This version is under review by CNR.


  • existing VREs have been maintained;
  • a new version of OpenSDMX has been released;
  • revised versions of some portlets have been produced, some of them have been released in 2.8.1;

WP7: Enabling-technology Integration and Distribution

WP Leader: E-IIS - Participants: CNR, NKUA

E-IIS coordinated the maintenance release gCube 2.8.1 in the first half of the month, running integration builds and deployment tests.

WP7 participants in collaboration with WP8 and FAO, CERN, CNR organizations, finalized the work for the support in Maven. In particular new features have been added to the Mavenizer and Maven Proxy to allow a) the generation and uploading of javadoc artifacts, b) the update of profile.xml files with component's Maven coordinates, c) the support of a new Maven repository (gcube-staging) in the release testing phase.

From the distribution point of view, E-IIS and CNR collaborated to implement the necessary tools to register integration builds outcomes in the Software Gateway service for distribution and dynamic deployment puropses.

Several changes have been applied to the ETICS configurations in order to support the execution of Mavenizer tools and the relase of new GCF.

Finally, documentation has been produced (E-IIS, CNR, CERN, FAO) on how to work with Maven in gCube and ETICS [1] [2].

Milestone MS32 have been achieved and documented by NKUA


  • release 2.9.0, initially announced for the last week of April, has been postponed to the first week of May to let WP7 to finalize work on Maven support
  • This month's T7.1 activities required more effort than originally planned. Some work planned in T8.1 has been postponed for the benefit of T7.1


  • SGClient 1.0.0 released
  • gCube 2.8.1 released
  • new gcube-staging Maven repository
  • maven-builder.1-0-0 released
  • MS32 "Make available the documentation for user, administrators, and community developers" achieved and documented

WP8: iMarine Data e-Infrastructure Enabling Technology Development

Most of WP8 activities were dedicated to provide tools, expertise, feedback and test cases to implement the new software integration and distribution procedure. Towards this goal, the enabling technology has been upgraded, the Software Gateway service has been revised, finalized and released, gCore has been refactored and enhanced in the resources management's section. In addition, T8.1 has contributed to the preparation of a guide about how to develop gCube Maven components.

Two new components has been prepared by T8.1, common-clients and common-gcore-clients, for enhancing gCube clients experience and oa third one, ghn-core-runtime, to support the new Maven-based development.

In the workflow area, T8.3 has been working towards the integration of PE2ng, more specifically the provider for the gCube environment which used by PE2ng, with the new Storage Manager service of gCube. Testing is currently in progress and communication has been established with WP9 in order for some issues to be clarified and to ensure correct usage. T8.3 has submitted tickets with some issues to fix in the PE2ng.

Finally, T8.3 has prepared the specifications for Workflow Management Facilities, MS35, due for this month. The milestone has been achieved and the specifications are available here.

Next plans:

  • complete D8.1
  • support to gCube Mavenization will likely take other two weeks in May
  • back on new foundations and management function in the second half of May
  • T8.3 to move on an implementation phase


Some activities planned in April for T8.1 and T8.4 have been postponed for the benefit of T7.1. D8.1 has been postponed to the second week of May.


  • the enabling technology has been upgraded to support the new software integration and distribution procedure implemented by WP7, and this includes the following releases:
  • release of Software Gateway service 1.0.0
  • release of SGClient 1.0.0
  • release of Deployer service 2.3.0
  • release of common-utils-encryption 1.0.1
  • release of gCore framework 1.4.0 (in collaboration with FAO)
  • release of legacy-wscore 1.4.0
  • release of Deployer service 2.3.0
  • release of Resource Manager service 1.1.0
  • release of gHN-Manager service 1.5.0
  • preparation of a guide about how to develop gCube Maven components
  • IS-Publisher, Deployer service, gCF and common-utils-encryption have been fully mavenized
  • release of IS-Publisher 2.2.0 for fixing ticket 254
  • release of common-clients 1.0.0
  • release of common-gcore-clients 1.0.0
  • release of ghn-core-runtime 1.0.0
  • release of WorkflowEngine service 1.2.1
  • Milestone MS35

WP9: Data Management Facilities Development

WP Leader: CNR - Participants: NKUA, CERN, TERRADUE, FAO

In T9.1:

  • FAO released the new tree-based access components in gCube 2.9.0
  • FAO has supported the development of tree-manager clients that are co-released in 2.9.0
  • CNR finished with the implementation of the CoL, Worms plugins for the Species Manager
  • CNR implemented an export procedure in Darwin Core format for species data

In T9.2:

  • CERN worked in the implementation of the first component for Data Transfer and released it in gCube 2.9.0
  • CERN worked on Etics to create the components configurations
  • CERN and NKUA prepared the wiki description of the milestone MS38 on Data transfer Specification
  • NKUA implemented the HTTP Broker that exposes the result set as an http endpoint. HTTP Broker has been developed as web servlet running on Apache Tomcat
  • Terradue finished the development operations of WPS-hadoop
  • Terradue created the related documentation on the g-cube wiki

In T9.3:

  • CNR solved some bugs on the Time Series application
  • CNR implemented a solution for securely exchanging data between Time Series-like applications and the RStudio IDE
  • CNR designed with the FAO team, a new streaming-based SDMX client


MS39 was delayed because of some technical aspects which have to be defined during the 2.9 gCube release. It is necessary to wait till the half of May.


  • Release of the new tree-based access components in gCube 2.9.0
  • Worms and CoL plugins
  • Darwin Core exporting function
  • Release of a first implementation of the Data Transfer components
  • Milestone MS38 accomplished
  • Implementation of the HTTP Broker for gCube Result Set
  • Implementation of the WPS-Hadoop solution
  • Secure interface to the RStudio IDE for information exchange
  • Design of a new SDMX client

WP10: Data Consumption Facilities Development

WP Leader: NKUA - Participants: CNR, FORTH, TERRADUE, FAO

In the direction of maintaining the quality of Data Retrieval facilities, work has been performed on resolving incidents reported in the production infrastructure. The incidents led to fixes in the Resource Registry set of components, as well as the Search System and the Full Text Index Lookup Service. All fixes were released in gCube 2.8.1. In addition, the development of enhancements for the Resource Registry was completed during this period, and testing of the components has been performed in the development infrastructure. Authoring and editing of deliverable D10.1, describing the specifications of the gCube Query Language has been completed.

In the area if Data Manipulation, the testing of the revised Data Transformation Service continued with test cases driven by demands from real use cases of the production environment. The tests were successful and the system was found to perform well, and execution times were proportional to the complexity of the tasks, verifying that the system is scalable. In addition, Milestone MS42, describing the specifications of Data Manipulation Facilities has been achieved.

Work related to Data Mining continued with experiments on running the Aquamaps algorithm on PE2ng. Performance charts and timing evaluation have been produced in order to trigger investigation by the related task in WP8.

In the context of Semantic Data Analysis, milestone MS45 has been produced. Furthermore, the final version of the paper “Web Searching with Entity Mining at Query Time” has been prepared. The paper, which describes the prototype of X-Search over FAO FIGIS and FLOD, was accepted and will be presented at 5th Informational Retrieval Facility Conference (IRF’2012).


The authoring and editing of deliverable D10.1 has been completed, but it was submitted for review with a delay.Effort was put to have the final document available before the end of the reporting period, in order to minimize actual delay and submit the document early in the next period.


  • Release of org.gcube.search.searchsystem.3-0-1 in gCube 2.8.1, fixing incident #402
  • Release of org.gcube.index-management.common-library.3-3-1 in gCube 2.8.1, fixing incident #404
  • Release of org.gcube.execution.RRGCubeBridge.1-2-2 in gCube 2.8.1, fixing incident #403
  • Completion of development and testing of Resource Registry enhancements.
  • Interaction of Statistical Manager with PE2ng was achieved and correctly managed
  • Performance Analysis of PE2ng was produced
  • Milestone MS42
  • Milestone MS45
  • IRF'2012 “Web Searching with Entity Mining at Query Time” paper was accepted

WP11: Data e-Infrastructures Integration and Interoperability Facilities Development

WP Leader: NKUA - Participants: CNR, CERN, E-IIS, FORTH, TERRADUE, FAO

During this month, work has progressed on both CL model and HTTP Front-End layer of the Integration and Interoperability framework. The architecture of the new version of the HTTP Front End framework has been designed and discussed with the WG members. The decisions for the framework re-structuring have been driven by the need for its extensibility and include the division of the monadical application of the monadical application (ASLHTTP) of the previous version into a set of smaller applications that can proliferate within the framework. An HTTP application in the new version of this layer, logically groups related functionality. From the existing implementation, the components derived are the following:

  • ASL_HTTP_InformationRetrieval: Aggregates mandated functionality to perform a gCube search listing
  • ASL_HTTP_ContentAccess: Aggregates functionality for accessing gCube content
  • ASL_HTTP_InfrastructureLogin: Aggregates functionality for logging in the system
  • Specifications Implementations: Implementations of HTTP Standards placed in separate application components

Moreover, a common utility component for Operations' Authentication, providing the functionality needed to authenticate the execution of an operation (integrating both session based authentication logic and anonymous access mode logic) has been implemented and is intended to be used by all components.

Work on CL model lead to re-editing and extending it towards:

  • addressing subtle interactions between replication and statefulness
  • refining accordingly binding and endpoint strategies
  • expanding accordingly Assumptions and Terminology with a clear description of replication and statefulness scenarios
  • completing the missing Section on Instance Properties
  • overall revising most of the existing test on proxy creation and lifetime

The work has been driven by intensive analysis of possible use cases and solutions. The work is now very close to completion and work on the remaining sections has been assigned. Work on the framework that supports model-compliant CL development has been planned and is due to start next. Preparation of D11.2 has also started. However, since gCube 2.9.0 has been delayed to May 4th, the deliveraby (bound) to such release will be completed in the second week of May.


None to report


  • Release of aslHttpInfrastructureLogin component that aggregates functionality for logging in the gCube system
  • Release of applicationSupportLayerHttpCompenent that is intended to be used by all components of the HTTP Front End layer and authenticate the execution of operations

Progress and Achievements by WP and Beneficiary

WP1: Administrative and Financial Management Monthly Activity By Task and Beneficiary

WP2: Project Management Monthly Activity By Task and Beneficiary

WP3: Governance and Policy Development Monthly Activity By Task and Beneficiary

WP4: Communication, Dissemination and Training Monthly Activity By Task and Beneficiary

WP5: iMarine Data e-Infrastructure Deployment and Operation Monthly Activity By Task and Beneficiary

WP6: Virtual Research Environments Deployment and Operation Monthly Activity By Task and Beneficiary

WP7: Enabling-technology Integration and Distribution Monthly Activity By Task and Beneficiary

WP8: iMarine Data e-Infrastructure Enabling Technology Development Monthly Activity By Task and Beneficiary

WP9: Data Management Facilities Development Monthly Activity By Task and Beneficiary

WP10: Data Consumption Facilities Development Monthly Activity By Task and Beneficiary

WP11: Data e-Infrastructures Integration and Interoperability Facilities Development Monthly Activity By Task and Beneficiary

Personal tools