2012.07 WP9: Data Management Facilities Development Monthly Activity By Task and Beneficiary

From IMarine Wiki

Jump to: navigation, search


This WP9 Activity Report described the activities performed in July 2012 by Beneficiary and Task.

It is part of July Monthly Activity Report.

T9.1 Data Access and Storage Facilities

FAO Activities

FAO has been working on new versions of most components in the tree-based access subsystem. The changes have been mostly induced by a new version of the gRS2 result set libraries which includes advanced facilities for failure transmission and handling. The changes have reflected into a new version of the common-streams library (2.0.0) and different APIs for the tree-manager-service (2.0.0). The tree-manager-library and the tree-repository have changed accordingly, with the former being also refactored for compliance with new CL framework. the work is well under way and should be completed by end of next month.

None to report.

None to report.

CNR Activities

Species Discovery Service

The CNR activity focused on introducing a cache to speed up the response time of the Species Discovery Service.

When a user sends a query, the service caches the result. Afterwards, when a new query is sent, the server gives them the cached result rather than re-querying, thus speeding up response times and potentially reducing the server resources required for generating and serving a request. LFU algorithm is used for memory management.

The expiration policy removes entities from the cache that are used the least.

SPQL Parser

The species discovery service has been provided with a query language: SPQL the SPecies Query Language. In order to support the language a parser has been created. The parser has been built using the ANTLR parser generator. The language has been defined using the ANTLR grammar definition language. The language supports Scientific Name and Common Name projections, data source selection, date and bound filter specification and finally return type declaration.


  • A cache for Species Discovery Service
  • Bugs fixed
  • A SPQL language parser

T9.2 Data Transfer Facilities

CERN Activities

The Data Transfer Scheduler

  • Design of Data Transfer Scheduler Service

An initial architecture has been created. The description of the main functions and libraries can be found in gCube wiki Data_Transfer_Scheduler

  • Data Transfer Scheduler DB Library

- Using a previous interface, moved from Hibernate to DataNucleus
- The Data Transfer DB Manager is almost completed. It contains the main methods for storing, getting and setting. Nevertheless probably there will be some adds in the future when integrated with the service.
- Integration with the service has already started.
- Configuration in ETICS.

  • Data Transfer Agent and Library

- The Service has been enhanced with async transfers handling , and method to retrieve transfer status and transfer results, where the async transfer activity is persisted on a local DB.

- Agent Library has been adapted accordingly.

- Performance tests have been started.

  • Design of Data Transfer Scheduler Service
  • Data Transfer Scheduler DB Library

NKUA Activities

NKUA has completed the developement of providing the capability for fault transportation between consumers and producer in the gRS2 library, as reported on the ticket #526. The fault transportation capabilities along with the TCP performance capabalities that we have been working on, will be released in the next minor release.

Furthermore, the components gRS2 and MadgikCommons has been turned into Maven projects.



Terradue Activities

No activity to report on T9.2 in July.

T9.3 Data Assessment, Harmonization and Certification Facilities

CNR Activities

Time Series Environment

The CNR is planning a complete refactoring of the Time Series Environment in order to address the requirements inherited from D4Science II project and the new ones from the I-Marine track system. The new requirements include the support of SPREAD system.We are also analysing the report produced in the ICIS validation process in order to prioritize the requirements.

The TS Portlet will use the new GXT framework in order to be compatible with the last browsers version. The TS Service will be made more modular in order to be reused in other infrastructure service.


Time Series Environment refactoring plan.

Personal tools