2012.03 WP9: Data Management Facilities Development Monthly Activity By Task and Beneficiary

[[Category:Restricted_Page]]
{|align=right
||__TOC__
|}

This WP9 Activity Report described the activities performed in '''''March 2012''''' by Beneficiary and Task.

It is part of the [[2012.03_Activity_Report|March 2012 Activity Report]].

== T9.1 Data Access and Storage Facilities ==

=== FAO Activities ===

FAO has re-designed and implemented a new version of the Stream API that supersedes the old one, and it has refactored all the components of the Tree-based access subsystem to make use of the new API. The API has been designed in response to requirements coming from WP11 relating to a proper integration with gRS2, and it offers a number of improvements over the older versions. Its documentation is currently available [http://dlib.sns.it/bscw/bscw.cgi/d244671/Streams.pdf here], and it will be migrated to the wiki in due course.

FAO has also re-aligned the tree-manager-library to the recommendations evolving in the CL model of WP11. Finally, FAO has prepared presentations for the TCOM and initiated discussion for the design options for infrastructure-based data access in the evolution of the CodeList Manager.

<xh4> Deviations and Corrective Actions </xh4>
none to report.

<xh4> Achievements </xh4>
- new Streams API

=== CNR Activities ===
CNR activity focused on the enhancement of the OBIS and GBIF integration, along with the exploration of new data sources. Obis and GBIF plugins were completed introducing the retrieval of occurrence points and specimens. The WORMS plugin was started and first interaction tests were successful.

<xh4> Deviations and Corrective Actions </xh4>
no deviations to report.

<xh4> Achievements </xh4>
* OBIS plugin was terminated
* GBIF plugin was terminated
* WORMS integration was started
* Occurrence points and specimens are retrieved successfully

== T9.2 Data Transfer Facilities ==

=== CERN Activities ===
During this period the architecture of the gCube Data transfer has been finalized.  CERN has decided to work on a "pure" gcube solution for Data Transfer instead of extending and integrating  the EMI FTS. Therefore CERN has continued to work on the Data transfer agent implementation, starting to work as well on the implementation of the Data Transfer Scheduler in particular on the DB interface.

The status of the activities has been presented during both the iMarine TCOM and the EGI CF 2012 in Munich

<xh4> Deviations and Corrective Actions </xh4>
None

<xh4> Achievements </xh4>
Finalized gCube Data Transfer Architecture.

Talk on gCube Data Transfer at EGI CF 2012

=== NKUA Activities ===

NKUA has been working on incorporating gRS2 into the new gCube Data Transfer Service. The following task is in progress in order to achieve this goal:
Supporting HTTP as gRS2 transfer method: 

*The development phase of the point to point proxies for gRS2 has finished. The messages that are transmitted from both parts are in XML format, which provides a structured way to represent the data in a format that is both human-readable and machine-readable. The locator that is produced by an http writer is a simple url with http as a scheme, thus it is accessible and readable from every side that supports HTTP post requests.

After stress testing the HTTP transfer method and comparing it to the TCP method the results led to some improvements on the later. The whole transfer time is reduced dramatically (at least 3 times faster) due to better synchronization between the reader and writer sides. 
Some more improvements and optimizations, such as data compression and better synchronization, are being examined in order to achieve even better transfer time in both methods. Also JSON format is examined as an alternative data format on the HTTP transfer method.

<xh4> Deviations and Corrective Actions </xh4> 
none

<xh4> Achievements </xh4>
The development phase of the point to point proxies for gRS2 HTTP support has finished. The performance benchmark results show that the HTTP method is about 10 faster than the previous TCP method.

=== Terradue Activities ===
No actions were reported about activities.

== T9.3 Data Assessment, Harmonization and Certification Facilities ==

=== CNR Activities ===
An enhancement of Time Series Environment and R-Studio Environment integration has been done. A Web Application has been created in order to support the environment parameters retrieving.

Incident ticket have been fixed for the Time Series Environment.

<xh4> Deviations and Corrective Actions </xh4>
None.

<xh4> Achievements </xh4>
* Is not more necessary to install a R-Studio Environment per VO and per portal.