2012.08 WP9: Data Management Facilities Development Monthly Activity By Task and Beneficiary
From IMarine Wiki
This WP9 Activity Report described the activities performed in August 2012 by Beneficiary and Task.
It is part of August 2012 Monthly Activity Report.
T9.1 Data Access and Storage Facilities
FAO has completed and then prepared the release of 7 components in the data-access framework for gCube 2.10.0, all major (tree-manager, tree-manager-stubs, streams, tree-repository, tree-manager-framework, tree-manager-library) except one minor (trees). Key changes have been propagated from streams, and the new error transport facilities in the new gRS. This has allowed simpler interface of the tree-manager itself, with impact on the tree-manager-library, the tree-manager-framework and the tree-repository. The tree-manager-library has been also re-factored to align with the new CL model, using the CL framework for the purpose. The tree-manager has lost its dependency on the tree-manager-sample-plugin, using mock plugins for testing in my-container. The full list of changes is reported in the changelogs of individual components.
None to report.
The release of the FAO components described above.
The beneficiary should report here a summary of the activities performed in the reporting period
The beneficiary should report here major issues faced in the reporting period and the identified corrective actions, if any.
The beneficiary should report here a bullet list highlighting the main achievements of the reporting period
In addition, the development of a filestorage component has been started for the gRS.
The filestorage component will provide a full abstraction to the clients by hiding the architecture of the storage machines and the protocols that are used from for the transfer. It is designed in order to be used as a general purpose library.
The activity reported is additional to the ones planned in the DoW.
None to report
T9.2 Data Transfer Facilities
A first version of the Data Transfer scheduler is under implementation. The components under implementation are:
- scheduler web service
- scheduler client library
- scheduler CLI
- scheduler IS interface
- scheduler DB interface
The architecture has been documented at:
The Agent Service has been also extended with some enhancements:
- configurable thread pool
- configurable transfer timeouts
- possibility to unzip archive files after the transfer.
all Agent and Scheduler components will be released in september in gCube 2.10.0
NKUA has been working on some bug fixes on the gRS2 and MadgikCommons components along with some more performance measurement tests.
Furthemore, gRS2Broker documentation has been written and is available here.
Release of gRS2 was delayed due to some bug fixes
gRS2Broker documentation has been written
The small task we have undertaken in this period was to study a way to automatically test the WPS-hadoop build in our continuous integration system, using a routine (it uses curl to post requests and manage the responses) to automatically test a dummy WPS-Hadoop Algorithm. Thanks to this we have identified the possibility to instead build a Java WPS Client to be integrated in the gCube libraries. We are proceeding with its implementation in September (very informally communicated to Gianpaolo via Skype on Septembre 5th).
T9.3 Data Assessment, Harmonization and Certification Facilities
Occurrence Reconciliation Service
Implementation of the occurrence reconciliation service architecture and logic. Design and implementation of a common library for remote executions handling based on JMX and ActiveMQ. Design of occurrence reconciliation service interaction policies with statistical service.
None to report.
- Initial implementation of the Occurrence Reconciliation Service.