2012.06 WP9: Data Management Facilities Development Monthly Activity By Task and Beneficiary
From IMarine Wiki
This WP9 Activity Report described the activities performed in June 2012 by Beneficiary and Task.
It is part of the June Monthly Report.
T9.1 Data Access and Storage Facilities
FAO has begun the re-factorisation of the tree-manager-library to the new Application Framework developed in the context of WP11. FAO has also created WIKI documentation for the tree-manager-framework.
None to report.
None to report.
CNR team finished the development of the Synonyms Capability which allows users to get synonyms by id. Furthermore CNR team has started to work on Layer Capability in order to retrieve layers from external sources.
Bugs were fixed in Names Mapping and Classification Capability.
- Synonyms Capability implementation
- Bugs fixed
T9.2 Data Transfer Facilities
During the period CERN was involved in extending the Data Transfer Agent in order to implement asyncronous operations handling 532. For this purpose the Agent is now using a DB ( trough DataNucleus) to store the transfer information and 2 operations have been implemented to monitor and retrieve the transfer results.
NKUA has been working on improving the TCP transfer method performance of gRS2 as reported on the ticket #302. Some experiments were made in order to measure cases that needed customization such as FileFields which lead to some recommended solutions (such as uploading the file and passing the URL, compressing the whole file before sending/after receiving). The results can be found on BSCW.
Furthermore, NKUA has been working on providing the capability of transporting errors between consumers and producer in the gRS2 library, as reported on the ticket #526.
Also, the gRS2Broker has been updated in order to present the data in XML format instead of HTML format.
During this month, Terradue continued with WPSHadoop developers support, helping Roberto Cirillo and Emmanuel Blondel in their own experiments and Algorithm implementation. We also fixed some bugs reported by these contributors (tickets #482, #482 "bis" and #466 contribution).
WPS-hadoop adoption is growing with CNR and FAO developers evaluating the framework.
T9.3 Data Assessment, Harmonization and Certification Facilities
Tabular data is a common data type in 9.3 task (and others), therefore a reusable component for tabular data visualization and elaboration is under development by the CNR developers.
During the June month a first alpha version of the Tabular Data widget, the UI part of the component, has been released.
A description model has been produced in order to describe a custom table with detailed information about the columns (labels, editability, sorting, etc.) and the expected value types.
The UI component has been built in a way to abstract from the data provider:
- an interface for data source definition has been created: The Datasource is a DAO that provides an abstract interface to the data persistence. The Datasource is in charge to provide the widget with a table description and the table data. The table description is provided using the description model, the table data is returned using the JSON format. The Datasource mechanism lets the widget to work without a Tabular Data Library implementation.
- a JDBC data source has been provided: The JDBC Datasource retrieves the necessary information through a JDBC connection. The table description is retrieved from the database meta-information provided by the JDBC driver. The table data is retrieved throught SQL queries.
- In order to handle different SQL language used by different Database Management System an SQL-Dialect system has been provided. An SQL Dialect interface provides the necessary SQL statement for the JDBCDatasource operations. Each SQL Dialect is capable to declaring if it supports or not the specified DBMS product and version.
- In order to support tables with no primary key an auto-generate column mechanism has been provided.
A Tabular Data Client web application has been produced in order to show the Tabular Data Widget capabilities. The sample application is available in the svn repository.
A first integration of the Tabular Data Widget has been reached with the Statistical Manager Portlet. The Statical Manager Portlet is now capable to show the experiment results stored in a remote DB. During the integration no particular problems have been found, only small issues promptly fixed.
- first alpha of Tabular Data Widget
- first integration of Tabular Data Widget