2012.09 WP10: Data Consumption Facilities Development Monthly Activity By Task and Beneficiary

From IMarine Wiki

Jump to: navigation, search

Contents

This WP10 Activity Report described the activities performed in September 2012 by Beneficiary and Task.

It is part of the Monthly Activity Report.

T10.1 Data Retrieval Facilities

NKUA Activities

The activities of NKUA included (i) further support and coordination of the integration of XSearch with gCube; (ii) the enhancement of the gCube Search System with ranking capabilities; (iii) minor bug fixing and enhancements; and (iv) in addressing issues in the integration of the gCube 2.10.0 minor release, including updated configurations into the release when necessary.

  • Regarding the integration of XSearch with gCube, collaboration with FORTH has continued with three conference calls, on 12/9, 17/9 and 25/9. The discussions helped towards addressing issues, minimizing delays and planning the activities for the 3rd TCOM meeting. The discussions in tickets #627 and #628 also initiated (i) the performance analysis and improvement activities of the ResultSetConsumer facility and the underlying gRS2 component, in the context of WP11 and WP9 respectively; and (ii) the activity of implementing unified ranking for the gCube Search System.
  • Bug fixes and enhancements include
    • The initialization of the PE2ng environment at the level of the Search System Service.
    • A fix of any (simple search) queries at the level of Lucene, so that the allIndexes field which corresponds to this functionality is not erroneously included in the constructed Lucene query.
    • Fixes in delete methods of Field and Searchable entities in Resource Registry.
  • Regarding the implementation of ranking functionality
    • In terms of development, a significant part of the work towards achieving this goal has been completed. The merge operator was enhanced and now provides a third mode of operation, in which the output set of records corresponding to results in a union of data sources is properly ranked provided that the input sets are ranked. Since data sources provide ranked results and the search system is designed in a way that joins are rare and practically non-existent in queries coming from the UI, the enhancement of the merge operator means that the Search System can now provide ranked results to the end user (including XSearch).
    • The query language has been extended with an extra rank clause, similar to the project clause, which can be set according to the desired functionality. In practice, the default mode will be the sort option which enables ranking. However, since presenting results per source can be useful for "browse" queries, this mode will still be able to be selected at the ASL level for queries of this type.
    • The activity resulted in modifications in the gCQLParser, SearchSystem and OperatorLibrary components. Also, in the WorkflowSearchAdaptor in WP8.
    • Next steps will be to also enhance the join operator and to integrate the new query structure at the ASL level.


None.


The following components have been released in gCube 2.10.0:

  • SearchSystem 3-1-0
  • SearchSystemService 2-0-2
  • SearchSystemServiceStubs 2-0-2
  • Operatorlibrary 1-1-0-1
  • OpenSearchLibrary 1-6-0
  • OpenSearchDataSource 1-6-0
  • OpenSearchDataSourceStubs 1-6-0
  • ResultsetGarbagecollector service archive 3-1-0-1
  • ResultsetClientLibrary servicearchive.3-1-1-1

FORTH Activities

none


none


none

Terradue Activities

The beneficiary should report here a summary of the activities performed in the reporting period


The beneficiary should report here major issues faced in the reporting period and the identified corrective actions, if any.


The beneficiary should report here a bullet list highlighting the main achievements of the reporting period

T10.2 Data Manipulation Facilities

NKUA Activities

Last month, NKUA focused on integration of Data Transformation related components with the minor release of gCube 2.10. Also, bug fixing and testing took place.


none


The following components have been released:

  • DataTransformationService
  • DataTransformationLibrary
  • DataTransformationHandlers
  • DataTransformationPrograms
  • WorkflowDTSAdaptor

CNR Activities

CNR concentrated on the finalization of the WPS-Hadoop experiments. These aimed to evaluate the performances of the WPS-H framework to the scopes of a distributed computational architecture with huge inputs. CNR attached the Storage Manager to the Resampling and Bathymetry algorithms for managing the inputs and outputs of the procedures.


no deviations to report.


  • Integration of the Storage Manager in the Resampling and Bathymetry algorithms

FAO Activities

The beneficiary should report here a summary of the activities performed in the reporting period


The beneficiary should report here major issues faced in the reporting period and the identified corrective actions, if any.


The beneficiary should report here a bullet list highlighting the main achievements of the reporting period

T10.3 Data Mining and Visualisation Facilities

CNR Activities

CNR concentrated on the implementation of the Statistical Manager, that will be release in gCube 2.11. The released has focused on different components. A the end of the month the Ecological Engine library included 6 algorithms for niche modeling, 3 algorithms for clustering, 3 evaluators and 7 transducers. A graphical interface has been implemented too along with a gCube service responsible for interrogating the Ecological Engine library.


no deviations to report.


  • Release of the Statistical Manager service
  • Release of the Ecological Engine library with an initial set of algorithms and methods
  • Release of the Statistical Manager portlet

NKUA Activities

The beneficiary should report here a summary of the activities performed in the reporting period


The beneficiary should report here major issues faced in the reporting period and the identified corrective actions, if any.


The beneficiary should report here a bullet list highlighting the main achievements of the reporting period

Terradue Activities

First part of this period was spent on delivering and documenting the tiffUploaderAlgorithm. After that, we used this time to a WPSClient, a very useful Java library Command Line utility to test all the WPS process delivered and next to come. The library provides the link between the EnvironmentExplorer Library and the WPS-hadoop and thus should be integrated in it.

FAO Activities

The beneficiary should report here a summary of the activities performed in the reporting period


The beneficiary should report here major issues faced in the reporting period and the identified corrective actions, if any.


The beneficiary should report here a bullet list highlighting the main achievements of the reporting period

T10.4 Semantic Data Analysis Facilities

FORTH Activities

During this period FORTH worked on making XSearch more robust and efficient. In particular FORTH identified that some of the problems are due to the configuration of TCPLocator (ticket #627). NKUA (WP10 leader) has been described the current approach followed by XSearch is memory-consuming (ticket #628) and provided some possible solutions to make it more efficient. FORTH agreed to investigate the process of “passing” the TCPLocator to the XSearch-service immediately after its creation and the service will “sleep” until the desired number of results has been reached (the default is 50).


none


none

FAO Activities

The beneficiary should report here a summary of the activities performed in the reporting period


The beneficiary should report here major issues faced in the reporting period and the identified corrective actions, if any.


The beneficiary should report here a bullet list highlighting the main achievements of the reporting period

Personal tools