2012.02 Activity Report
From IMarine Wiki
This Activity Report captures the activities performed in February 2012.
This report is organized in two parts:
- Progress and Achievements at the WP level reports a concise overview of the activities, issues and corrective actions, and major achievements produced by each WP Leader;
- Progress and Achievements by WP and Beneficiary reports a detailed description of the activities, issues and corrective actions, and major achievements produced by each beneficiary at the level of Project task.
Progress and Achievements at WP level
WP1: Administrative and Financial Management
WP Leader: ERCIM - Participants: CNR
The work performed in M4 includes:
- Monitoring and submission of the due deliverables and milestone achievement statements;
- Supporting travel, reimbursement guidelines and other logistics for participants coming to the 1st meeting of iMarine Board Advisory Council;
- Making analysis of effort used in the past 4 months;
- Several partners have still not uploaded their bi-monthly effort report for M1-2 & M3-4.
The project is on track in general with the effort used in the first 4 months versus plan
WP2: Project Management
WP Leader: CNR - Participants: NKUA, FAO
Scientific and technical activities were supervised and monitored. Several interactions were held with FAO to identify priorities and potential risks, especially concerning the activities related to CoP identified needs. The monthly PEB (6 February) and SB (7 Feb 2012) teleconferences were organised and minutes were prepared and distributed.
Nothing to report.
- The project activities were monitored;
- Monthly SB and PEB meetings were held;
- The Agenda of the next f2f TCom(14-16 March 2012) was further detailed.
WP3: Governance and Policy Development
WP Leader: FAO - Participants: ERCIM, CNR, NKUA, FORTH, TERRADUE, FIN, UNESCO, CRIA, IRD
In Task 3.1, the preparation of the upcoming iMarine Board was the main achievement:
- Discuss the agenda with relevant Board members
- Discuss availability and presence of Board members
In Task 3.2, 2 main documents are under preparation
- Data access and sharing policies
- Software sharing policies
These will be discussed at the iMarine Board meeting
In Task 3.3 several ongoing initiatives continued
- For SPREAD, the technical backbone for re-allocation across spatial areas was discussed. A JAVA component will be developed with the community
- For ICIS, the FAO component for
- For SDMX, FAO participated to several meeting (in person and by phone) e.g. to discuss options to develop components to generate SDMX from existing data-structures, such as FishFrame.
- For FLOD, enriching the contents of the KB and activating the WP Data Access client were undertaken
- OBIS has started reviewing their needs for e.g. GBIF data access, OBIS data manipulation with a Postgresql db, and synchronization with other OBIS data providers. A discussion with FIN on the need for species attributes services (names, biotope, distribution) has started, but requires further input from FIN.
The following deliverable is in delay:
- D3.1 EA-CoP Operation, M3, FAO
- The reason for delay of D3.1 is that the operational procedures and guidelines have to be adopted by the iMarine Board at its March meeting in Rome.
The WP leader should report here a bullet list highlighting the main achievements of the reporting period
WP4: Communication, Dissemination and Training
WP Leader: TRUST-IT - Participants: CNR, NKUA, FAO
- Web conference system, iMarine Board Channel area population & TCom event registration on-line;
- iMarine flyer generation;
- Dissemination & communication content for 3rd Marine Board Forum, FAO mailing list and contact DB;
- News items on external web sites (e.g. ISTGW);
- Generation of synergy simulations with iMarine partners and board members (internal to Trust to-date);
- iMarine Visibility and Content - Feb 2012 Highlights to all partners to stimulate content;
- Paper generated by NKUA and accepted for 21st International Conference on World Wide Web, WWW 2012, Lyon, France, April 2012.
None to report
- iMarine presented at 3 events (IRCDL - Bari, Italy on 9 feb'12, Sciencesoft - Geneva, Switzerland & CloudscapeIV - Brussels, Belgium on 24th Feb '12;
- iMarine Blue Technology Presentation accepted for 18th April 2012 meeting;
WP5: iMarine Data e-Infrastructure Deployment and Operation
WP Leader: CERN - Participants: CNR, NKUA, FAO, FIN
CERN planned and performed the production infrastructure upgrade to gCube 2.7.3 which included the :
- Implementation of Software Patches
- Implementation of GHN patches
- Remote Upgrade of the nodes.
In order to perform the upgrade CNR , NKUA and FAO site managers were also involved.
In addition CERN performed the dismissal of the Arts-Humanties VOVirtual Organization; and gMan VRE in collaboration with CNR.
Regarding the activity related to Infrastructure Policy definition, CERN together with CNR decided to restructured the wiki page describing the policies  into Global, SLA and Participation sections, and to split among them the policy definition tasks. CERN therefore started the definition of Software SLAs and Participation policies.
CNR installed a new VM (geoserver2.d4science.research-infrastructures.eu) where a Tomcat with geoserver instance have been deployed and as well installed a geonetwork instance in geoserver.d4science-ii.research-infrastructures.eu
NKUA upgraded successfully all the gLite middleware nodes (glite 3.2 and glite 3.1), that it provides, to UMD middleware nodes (UMD 1)
Two activities started during this period regarding T5.3 by CERN:
- Porting of the Portal Accouting Portlet to GXT, activity to be performed in junction with some enhancement
- Integration of new type of Portal Accounting information ( Report, Template, War management and Document Workflow).
In addition CERN had a phone conference with ENG in order to understand the integration of the Accounting and Billing Service developed for VENUS-C in gCube. The integration activity will be presented at the forthcoming TCOM Meeting in Rome.
Lastly in the context of T5.3 a new gCube Monitoring application version has been released by CNR with 2 new features: removal and "add to scope" operations for the new type of resource runtime resource.
An hardware fault occurred at CNR site on a SAN for a power outage. This failure has deactivated part of the CNR site. The gHN containers and the services running on those machines were restarted.
Regarding T5.3, CERN realized that the code changes in order to use GXT instead of GWT-EXT are deep, basically the portlet has to be rewritten. Therefore it has been decided to apply the enhancement and the extensions directly to the GWT-EXT version of the Portal Accounting Portlet.
- Infrastructure upgrade to gCube 2.7.3;
- Arts-Humanities VOVirtual Organization; dismissal;
- Started definition of Infrastructure policies;
- Successfully upgraded all NKUA site glite services to UMD 1;
- Integration of new type of Portal Accounting information in the gCube Accounting system (to be released in 2.9.0);
- The gCube Monitoring application supports new features for RuntimeResources.
WP6: Virtual Research Environments Deployment and Operation
WP Leader: CNR - Participants: ERCIM, NKUA, FAO, FIN, UNESCO, CRIA, IRD
The activities performed in this reporting period focused on (a) the maintenance of the existing VREs, (b) resources and tools provision, (c) the enhancement of the portal technology;
For what concerns "VREs operation", the 13 VREs defined and deployed in the previous period (inherited by the D4Science-II project) have been maintained. Actually, during this reporting period one of the existing VREs, gMan, has been dismissed. There are no pending requests for additional VREs stemming from the EA-CoP through the WP3.
For what concerns "resources and tools provision", a species data sets discovery portlet (relying on Species Manager service for seamless access to species data sources) has been designed and implemented. This facility supports species data sets discovery based on criteria ranging from name-based (eg common name, scientific name) to time-based and area-based (latitude and longitude) ones.
For what concerns "common interfaces and tools", the entire set of gCube portlets has been revised and released to make them compliant with the GWT and LifeRay latest versions. Moreover, "guided tour" facilities has been added to the following portlets: gCube Template Portlet, gCube Report Portlet, gCube Workflow Manager Portlet, and gCube Workflow Templates Portlet. Besides this, a 'Runtime Resource Creation' portlet has been developed and released.
- D6.2 "Applications and Tools Development Plan" (Lead beneficiary FAO, due date M3 a.k.a. January 2012) has not been produced. The draft initially produced by FAO is under revision. CNR will support the finalisation of this deliverable in the next days;
- The reason for delay of D6.2 is that planning of Applications and Tools require Work Plans, which can only be made once several options are balanced. These require more information than available at M3.
- This was somewhat corrected by establishing framework Work Plans and Component descriptions, however, these do not comprise a replacement of concrete work plans involving several WP's in a concerted manner.
- The gMan VRE has been dismissed;
- A first prototype of the Species Data Sets discovery portlet has been released;
- The entire set of gCube portlets has been revised and released to make them compliant with the new versions of GWT and LifeRay;
- Four portlets have been equipped with the "guided tour" facilities;
- A Runtime Resource Creation portlet has been released;
WP7: Enabling-technology Integration and Distribution
WP Leader: E-IIS - Participants: CNR, NKUA In the reporting period, Work Package's participants worked for release and distributed two gCube releases: gCube 2.7.3 and gCube 2.8.0 (still in the release process). Release 2.8.0 ships several updates of existing software (gHN-distribution 3.2.6, search system 3.0.0, aquamaps 3.1.1, Liferay 6.0.6, GWT 2.4.0) and some new components (gcube-guidedtour-widget, ecological-engine, workflow-portlets)
E-IIS installed the new (still minimal) gCube testing infrastructure. This achievement allows execution of automatic packages certification and deploymnet test. First
Activity on the support for maven in gCube continued with two main achievements: a) first release of maven-builder and the first maven native component (
common-utils-encryption) in gCube 2.8.0, and b) the design, jointly with task T8.1, of a new Software Gateway component that interfaces with gCube Maven Repository for providing installable software within a gCube infrastructure and an extension of mavenizer to allow a more complete mavenization of legacy components compatible with the Software Gateway
The gCube distribution site has been updated providing access to artifacts of gCube 2.7.3 release.
- the guide to the usage of maven in gCube is still under development. However, a presentation on this topic will be held at the next TCOM meeting on 16th of March
- gCube 2.7.3
- start of integration and test of gCube 2.8.0
- gHN 3.2.6
- portal-framework.1-1-0 (shipping Liferay 6.0.6, GWT 2.4.0)
- installation of testing infrastrucutre
- designed extended version of mavenizer to address Software Gateway requirements
WP8: iMarine Data e-Infrastructure Enabling Technology Development
WP Leader: CNR - Participants: NKUA, E-IIS, FAO
The major outcome of the task was the production of the milestones MS33 (by T8.1 and T8.4) and partially MS34 (by T8.2). Such milestones wrap up and give external visibility to the work done so far on the evolution of the foundation technologies (MS33) and security mechanisms (MS34). Both of them gave the opportunity to draw a line and prepare the ground for the next implementation steps on these areas.
In the context of T8.1:
- interactions with T7.1 led CNR to the design of a new service,
Software Gateway, to interface a cluster of Maven Repositories. This is a direct consequence of the decision that gCube components will not be any longer stored in a custom repository but in more standard (from the distribution point of view) Maven Repositories. Software Gateway will replace the Software Repository service once the gCube mavenization process will be concluded in WP7;
- the feedback received from T6.3 called CNR for changes in the Runtime Resources definition and implementation. Afterwards, the work on this point has been finalized and the related component (
common-resources) released as part of gCube 2.8.0;
- FAO has worked towards reducing the deployment and dependency requirements for clients of gCube services. In doing so, FAO is addressing a requirement for minimality raised by the work on Java clients in WP11. The distribution and the libraries are available in our Maven repositories and will be soon configured for integration in Etics.
In T8.2, E-IIS (apart for the work on MS34) dedicated the reporting period to design a flexible Policy Oriented Security Module. This module will represent a strong improvement of the previous D4Science Authorization module.
T8.3 held a NKUA had a conference call with CNR in order to understand the needs for parallelization of the algorithms invlolved in iMarine scientific scenarios and to understand what the needs are for evolving
PE2ng and its model of resources and operation. As a first step towards achieving the goal for algorithm parallelization, NKUA is investigating the RainyCloud approach which is used currently in Aquamaps and will develop a PE2ng adaptor for the latter.
Finally, T8.4 worked on writing an early "formal" draft of the second generation of the Resource Model, with a particular focus on the Software Resource modeling. The resulting classification resources will be a great input for T8.1 towards the implementation of the new technological foundations.
New requirements on Runtime Resource called for a little redesign and implementation on
- milestone MS33
common-utils-encryption1.0 released as part of gCube 2.8.0
common-resources1.4 released as part of gCube 2.8.0
Software Gateway's design and early prototyping
Software Gateway's interface definition
- a minimal client distribution,
ghn-client-distro, which pairs previously related libs, to exhaust all the required client dependencies
- a set of infrastructure-specific libraries (
- agreement on future evolutions of the PE2ng at WP level
- an early draft of the second generation of the Resource Model
WP9: Data Management Facilities Development
WP Leader: CNR - Participants: NKUA, CERN, TERRADUE, FAO
- FAO has completed a prototype of the SPARQL plugin for the Tree Manager services. Based on a Jena backend, the prototype supports arbitrary tree-based lookup and queries of RDF sources that expose a standard SPARQL query interface.
- CNR completed a prototype of the Species Manager Service, along with the implementation of some related plugins (Obis, Gbif, Catalogue of life). The service is under testing and the prototype version is running on the development infrastructure. It can be accessed using the species discovery portlet on devportal.
- CERN analyzed the crucial points of the integration of FTS2/FTS3 with respect to gCube Data Transfer and a first architecture for the gCube Data Transfer. Furthermore they helped in bug discovery on the Storage Manager.
- NKUA worked on incorporating the novel gCube Result Set (gRS2) into the new gCube Data Transfer Service. The new features involve Multiple transfer protocols and http support. The URL Resolution Library, which is part of the core of the gRS2, has been released in gCube 2.8.0.
- Terradue activity regarded the integration of the Hadoop framework behind the OGC WPS implementation of 52 North. A demo has been setup on Terradue's development environment with the implementation of a resampling WPS process.
- CNR integrated the new GIS Viewer application in the ICIS\Time Series VRE, along with the connection to a Geo Network based architecture, which manages a distributed installation of GeoServers.
no deviations to report.
- Milestone MS37 Data Access and Storage Facilities Specification
- production of a SPARQL plugin for the Tree Manager service
- implementation of the Species Service with related plugins for some official datasources
- analysis of data transfer softwares FTS2/FTS3
- incorporation of gRS2 in gCube supporting multiple transfer protocols and http
- integration of hadoop with WPS server
- introduction of Gis Viewer and GeoNetwork for storing and retrieving geospatial data
WP10: Data Consumption Facilities Development
WP Leader: NKUA - Participants: CNR, FORTH, TERRADUE, FAO
In the context of Data Retrieval, milestone MS41 has been produced, describing the specifications of Data Retrieval Facilities. The specifications include key points in the fields of architecture, deployment scenarios and use cases of the target area. This specifications will contribute towards further evolving data retrieval facilities in the project and will facilitate the external visibility of the software artifacts and services. In addition, a new version of the gCube Search System incorporating query caching and result snippet capabilities which were implemented during the previous periods was released in gCube 2.8.0. The Searh Operator Library employed by the gCube Search System was also enhanced to support parameterizable capacities for the underlying buffers used by gRS2 with the aim to enable the Search System execution planner to be more adaptive against the number of collocated operator executions. In the area of enhancing Data Retrieval Facilities with semantic capabilities, the development of a prototype application which uses the gCube Search System as its back end is ongoing, and a conference call was organized to discuss and clarify a number of points regarding this approach.
Progress in the direction of providing Data Manipulation Facilities is continuing with the implementation of the Data Transformation Service under the revised design which was finalized during the previous period. In particular, the core elements of the new implementation are now available and the next step will be the combination of the latter into a functional PE2ng execution plan. The elements which have been implemented are namely the Data Source, Transformation and Merger Operators.
In the context of Data Mining and Visualization, the Statistical Manager Service was enriched by the implementation of new features and the enhancement of existing ones. The Classification Quality and Discrepancy Analysis capabilities of the service have been enhanced and the novel concept of Habitat Representatives Score has been applied to the experiments conducted during the previous period, with a slight change to the method of computing the overall score in order to improve the quality of the results. Finally, Principal Component Analysis was implemented as a core functionality of HRS.
Work regarding Semantic Data Analysis was mainly concentrated in the investigation of ways to exploit semantic web formats for publishing advanced search results (i.e. clusters, mined entities). A web conference has been organized with participants in WP10 and WP11 to discuss several issues in the architecture and implementation of these services, currently referred to as X-Search. In this meeting it was agreed that X-Search can function on top of the gCube Search System in order to exploit the full set of its features. Furthermore, ways of identifying whether contents are textual and of retrieving the actual contents of a hit have been discussed. In parallel, a configuration of architectural components that can instantiate a retrieval mechanism of Fisheries documental resources leveraging X-Search and the Fisheries Linked Data resource FLOD is being investigated upon.
MS41 has been achieved with a delay of 2 working days but is included in the present report as most work toward its achievement took place during the current and previous periods.
- Milestone MS41
- Release of new versions of gCube Search System and relevant subsystems including new features targeting performance and adaptability
WP11: Data e-Infrastructures Integration and Interoperability Facilities Development
WP Leader: NKUA - Participants: CNR, CERN, E-IIS, FORTH, TERRADUE, FAO
During this period of the project, work in WP is mostly around the definition and finalization of the objectives and actions that will take place within each identified working group. Great progress has been achieved in the context of the CL working group and the designation of an implementation model for Java Client APIs that mediate access system services.
Members have now been assigned to all the working groups described during the previous period. Discussions have been instantiated within each working group focusing on the definition of the main principles governing the design of the related framework layers. The detailed organization of activities within each working group has driven the first deliveratble of the workpackage. Moreover, it strengthens the interaction with other "technology development" workpackages that will adopt the Integration and Interoperability concepts generated. Part of the activities focused on the analysis of the current Application Services Layer components to lead to evolution needs towards integration and interoperability, while the design model of the CL layer of the framework has been of main concern and was prioritised within the discussions of the respective working group.
In the context of the Application Service layer framework, inspecting the adoption and development of ASL has been identified as the most deficient area and consequently as a main point of interest within the WP objectives. The HTTP API framework has been revised in terms of the current architecture and design leading to identification of needs for extensions, control over use and development as well as rules governing the compliance to specifications and the adoption of standard data interchange formats (JSON).
In the context of the CL framework layer, the work done for clarifying the scope of the activities lead to the definition of the cross-cutting concerns for the design of the framework and the necessity for a distinguishment between a design model and a development model. A charter for the working group has been defined and a first draf of the development model for Java client APIs has been produced. The draft already details:
- the motivations, scope, assumptions that underlie the model, with corresponding terminology;
- the proxy pattern that characterizes its design;
- the lifetime of service proxies;
- the direct and discovery operation modes of service proxies;
- the caching strategy required for service proxies that operate in discovery mode;
- a discussion on the design scope for the API of service proxies with an example that illustrate possibilities;
- a model of failures and a corresponding approach to failure handling
Finally, as a result of the use cases analysis for the Xsearch services that will function on top of gCube Search system, the conditions for the use of the OpenSearch implementation for gCube have been described transformed into requirements.
Tasks T11.2 and T11.3 provide input for the bottom-up approach of the framework definition process, according to the methodology defined in T11.1. Therefore, the timeline of these tasks has been aligned to T11.1 and the activities on the Data Management and Data consumption APIs will be intensified in the next period of the project.
- A first draft of a model for Java client API that mediate access system services has been produced. The current draft can be found here and will migrated to the gCube wiki as soon as it reaches sufficiently stability and coverage.
- Completion of D11.1 and submission for review