2012.10 Activity Report
From IMarine Wiki
This Activity Report captures the activities performed in October 2012:
This report is organized in two parts:
- Progress and Achievements at the WP level reports a concise overview of the activities, issues and corrective actions, and major achievements produced by each WP Leader;
- Progress and Achievements by WP and Beneficiary reports a detailed description of the activities, issues and corrective actions, and major achievements produced by each beneficiary at the level of Project task.
Progress and Achievements at WP level
WP1: Administrative and Financial Management
WP Leader: ERCIM - Participants: CNR
- 1st Review participation
- 1st Periodic Report submitted
- 2nd advance payment completed for: FORTH, E-IIS, UNESCO, IRD, FAO
- 1st Periodic Report submission
WP2: Project Management
WP Leader: CNR - Participants: NKUA, FAO
- Participation as iMarine delegate in the RDA Workshop (1-2 October, Washington);
- Active participation in the General Assembly (3 October 2012);
- Organization and chairing of the “Marine Knowledge All Projects Meeting”(11-12 October in Brussels);
- Participation as iMarine delegate in the ScienceSoft meeting (15 October, CERN)
- Contribution and editing of the first Periodic Report;
- Organization and chairing the review Rehearsal Meeting (28-29 October);
- Finalization of the September activity report and preparation of the template for the October one;
- Organization and chairing of the monthly SB (10 October);
- Organization and chairing of the 3rd TCom (1-3 October);
- Participation in several telcos dedicated to plan technological progresses and agree on corrective actions (16 October (how to support URI), 16 October (WP8); 23 October (WP10) ; 23 October (WP9); 25 October (CNR-Forth);
- Participation in a CNR-FAO SDMX f2f meeting dedicated to plan activities for the next months (31 October, Rome);
- Day-by-day monitoring of the project activities.
None to report.
- General Assembly (3 October 2012);
- “Marine Knowledge All Projects Meeting”(11-12 October in Brussels);
- First Periodic Report;
- Rehearsal Meeting (28-29 October);
- September activity report;
- Several meetings dedicated to plan technological progresses and agree on corrective actions;
- Monthly SB (10 October);
- 3rd TCom (1-3 October).
WP3: Governance and Policy Development
WP Leader: FAO - Participants: ERCIM, CNR, NKUA, FORTH, TERRADUE, FIN, UNESCO, CRIA, IRD
In task 3.1; Mediation
- Conduct and follow up to the Board2 meeting, collection feedback on topics such as Governance, data Access and Sharing policies
- Arrange a preliminary agreement on the use and exploitation of a Species Fact Sheets VRE
- Meet with other EU projects at the EU Marine Meeting
- Prepare for the review
- Promote iMarine at e.g. the OBIS steering committe meeting and for FishFinder group.
In task 3.2; Governance
- progress on iMarine Data Access and Sharing Policies
- Validation started with FIN. ICIS validation was completed. FCPPS validation resulted in a preliminary findings. Validation of other componenets is under way.
In task 3.3; Harmonization
- Work continued on SPREAD, now also becoming a WP6 activity, as functionality if added to iMarine geospatial data management
- Work continued on FLOD, now also calling on effort from WP10.4; where semantic technologies are developed in unison with FORTH and IRD; the TLO and preparing XSearch support
- A consultant was identified to work on Code list management. (Cotrix)
- SDMX activities are technically on hold until this person becomes available, several meetings were held to align CNR and FAO activities.
The major issues faced in the reporting period and the identified corrective actions should be reported here
MS18 Validation Reports was produced.
WP4: Communication, Dissemination and Training
WP Leader: TRUST-IT - Participants: CNR, NKUA, FAO
iMarine Poster session at the Geo-Seas International Workshop, University College Cork, Ireland, 9 & 10 October 2012 : Scientific Poster presentation & Bookmarks distribution
iMarine Board experts contacted to complete the expert page on the web channel
Mobile app finalization: comments coming from the iMarine Board members have been integrated. Social networks connections (Twitter and LiunkedIN) added, new functionalities for an easy and friendly navigation have been integrated.
Development of the Web application for DG MARE
First period (M1 – M10) review preparation: partners input collection and presentation finalisation
iMarine calendar printing and distribution to partners
Filming of 5 video interviews at the Rome Meeting: Nicolas Bailly, Fishbase Information & Research Group Inc. (FIN), Board Deputy Chair; Neil Holdsworth, International Council for the Exploration of the Sea (ICES); Ward Appeltans, UNESCO & Ocean Biogeographic Information System (OBIS), Anton Ellenbroek, FAO, Board Secretary, Pasquale Pagano, CNR, iMarien Technical Director
Editing & circulation of iMarine October 2012 Newsletter
Telephone interview with Paolo Galli, University of Milan – Bicocca: Magoodhoo & Milan – Worlds Apart?
Contact established with the Marine Knowledge Gate 1.0.
Organization of the “Marine Knowledge All Projetcs” meeting , 11-12 Oct 2012, Brussels: iMarine together with some of the major marine-related initiatives in Europe (CSA Ocean, EMODNET, EuroMarine, ASSEMBLE, EMBRC) has leveraged its network of contacts and co-organised the meeting “Marine Knowledge: All projects meeting”. The overarching objective of the meeting was to present the activities carried out by each project and their achievements in order to share common future guidelines and drive forward joint efforts
Generation & circulation to 15 press contacts of the iMarine press release on the iMarine first period achievements: The iMarine e-infrastructure: integrating technologies to unlock knowledge on marine living resources
Revision of the Science Gateway look and feel
iMarine channel regular updates and upload of the two new applications: Scalable data mining & Documents Workflow
FLOD webinar: promotion of the first iMarine webinar: The Semantic Puzzle: The Fisheries Linked Open Data project – data harmonization with semantic technologies both on the webchannel and on slideshare
The major issues faced in the reporting period and the identified corrective actions should be reported here
During the October iMarine Board meeting the iMarine mobile application has been presented. It has received positive comments from all the experts. Francky Callewaert, European Commission, DG MARE asked to develop a similar web application (with the same functionalities of the mobile one) to be integrated in the DG MARE portal. The web application will be release by November 2012.
iMarine outcomes from the “Marine Knowledge: All projects meeting”: Next steps will be focused on coordinated and concrete efforts in terms of e-infrastructures interoperability, joint training initiatives and support of marine communities emerging needs.
iMarine disseminated at the Geo-Seas International Workshop leveraging on the synergy established with Geo-Seas
Publication on the iMarine channel of the video-interview with Ward Appeltans, UNESCO & Ocean Biogeographic Information System (OBIS)
Publication of iMarine information on the Marine Knowledge Marine Knowledge Gate 1.0. http://www.kg.eurocean.org/proj.jsp?load=40549
WP5: iMarine Data e-Infrastructure Deployment and Operation
WP Leader: CERN - Participants: CNR, NKUA, FAO, FIN
CERN was involved in the installation and validation of the gCube release 2.10 and 2.11 in the pre-production environment. In parellel it has prepared and coordinated the installation of the gCube 2.11 release in production by:
- documenting the wiki pages documenting the upgrade procedure
- writing upgrading scripts
- performing remote Services upgrade on the infra
A new VRE for Statistical Data Analysis has been also deployed.
The NKUA executed the installation of the gCube 2.11 release in production, on nodes that are provided by NKUA. Additionally has been keeping up and running the provided nodes hosting GHNs, UMD and the gcube wiki.
Successfully upgraded 2 nodes hosted in FAO
Some investigation have been performed by CERN to understand how integrate the Nagios web interface inside a Liferay Portal. Portletbridge-portlet-1.1.2 has been tested but still should be checked how to integrate Nagios authentication.
- CNR No activity reported despite the known contribution in the WP.
- FIN server still not available.
New VRE for Statistical Data Analysis has been deployed
Infrastructure and portal upgrade to gCube 2.11
WP6: Virtual Research Environments Deployment and Operation
WP Leader: CNR - Participants: ERCIM, NKUA, FAO, FIN, UNESCO, CRIA, IRD
The previously deployed Virtual Research Environments have been maintained in operation (e.g. requests for subscriptions have been evaluated and either approved or rejected).
For resources and tools provision:
- 15 collections has been integrated in the iMarine infrastructure via the OAI-PMH protocol;
- IRD has been working on geospatial cluster to deliver WFS from Postgres/Postgis database and to collect processing in order to set up WPS;
For common interfaces and tools:
- Collections Explorer portlet has been revised to rely on the new TreeManager Service;
- Resource Management Portlet has been revised to integrate the new Create Activation Record wizard to trigger the TreeManager Service for creating new collections;
- Workspace Portlet has been revised to include the Messaging System, a new messaging widget, allowing multiple users and file attachments;.
- Development of Social Networking facilities continued, e.g. two portlets are currently under testing phase, ShareUpdates and NewsFeed;
- Species Products Discovery Portlet has been revised for including an about for each data source and for reinforcing the view details panel with capabilities for searching linked data sources;
- Occurrence Management Portlet has been designed and developed. This portlet offer the facilities for importing and operating on occurrence points (stored in csv files) produced via the Species Discovery or AquaMaps application.
- IRBootstrapper portlet is enhanced to support the indexing of collections coming from TreeManager.
- In the context of providing VRE functionality to the end users a native android application has been developed to provide search facilities over the gCube system and the available VREs;
- Some issues have been identified during the indexing phase of the newly created collections. This prevents the fully integration and exploitation of such data sources. The issue is documented in #488.
- D6.3 Virtual Research Environments Activity Report has been completed;
- 15 new collections has been integrated in the iMarine infrastructure;
- new/revised portlets have been released in gCube 2.11 (Collection Explorer, Species Products Discovery, Workspace, Resource Management);
WP7: Enabling-technology Integration and Distribution
WP Leader: E-IIS - Participants: CNR, NKUA
- E-IIS finalized integration of gCube 2.10.0 in the first half of this period
- E-IIS updated BTRT tool in order to show alerts and errors coming from SoftwareGateway registration. It makes much more simple to understand SoftwareGateway registration issues.
- Starting from October the 19th gCube 2.11.0 has been integrated. Integration is still running at the end of the reporting period and will be finalized in November. As usual, testing infrastructure has been updated and deployment tests have been executed on it.
- NKUA updated the distribution site with release's news and packages for releases 2.10.0 and 2.11.0
- CNR upgraded gHN bundle released in gCube 2.11.0
Nothing to report
- gCube 2.10.0 released
WP8: iMarine Data e-Infrastructure Enabling Technology Development
WP Leader: CNR - Participants: NKUA, E-IIS, FAO
CNR delivered an enhanced version of the Resource Manager, Deployer, and SoftwareGateway software components.
ghn-client-runtime<code> 2.0.0 and <code>ghn-client-notification 1.0.0, two components that enable clients to invoke gCube services under a fully embedded and optimised gHN stack.
ghn-client-runtime 2.0.0 also optimises the required stack, greatly reducing the size of the embedded stack (before, the stack was defined by
ghn-core-runtime, which was created for service-side use, hence includes components that are not needed client-side)
In the context of the architecture to support HTTP URI resolution, than spans WP8, WP9, and WP11,
common-uri is a WP8 component that provides cross-service support for minting HTTP URI without hard-coded dependencies on given authorities (servers) and without explicit dependencies on scope.
The support of Mavenization of the build and integration infrastructure also makes a step forward with the development of a gCube BOM component that will control the version of direct and indirect component dependencies and alert developers well ahead of production of clashes with library versions included in the distribution. FAO ha proved that this is possible also for gCube services (not only library), exemplifying the solution for the Tree Manager service. The rolllout of this solution is now under consideration in collaboration with ENG and CERN.
FAO has introduced in
common-scope 1.1.0 facilities to transparently reconcile thread pools with scope propagation requirements.
ENG completed (but not yet delivers) the development of the most of GCube Security Handler. Minor integration issues have still to be solved.
FAO lunched a first well-defined round of tests in order to evaluate the performance of PE2ng. This leads to the fix of few bugs in Node selection policies. Also, WorkflowSearchAdaptor has been refactored
Nothing to report
The deployment of REST services within the architecture is, to date, limited to Virtual Platforms. The tools available for this have been proved to assume 3rd-party services, rather than full gCube components released with the rest of system. FAO has opened a ticket about these limitations (https://issue.imarine.research-infrastructures.eu/ticket/804) and a discussion with CNR has followed on ways to go beyond the initial design assumption of deployment tools.
- Resource Manager,
- gHN Client Runtime
- gHN Client Notification
- Common Scope
WP9: Data Management Facilities Development
FAO presented a plan for the integration of Tree Manager in gCube.
FAO supported CNR in the development of the Tree Manager plugin for the Species Discovery Service.
FAO released "trees 1.1.0", a component which supports URI generation for the trees returned by the Tree Manager Service.
CNR improved the data transfer facilities between Species Discovery Service and its plugins.
CNR released a new version of the Storage Manager using "volatile" ares to store temporary files.
CNR developed a plugin for the Tree Manager services that defines and maintains tree views of biodiversity data sources exposed by the Species Discovery Service.
CERN developed a first version of the Data Transfer Scheduler portlet. They also fixed the Data Transfer Scheduler and Agent.
NKUA fixed bugs in TCPConnectionManager and enhanced the RandomReader and RecordReaderDelegate APIs.
Terradue concluded the delivery of the WPS-hadoop and WPS Client libraries.
CNR setup a study meeting with FAO about OpenSDMX. They also made enhancements on the Tabular Data Widget to support several datasource factories.
No deviations to report.
- Release of the trees 1.1.0 component
- Improvements in the Species Discovery Service
- Development of a first version of the Data Transfer Scheduler portlet
- Delivery of the WPS-Hadoop library and client
- Study meeting with FAO about OpenSDMX
- Enhancements in the Tabular Data Widget
WP10: Data Consumption Facilities Development
WP Leader: NKUA - Participants: CNR, FORTH, TERRADUE, FAO
During this period, progress has been noticed in all involving tasks. In the context of T10.1, the activities include (i) further support and coordination of the integration of XSearch with gCube and (ii) minor bug fixing and enhancements in Resource Registry: Regarding the integration of XSearch with gCube, collaboration continued with some private discussions. These discussions helped towards addressing issues in the use of gRS2 component and gCube search extensibility.
In T10.2, work has been done on the integration of Species Products Discovery with Data Transformation in order to export out of the source the desired rowsets for indexing procedure. This task includes the creation of a new DataSource, general enough for every Tree Manager plugin, that models the specific source along with the corresponding XSLTs that are used during transformation process.
Also in T10.3, algorithms and methods included in the Ecological Engine Library have been enhanced. Moreover the Statistical Manager has been refactored for future maintenance purposes. Also, a distributed architecture has been adopted for the incoming requests to the SM and extensively tested.
Furthermore, first steps have been made towards analysis of R Spatial as a mean to provide geo-spatial data visualization approaches for raw and aggregated data.
Finally, T10.4 focused on making the XSearch components (XSearch-portlet, XSearch-service) more efficient. In this approach the XSearch-portlet establishes a connection with XSearch-service, starts sending results through that communication channel and triggers XSearch-service to start retrieving them. The service is then waiting for the desired number of results and then start analyzing them. Regarding the XSearch-service there was a change in the way it is initialized and configured. In particular the service was updated to download at deployment time the required configuration files. This requires no post activities for making the service properly runnable.
- Production of D10.4 iMarine Data Consumption Software
- Release of Data Transformation components in gCube 2.11.0
- Release of Resource Registry components in gCube 2.11.0
- Bug fixes of the SM service, portlet and library
- Enrichment of the algorithms
- Tests on D4Science via GenericWorker
- Tests on local computations by simultaneous users
- Release of XSearch components in gCube 2.11.0
WP11: Data e-Infrastructures Integration and Interoperability Facilities Development
WP Leader: NKUA - Participants: CNR, CERN, E-IIS, FORTH, TERRADUE, FAO
During October, work has progressed in all tasks of WP11. In T11.1, the security part of the framework has been accomplished within the common-security library, whose implementation is now complete. Its integration to the system is about to be tested together with the rest of SOA3 Framework.
Common-gcore-service has changed towards eliminating dependencies on GCUBEScopeManager in favor of the ScopeProvider, following the deprecation of the former in the upcoming gCF 1.6.0. Moreover, a fix has been applied to the library, that prevented correct query formulation for stateful services.
In T11.2, an architecture was presented for the HTTP URI resolution of data which is otherwise accessible through ad-hoc SOAP interfaces in the data management layer of the system. This activity has been launched in the broader context of integration issues within WP9, and will provide gCube clients with uniform, standard interface for data lookups within the infrastructure, pushing service-ward the resolution of technology integration and interoperability issues. A lookup intefrace to data, based on HTTP URIs is in fact a first basic step towards the REST-ification of existing SOAP APIs. The discussions on the architecture have been followed by its entire implementation and release in gCube 2.11 through a set of components that fall in the scope of WP8, WP9, and WP11. Moreover, in the scope of WP11, a resolver for HTTP URIs for trees, named tree-uri-resolver, has been released as a JAX-RS REST service that is deployable on Virtual Platforms based on servlet containers, such as Tomcat.
In T11.3, the HTTP Framework - based components that fall under the Data Consumption area have been thoroughly tested through the integration with the new Android application for gCube search. The application uses ASL HTTP Framework to allow the mobile device user to log in the system (using LDAP credentials), log in an Infrastructure Scope, view the searchable collections of a VRE and search over one or more collections via 'simple' gCube search mode. The access of the mobile client to the system resources is based on the session mechanism managed within ASL framework, and the data interchange with the framework uses both JSON and XML supported APIs depending on the app needs. Finally, D11.3 for Application Programming Interface Software has been produced, including the list of software released up to M10, describing their novelties and evolution, and documenting their utility towards meeting the interoperability objectives.
The deployment of REST services within the architecture is, to date, limited to Virtual Platforms. The tools available for this have been proved to assume 3rd-party services, rather than full gCube components released with the rest of system. A relative discussion has been initiated on the possible ways to go beyond the initial design assumption of deployment tools.
- Production of D11.3: Application Programming Interface Software
- Integration of ASL HTTP Framework with new Android application for gCube search