2012.05 WP8: iMarine Data e-Infrastructure Enabling Technology Development Monthly Activity By Task and Beneficiary

From IMarine Wiki

Jump to: navigation, search

Contents

This WP8 Activity Report described the activities performed in May 2012 by Beneficiary and Task.

It is part of the Monthly Activity Report.

T8.1 iMarine Data e-Infrastructure Enabling-technology Development

CNR Activities

As expected, the integration and release of gCube 2.9.0 (the first ever made according to the new distribution procedure based on Maven Repositories) called for tuning activities on the enabling components released last month.

Following some integration issues raised by T7.1, the Deployer service has been internally refactored. Its original design did not allow native management of software not wrapped in a Software Archive. However, because of the new distribution procedure, virtually any software available in a Maven Repository is now eligible for being deployed in the Data e-Infrastructure. Thus, a new clean approach has been designed and implemented to support extensible and pluggable deployment mechanisms. These new features have been deeply exploited in the deployment tests performed as part of the release process of gCube 2.9.0.

The Software Gateway service has been tuned according to the directions raised by the integration phase. The report returned following the software registration has been enriched in order to provide the information required by the Mavenizer and the gCube distribution site. The interface of the service has been slightly changed following a different approach proposed by T7.1.

A new feature has been added to the maven-service-plugin to manage WSDLs defined in multiple XML namespaces.

As part of another thread of features, VREModeler and Resource Manager services are planned to offer a new simplified way to create VREs. In particular, Resource Manager will soon become a multi-scoped service able to manage multiple VREs with a single instance of the service(actually an instance per each scope must be deployed). Implementation is under way and it's scheduled to be completed in the next month.

CNR was also responsible for the production of D8.1 (delayed from last month). The deliverable was edited, reviewed, adjusted and officially released in May.

Finally, a significant part of the development infrastructure has been upgraded to gCube 2.9.0 granting to all developers the possibility to exploit the latest features of the enabling technology as early as possible.


None to report.


  • D8.1
  • Tuning of the distribution and deployment components
  • Proven effectiveness of the enabling components in the mavenized gCube release
  • Software Gateway developer's guide

FAO Activities

FAO has supported the evolution of the build infrastructure towards the adoption of Maven for the latest release of gCube. In particular, FAO has provided supervision to WP7 members and individual developers on the use Maven in local projects and in Etics builds. FAO has also started to work on the next version of gCore, 1.4.1 adding bridging support with new foundational libraries, particularly common-scope. FAO is continuing work on the new Resource Model in view of its presentation in D8.4 due by the end of June.


None to report.


None to report.

T8.2 iMarine Data e-Infrastructure Policy-oriented Security Facilities

E-IIS Activities

The collection of requirements and design activities has been completed at the end of April. A detailed workplan with effort and timing has been presented and shared to the WP. According to the workplan, the implementation of SOA3 framework is scheduled to be completed by the middle of November. The plan is still under discussion within the WP8 members because of the impact it will have on the system. The major point of the discussion is about the possibility to change the plan in order to have AuthN implemented firstly and then AuthZ will follow. In the meantime, tickets for the original workplan were submitted and accepted. Currently no final decision have been taken, yet.

ENG has started the implementation work basing on the first version of workplan, i.e. the extended Authorization module supporting policies based on multiple attributes and context based attributes. The work is currently in progress. ENG is also working in parallel on a new UserManagement module based on LDAP, which will be used for the Authentication process. The release is planned for the first days of June.


None to report


  • The first version of the workplan

T8.3 Workflow Management Facilities

NKUA Activities

During this period, NKUA has completed the integration of PE2ng with the new Storage Manager facility of gCube. For this purpose, the storage system provider used by PE2ng in the context of gCube has been refactored and tested. The completion of this activity addresses one of the reasons behind the performance issues reported in #263. The new version of the corresponding component has been released in gCube 2.9.0.

In addition, further investigation of the issues reported in #263 revealed that the matchmaking strategy followed by PE2ng was not optimal, as in the case of JDL DAG jobs the Information System was queried one time for each node selection task that should be performed. This strategy was identified as the reason behind the long submission times reported in the aforementioned ticket. In particular, the submission time increased in proportion with the number of nodes involved in the submitted task. Given that each query to the Information System retrieves all hosting nodes in the infrastructure which satisfy a set of requirements and that most of JDL DAG tasks targeting the infrastructure will contain sub-tasks containing identical requirements, a matchmaking preparation phase has been added to the plan generation procedure. This phase consists of gathering the common set of requirements among the JDL sub-tasks and retrieving and retaining the matching nodes from the information system. The prefetched set of matching nodes is used to select nodes for the sub-tasks which do not have additional requirements. Additional IS queries are submitted only when the sub-tasks specify requirements disjoint to the common set. This strategy leads to constant matchmaking computational costs for the majority of JDL DAG tasks. This activity was monitored in ticket #293. This activity resulted in a new versions for (i) the Workflow Engine library; and (ii) for the Environment Provider library, in order to enable retrieval of the set of all matching nodes without node selection. Both components have been released in gCube 2.9.0.

The following enhancements and fixes have been made:

  • To address one of the enhancements requested in #263, the Execution Progress Event structure of the Execution Engine library has been enriched with additional information. In particular, clients can now retrieve the logical name, host name and port of nodes which have completed execution. The changes performed have also been propagated to the Workflow Engine Service in order to be exposed to clients, as well as to the sample client of PE2ng.
  • In case of execution failure, information about the node from where the problem originates are included in the error.
  • A defect which caused output files originating from different nodes of JDL DAG jobs to be ovewritten in case the FTP storage option is enabled was fixed.

The above activities were monitored in tickets #294 and #295. The enhancements and fixes reported were released in gCube 2.9.0.

Finally, during this period NKUA compiled a list with all objectives achieved and software components released by T8.3 during the first six months of the project and contributed it as part of deliverable D8.1: iMarine Data Infrastructure Enabling Software.


The beneficiary should report here major issues faced in the reporting period and the identified corrective actions, if any.


The following components have been released in gCube 2.9.0:

  • org.gcube.execution.EnvironmentProvider.1-2-0
  • org.gcube.execution.GCubeEnvironmentProvider.1-3-0
  • org.gcube.execution.ExecutionEngine.1-1-5
  • org.gcube.execution.WorkflowEngine.1-2-1
  • org.gcube.execution.WorkflowEngineService.1-2-1
  • org.gcube.execution.WorkflowEngineService.Stubs.1-2-1
  • org.gcube.execution.WorkflowEngineServiceClient.1-2-1

FAO Activities

The beneficiary should report here a summary of the activities performed in the reporting period


The beneficiary should report here major issues faced in the reporting period and the identified corrective actions, if any.


The beneficiary should report here a bullet list highlighting the main achievements of the reporting period

T8.4 Resource Model

FAO and CNR had a conference call late in May to resume the work on the 2nd generation of the gCube Resource Model. The analysis was made taking into consideration the deliverable D8.4 due to M8.

Starting from this version of the Resource Model, gCube will take a more modular and abstract view over its management functions. Functions will be described solely by their APIs, and different implementations of the same functions (provided by T8.1) may exist within the system, over time or even concurrently.

June is scheduled to be a very busy month for T8.4 with the formalization of the model, the most common resource's types and their related interfaces and the release of D8.4.

FAO Activities

Preparation of the TOC for D8.4.


None to report.


None to report.

CNR Activities

Contributed to the layered design of the new Model.


None to report.


None to report.

NKUA Activities

None to report.


None to report.


None to report.

Personal tools