31.10.2012 SDMX

From IMarine Wiki

Jump to: navigation, search

Contents

Agenda

Time: Wednesday October 31, 2012, FAO Headquarter, Rome

  • FishStatJ and SDMX integration (Luigi Fortunati, Fabrizio Sibeni)
  • data.fao.org (Erik van Ingen, Dario Fabbri, Giorgio Sorbara, Luigi Fortunati)
  • Fusion Registry (Erik van Ingen, Luigi Fortunati)
  • Eurostat Registry (Erik van Ingen, Luigi Fortunati)
  • SdmxSource (Erik van Ingen, Luigi Fortunati)
  • OpenSDMX (Erik van Ingen, Luigi Fortunati)
  • Cotrix (Erik van Ingen, Luigi Fortunati)
  • Datastructure design (Erik van Ingen, Luigi Fortunati)

Participants

CNR: Luigi Fortunati;

FAO: Erik van Ingen, Fabrizio Sibeni, Dario Fabbri, Giorgio Sorbara;

Main Discussions and Actions

FishStatJ and SDMX integration

Fabrizio Sibeni has shown FishStatJ application features and the desired future SDMX integration in FishstatJ (figis/visitor). FishStatJ should allow in a future version the ingestion of data in SDMX format. We discussed FishStatJ information model by making comparisons with SDMX. One problem regarding the ingestion of SDMX structures regards multiple code (or any structure) IDs, which seems to not be supported in current SDMX specifications. Fabrizio explained that future modification of SDMX standard specification should probably take place in the future in order to fix this rather common aspect of data representation.

data.fao.org

Erik van Ingen introduced Luigi to Dario Fabbri and Giorgio Sorbara. Dario is responsible for the SDMX dissemination function in data.fao.org. This function runs upon Olap4J/Mondrian. The dataset generation starts with MDX, the multidimensional query language. We had a short discussion about the development of SDMX technologies and adoption of SdmxSource library. As MedataTechnology position on SdmxSource library licensing and future. We agreed that it is important to check SdmxSource licensing and future compatibility with new versions of SDMX standards.

Fusion Registry

Fusion Registry is an SDMX registry developed by MetadataTechnology, free to use, but not open source. We discussed possible integration of this component in the D4Science infrastructure. Fusion Registry can provide the sdmx registry interface required by CNR (Structural Metadata and Reference Data/Metadata subscription/querying). Therefore the registry can be supported by an SDMX Data Repository implementation for exposing data/structures. Up to now OpenSDMX provides Structural metadata and Reference data query capabilities.

Eurostat Registry

The Eurostat SDMX Registry could be a valid alternative of the MetadataTechnology Registry. We could evaluate its future adoption if licensing with the metadatechnology product does not suit our needs. We agreed that Erik would send Eurostat contacts to Luigi in order to enable him to do an investigation of their Registry. Most probably this Registry is using also SdmxSource.

SdmxSource

SdmxSource is a library developed by MetadataTechnology which helps with the management of SDMX artifacts. It provides a version agnostic API to generate and write SDMX artifacts. It can be integrated in current development version of OpenSDMX in order to add SDMX version abstraction, thus allowing management of a broader set of SDMX documents. SdmxSource also allows easy generation of data document according to the SDMX compact format. We tried to outline possible architecture models. Further investigation of SdmxSource library is required in order to understand how it is possible to deal with parameterized queries (i.e. PeriodFilter, KeyFilter) and what could be the performance issues, if any.

OpenSDMX

When SdmxSource will be published officially, OpenSDMX will replace low level modules with the SdmxSource alternatives. OpenSDMX as such will be a container of adapters. Most important adapter is the Olap4j Mondrian adapter, used in data.fao.org. The implementation of OpenSDMX as a Registry is far from complete, it does not implement for instance submission of data and structural metadata, datasources, publish/subscribe mechanism. FAO has not implemented this because it is yet focused only on data dissemination.

Cotrix

Cotrix will support the lifecycle of Codelists. It’s definition of a codelist is a supersedes the model of SDMX. Cotrix will publish a codelist in a SDMX Registry (local SDMX Registry or distributed).

Datastructure design

It could be interesting to use iMarine infrastructure as a collaboration platform to design and harmonise datastructures. This could have been useful in projects like SEIF.

Agreed Actions Summary

  • Luigi will check if the criticalities emerged with the integration of SDMX data model with FishStatJ can also be experienced with CNR data representation model.
  • Erik will investigate on SdmxSource, Fusion registry licensing and future compatibilty with new versions of SDMX standard(done, no response yet).
  • Erik will contact Bengt-Ake Lindtblad from Eurostat in order to ask for their implementation of a Registry(done, no reponse yet).
  • If MetadataTechnology licensing or future support of their products do not meet CNR and FAO needs, Eurostat Registry will be considered as an SDMX Registry. Erik will support Luigi with understanding the latter.
  • Luigi will discuss architectures and further development of SDMX system capabilities in gCube platform with Pasquale Pagano and (probably) Lucio Lelii in order to outline a roadmap and the development of related components.
  • Luigi will start testing SdmxSource library capabilities, looking for integration into an implementation of SDMX data source.
Personal tools