WP2 - Data curation, ingestion and pre-processing

Guided by WP1 achievements, WP2 consists of four tasks and is creating real and synthesised (Data Fabrication Platform) industrial and cross-sectorial data, that will facilitate the validation phase of I-BiDaaS project. WP2 will also produce an interactive visualization toolkit for IT and non-IT Big Data experts.

 

  • Define I-BiDaaS data nature and format and prepare the respective datasets for simplified processing;
  • Deploy an end-to-end solution for data on-boarding;
  • Provide a mechanism for transforming and integrating both streaming and historical data produced by heterogeneous sources;
  • Fabricate synthetic and realistic big data so as to facilitate the development of the I-BiDaaS technologies, as well as the validation of the I-BiDaaS solution against industrial cross-sectorial experiments;
  • Create visualization tools and interfaces for both IT and non-IT experts for the realization of Big Data as a Self-Service.

In the context of this WP four reports and three prototypes will be produced (more than one version per type). The report related to the data format is confidential and all other will be publicly available.

This WP has produced 4 public and 1 confidential report according to the plan. The work carried out so far led to the definition of the datasets for the industrial use cases, an end-to-end solution for data on-boarding was deployed, synthetic data for several use cases was fabricated and the visualization tools and interfaces were created.