WP3 - Batch processing innovative technologies for rapidly increasing historical data

WP3 consists of one of the basic technological packages that are offered by the I-BiDaaS solution, namely Batch Processing Technologies and is realized by four tasks. As historical industrial data are increasing rapidly and fast, data owners have now the opportunity to rein the data exposition by adopting I-BiDaaS package.


  • Introduce innovative, distributed big data analytics architectures based on fully asynchronous Innovative Distributed Solvers (IDS) that will efficiently solve corresponding optimization problems;
  • Provide application layer tools (e.g. COMPSs, Hecuba and Data Fabrication Platform) to enable both developers and not-IT users to manage batch data by comprising storage of the master dataset and computing arbitrary views using Big Data analysis;
  • “Automatic ML-based creation” of data fabrication rules;
  • Offer a scalable and reliable distributed data management, with an easy-to-use interface and independent of the underlying storage system. The latter will be part of the distributed large-scale layer of the I-BiDaaS solution.

In the context of this WP three reports and three prototypes will be produced (more than one version per type). All material will be publicly available.

This WP has produced 2 public reports to the plan. The work carried out so far led to scientific advances in the context data analytics algorithms, setting up an initial prototype of the batch platform, and an innovative way to visualize high-dimensional data to explore anomalies.