The detailed description of the I-BiDaaS architecture (the six work flow steps together with layered system and User interface) is presented below:
Step 1: The data (real or synthetic produced via data fabrication tool) are ingested into the batch processing and the streaming analytics modules via the Universal Message broker.
Step 2: The analytic modules perform analytics on the ingested streaming data, also referencing historic information where necessary, to identify business patterns that have happened or are about to happen.
Step 3 (I-BiDaaS innovation): The batch and real time analytic results are fed to the advanced visualization tools. An innovation is that part of the analytics can be offloaded to the parallel GPU-accelerated engine to further speed-up the execution of streaming analytics.
Step 4: The collected data can be stored in Hecuba, that uses the Apache Cassandra as a back-end, which can then be processed by the COMPSs pool of distributed machine learning algorithms.
Step 5 (I-BiDaaS innovation): The correlations produced by the analysis can fed back to the data fabrication platform, to be used for training and help building rules that will be used for future data generation purposes.
Step 6 (I-BiDaaS innovation): The real-time processing module feeds the batch-processing module with inputs that enable periodic refinements of models used in machine learning methods. The proposed solution goes beyond the traditional lambda architecture in terms of interleaving batch and stream processing.
I-BiDaaS User Interface: The proposed solution offers also a multi-purpose interface which can be used by different categories of users.It provides different levels of abstractions such as Programming API (providing access to every level of our software stack); Domain language (providing access to the application layer); Pre-defined analytics (providing simplest form to non-IT users to easily combine and multiplex with the desired data sources, to form a data processing pipeline).
I-BiDaaS layered system: The I-BiDaaS architecture composes of three principal layers: the infrastructure layer (providing the actual underlying storage and processing infrastructure of the I-BiDaaS solution); the distributed large-scale layer (controling the orchestration and management of the underlying physical computational and storage infrastructure) and application layer (refering to the architecture aspects and components that are involved in the actual workflow of extracting actionable knowledge from the big data, starting from data preparation, to analytics, to delivering analytics results for supporting decision making).