I-BiDaaS is a self-service solution, aiming to empower users to easily utilize and interact with big data technologies by designing, building and demonstrating a unified framework that significantly increases the speed of data analysis while coping with the rate of data asset growth and facilitates cross-domain data-flow towards a thriving data-driven EU economy.
I-BiDaaS will shift the power balance within an organisation, increase efficiency, reduce costs, improve employee empowerment and increase profitability. Moreover, I-BiDaaS will deliver a full array of big data business analytics solutions for structured, unstructured, noisy and potentially synthetic data for companies in multiple industries that are more accessible, cost effective and employee-empowering than existing solutions, which gives companies the confidence to deploy Big Data Self-Service solutions across the organisation, from consumer-facing employees with little IT experience or expertise to top management and helps companies to optimize decision-making at the tactical, operational and strategic levels.
The domains that can exploit such self-service solutions are numerous; I-BiDaaS will explore three critical ones with significant challenges and requirements: banking, manufacturing, and telecommunications.
To develop, validate, demonstrate and support, a complete and solid big data solution that can be easily configured and adopted by practitioners.
To break inter- and intra-sectorial data-silos, create a data market, offer new business opportunities and support data sharing, exchange as well as interoperability.
To construct a safe environment for methodological big data experimentation for the development of new products, services, tools.
To develop data processing tool and techniques applicable in real-world settings and demonstrate significant increase of speed of data throughput and access.
To develop technologies that will increase the efficiency and competitiveness of all EU companies and organisations that need to manage vast and complex amounts of data.
The detailed description of the I-BiDaaS architecture (the six work flow steps together with layered system and User interface) is presented below:
Step 1: The data (real or synthetic produced via data fabrication tool) are ingested into the batch processing and the streaming analytics modules via the Universal Message broker.
Step 2: The analytic modules perform analytics on the ingested streaming data, also referencing historic information where necessary, to identify business patterns that have happened or are about to happen.
Step 3 (I-BiDaaS innovation): The batch and real time analytic results are fed to the advanced visualization tools. An innovation is that part of the analytics can be offloaded to the parallel GPU-accelerated engine to further speed-up the execution of streaming analytics.
Step 4: The collected data can be stored in Hecuba, that uses the Apache Cassandra as a back-end, which can then be processed by the COMPSs pool of distributed machine learning algorithms.
Step 5 (I-BiDaaS innovation): The correlations produced by the analysis can fed back to the data fabrication platform, to be used for training and help building rules that will be used for future data generation purposes.
Step 6 (I-BiDaaS innovation): The real-time processing module feeds the batch-processing module with inputs that enable periodic refinements of models used in machine learning methods. The proposed solution goes beyond the traditional lambda architecture in terms of interleaving batch and stream processing.
I-BiDaaS User Interface: The proposed solution offers also a multi-purpose interface which can be used by different categories of users.It provides different levels of abstractions such as Programming API (providing access to every level of our software stack); Domain language (providing access to the application layer); Pre-defined analytics (providing simplest form to non-IT users to easily combine and multiplex with the desired data sources, to form a data processing pipeline).
I-BiDaaS layered system: The I-BiDaaS architecture composes of three principal layers: the infrastructure layer (providing the actual underlying storage and processing infrastructure of the I-BiDaaS solution); the distributed large-scale layer (controling the orchestration and management of the underlying physical computational and storage infrastructure) and application layer (refering to the architecture aspects and components that are involved in the actual workflow of extracting actionable knowledge from the big data, starting from data preparation, to analytics, to delivering analytics results for supporting decision making).