An efficient time optimized scheme for progressive analytics in big data

Big data analytics is the key research subject for future data driven decision making applications. Due to the large amount of data, progressive analytics could provide an efficient way for querying big data clusters. Each cluster contains only a piece of the examined data. Continuous queries over these data sources require intelligent mechanisms to result the final outcome (query response) in the minimum time with the maximum performance. A Query Controller (QC) is responsible to manage continuous/sequential queries and return the final outcome to users or applications. In this paper, we propose a mechanism that can be adopted by the QC. The proposed mechanism is capable of managing partial results retrieved by a number of processors each one responsible for each cluster. Each processor executes a query over a specific cluster of data. Our mechanism adopts two sequential decision making models for handling the incoming partial results. The first model is based on a finite horizon time-optimized model and the second one is based on an infinite horizon optimally scheduled model. We provide mathematical formulations for solving the discussed problem and present simulation results. Through a large number of experiments, we reveal the advantages of the proposed models and give numerical results comparing them with a deterministic model. These results indicate that the proposed models can efficiently reduce the required time for returning the final outcome to the user/application while keeping the quality of the aggregated result at high levels

Manufacturing Data Analytics for Manufacturing Quality Assurance

The authors acknowledge the European Commission for the support and funding under the scope of Horizon2020 i4Q Innovation Project (Agreement Number 958205) and the remaining partners of the i4Q Project Consortium.Nowadays, manufacturing companies are eager to access insights from advanced analytics, without requiring them to have specialized IT workforce or data science advanced skills. Most of current solutions lack of easy-to-use advanced data preparation, production reporting and advanced analytics and prediction. Thanks to the increase in the use of sensors, actuators and instruments, European manufacturing lines collect a huge amount of data during the manufacturing process, which is very valuable for the improvement of quality in manufacturing, but analyzing huge amounts of data on a daily basis, requires heavy statistical and technology training and support, making them not accessible for SMEs. The European i4Q Project, aims at providing an IoT-based Reliable Industrial Data Services (RIDS), a complete suite consisting of 22 i4Q Solutions, able to manage the huge amount of industrial data coming from cheap cost-effective, smart, and small size interconnected factory devices for supporting manufacturing online monitoring and control. This paper will present a set of i4Q services, for data integration and fusion, data analytics and data distribution. Such services, will be responsible for the execution of AI workloads (including at the edge), enabling the dynamic deployment industrial scenarios based on a cloud/edge architecture. Monitoring at various levels is provided in i4Q through scalable tools and the collected data, is used for a variety of activities including resource monitoring and management, workload assignment, smart alerting, predictive failure and model (re)training.publishersversionpublishe

An intelligent information forwarder for healthcare big data systems with distributed wearable sensors

© 2016 IEEE. An increasing number of the elderly population wish to live an independent lifestyle, rather than rely on intrusive care programmes. A big data solution is presented using wearable sensors capable of carrying out continuous monitoring of the elderly, alerting the relevant caregivers when necessary and forwarding pertinent information to a big data system for analysis. A challenge for such a solution is the development of context-awareness through the multidimensional, dynamic and nonlinear sensor readings that have a weak correlation with observable human behaviours and health conditions. To address this challenge, a wearable sensor system with an intelligent data forwarder is discussed in this paper. The forwarder adopts a Hidden Markov Model for human behaviour recognition. Locality sensitive hashing is proposed as an efficient mechanism to learn sensor patterns. A prototype solution is implemented to monitor health conditions of dispersed users. It is shown that the intelligent forwarders can provide the remote sensors with context-awareness. They transmit only important information to the big data server for analytics when certain behaviours happen and avoid overwhelming communication and data storage. The system functions unobtrusively, whilst giving the users peace of mind in the knowledge that their safety is being monitored and analysed

IIoT Data Ness: From Streaming to Added Value

In the emerging Industry 4.0 paradigm, the internet of things has been an innovation driver, allowing for environment visibility and control through sensor data analysis. However the data is of such volume and velocity that data quality cannot be assured by conventional architectures. It has been argued that the quality and observability of data are key to a project’s success, allowing users to interact with data more effectively and rapidly. In order for a project to become successful in this context, it is of imperative importance to incorporate data quality mechanisms in order to extract the most value out of data. If this goal is achieved one can expect enormous advantages that could lead to financial and innovation gains for the industry. To cope with this reality, this work presents a data mesh oriented methodology based on the state-of-the-art data management tools that exist to design a solution which leverages data quality in the Industrial Internet of Things (IIoT) space, through data contextualization. In order to achieve this goal, practices such as FAIR data principles and data observability concepts were incorporated into the solution. The result of this work allowed for the creation of an architecture that focuses on data and metadata management to elevate data context, ownership and quality.O conceito de Internet of Things (IoT) é um dos principais fatores de sucesso para a nova Indústria 4.0. Através de análise de dados sobre os valores que os sensores coletam no seu ambiente, é possível a construção uma plataforma capaz de identificar condições de sucesso e eventuais problemas antes que estes ocorram, resultando em ganho monetário relevante para as empresas. No entanto, este caso de uso não é de fácil implementação, devido à elevada quantidade e velocidade de dados proveniente de um ambiente de IIoT (Industrial Internet of Things)

Implementation of a NFV monitoring system for reactive environments

This work aims at researching the existent solutions of monitoring and alerting techniques, as well as defining a suitable architecture, design and implementation of a complete and customizable monitoring and alerting framework used to inspect and notify specific conditions on dynamically instantiated applications operating in the network. Such Network Services (NS) are used in the Network Function Virtualization (NFV) architecture, allowing rapid instantiation and configuration of virtualized environments that handle network configuration. This design and implementation seek to provide more flexibility and dynamicity to the network operator to monitor custom or generic metrics and trigger notifications based on custom thresholds, without depending on the Virtual Network Function (VNF) developer to adapt its descriptor and onboard each version into the NFV Orchestrator (NFVO) prior to each usage. The framework here developed follows a modular architecture that separates the monitoring and alerting policies from the onboarding and instantiation process of the Network Functions. The architecture also facilitates the integration with other systems and adapting the functionality of an operational environment thanks to its decoupled and modular approach. The presented work considers a monitoring and alerting framework that is especially useful for dynamic environments such as those relying in NFV, like those in the EU H2020 PALANTIR project. There, the framework is used to help assessing the correct behavior of the Security NSs that are used to prevent or mitigate security anomalies in the network of each client. If abnormalities are found, remediation measures will take place to replace the potentially compromised NS instances with clean, appropriate ones.Objectius de Desenvolupament Sostenible::9 - Indústria, Innovació i Infraestructur

PHOENI2X -- A European Cyber Resilience Framework With Artificial-Intelligence-Assisted Orchestration, Automation and Response Capabilities for Business Continuity and Recovery, Incident Response, and Information Exchange

As digital technologies become more pervasive in society and the economy, cybersecurity incidents become more frequent and impactful. According to the NIS and NIS2 Directives, EU Member States and their Operators of Essential Services must establish a minimum baseline set of cybersecurity capabilities and engage in cross-border coordination and cooperation. However, this is only a small step towards European cyber resilience. In this landscape, preparedness, shared situational awareness, and coordinated incident response are essential for effective cyber crisis management and resilience. Motivated by the above, this paper presents PHOENI2X, an EU-funded project aiming to design, develop, and deliver a Cyber Resilience Framework providing Artificial-Intelligence-assisted orchestration, automation and response capabilities for business continuity and recovery, incident response, and information exchange, tailored to the needs of Operators of Essential Services and the EU Member State authorities entrusted with cybersecurity