A stream processing architecture for heterogeneous data sources in the Internet of Things

The number of Internet of Things (IoT) and smart devices capable of producing, consuming and exchanging information is constantly increasing. It is estimated there will be around 30 billion of them in 2020. In most cases, the structures of the information produced by such devices are completely different, thus providing heterogeneous information. This is becoming a challenge for researchers working on IoT, who need to perform homogenisation and pre-processing tasks before using the IoT data. This paper aims to provide an architecture for processing and analysing data from heterogeneous sources with different structures in IoT scopes, allowing researchers to focus on data analysis, without having to worry about the structure of the data sources. This architecture combines the real-time stream processing paradigm for information processing and transforming, together with the complex event processing for information analysis. This provides us with capability of processing, transforming and analysing large amounts of information in real time. The results obtained from the evaluation of a real-world case study about water supply network management show that the architecture can be applied to an IoT water management scenario to analyse the information in real time. Additionally, the stress tests successfully conducted for this architecture highlight that a large incoming rate of input events could be processed without latency, resulting in efficient performance of the proposed architecture. This novel software architecture is adequate for automatically detecting situations of interest in the IoT through the processing, transformation and analysis of large amounts of heterogeneous information in real time.El número de dispositivos del Internet de las Cosas (IoT) y dispositivos inteligentes capaces de producir, consumir e intercambiar información está aumentando constantemente. Se estima que habrá alrededor de 30 mil millones de ellos en 2020. En la mayoría de los casos, las estructuras de la información producida por dichos dispositivos son completamente diferentes, proporcionando así información heterogénea. Esto se está convirtiendo en un desafío para los investigadores que trabajan en IoT, quienes necesitan realizar tareas de homogeneización y preprocesamiento antes de utilizar los datos de IoT. Este documento tiene como objetivo proporcionar una arquitectura para procesar y analizar datos de fuentes heterogéneas con diferentes estructuras en ámbitos de IoT, permitiendo a los investigadores centrarse en el análisis de datos, sin tener que preocuparse por la estructura de las fuentes de datos. Esta arquitectura combina el paradigma de procesamiento de flujo en tiempo real para el procesamiento y transformación de información, junto con el procesamiento de eventos complejos para el análisis de información. Esto nos proporciona la capacidad de procesar, transformar y analizar grandes cantidades de información en tiempo real. Los resultados obtenidos de la evaluación de un estudio de caso del mundo real sobre la gestión de la red de suministro de agua muestran que la arquitectura puede ser aplicada a un escenario de gestión de agua de IoT para analizar la información en tiempo real. Además, las pruebas de estrés realizadas con éxito para esta arquitectura destacan que una gran tasa de entrada de eventos de entrada podría ser procesada sin latencia, lo que resulta en un rendimiento eficiente de la arquitectura propuesta. Esta novedosa arquitectura de software es adecuada para detectar automáticamente situaciones de interés en el IoT a través del procesamiento, transformación y análisis de grandes cantidades de información heterogénea en tiempo real.This work was supported in part by the Spanish Ministry of Science and Innovation and the European Union FEDER Funds (No. TIN2015-65845-C3-3-R and No. RTI2018-093608-B-C33) and in part by the pre-doctoral program of the University of Cádiz (2017-020/PU/EPIF-FPI-CT/CP). In addition, we would like to thank GEN Grupo Energético for sharing their data for testing purposes

When Things Matter: A Data-Centric View of the Internet of Things

With the recent advances in radio-frequency identification (RFID), low-cost wireless sensor devices, and Web technologies, the Internet of Things (IoT) approach has gained momentum in connecting everyday objects to the Internet and facilitating machine-to-human and machine-to-machine communication with the physical world. While IoT offers the capability to connect and integrate both digital and physical entities, enabling a whole new class of applications and services, several significant challenges need to be addressed before these applications and services can be fully realized. A fundamental challenge centers around managing IoT data, typically produced in dynamic and volatile environments, which is not only extremely large in scale and volume, but also noisy, and continuous. This article surveys the main techniques and state-of-the-art research efforts in IoT from data-centric perspectives, including data stream processing, data storage models, complex event processing, and searching in IoT. Open research issues for IoT data management are also discussed

City Data Fusion: Sensor Data Fusion in the Internet of Things

Internet of Things (IoT) has gained substantial attention recently and play a significant role in smart city application deployments. A number of such smart city applications depend on sensor fusion capabilities in the cloud from diverse data sources. We introduce the concept of IoT and present in detail ten different parameters that govern our sensor data fusion evaluation framework. We then evaluate the current state-of-the art in sensor data fusion against our sensor data fusion framework. Our main goal is to examine and survey different sensor data fusion research efforts based on our evaluation framework. The major open research issues related to sensor data fusion are also presented.Comment: Accepted to be published in International Journal of Distributed Systems and Technologies (IJDST), 201