Real-Time Data Processing With Lambda Architecture

Data has evolved immensely in recent years, in type, volume and velocity. There are several frameworks to handle the big data applications. The project focuses on the Lambda Architecture proposed by Marz and its application to obtain real-time data processing. The architecture is a solution that unites the benefits of the batch and stream processing techniques. Data can be historically processed with high precision and involved algorithms without loss of short-term information, alerts and insights. Lambda Architecture has an ability to serve a wide range of use cases and workloads that withstands hardware and human mistakes. The layered architecture enhances loose coupling and flexibility in the system. This a huge benefit that allows understanding the trade-offs and application of various tools and technologies across the layers. There has been an advancement in the approach of building the LA due to improvements in the underlying tools. The project demonstrates a simplified architecture for the LA that is maintainable

A comparative analysis of scalable and context-aware trust management approaches for internet of things

© Springer International Publishing Switzerland 2015. The Internet of Things - IoT - is a new paradigm in technology that allows most physical ‘things’ to contact each other. Trust between IoT devices is a critical factor. Trust in the IoT environment can be modeled using various approaches, such as confidence level and reputation parameters. Furthermore, trust is an important element in engineering reliable and scalable networks. In this paper, we survey scalable and context-aware trust management for IoT from three perspectives. First, we present an overview of the IoT and the importance of trust in relation to it, and then we provide an in-depth trust/reliable management protocol for the IoT and evaluate comparable trust management protocols. We also investigate a scalable solution for trust management in the IoT and provide a comparative evaluation of existing trust solutions. We then pre-sent a context-aware assessment for the IoT and compare the different trust solutions. Lastly, we give a full comparative analysis of trust/reliability management in the IoT. Our results are drawn from this comparative analysis, and directions for future research are outlined

Sensor Observation Streams Within Cloud-based IoT Platforms: Challenges and Directions

Observation streams can be considered as a special case of data streams produced by sensors. With the growth of the Internet of Things (IoT), more and more connected sensors will produce unbounded observation streams. In order to bridge the gap between sensors and observation consumers, we have witnessed the design and the development of Cloud-based IoT platforms. Such systems raise new research challenges, in particular regarding observation collection, processing and consumption. These new research challenges are related to observation streams and should be addressed from the implementation phase by developers to build platforms able to meet other non-functional requirements later. Unlike existing surveys, this paper is intended for developers that would like to design and implement a Cloud-based IoT platform capable of handling sensor observation streams. It provides a comprehensive way to understand main observation-related challenges, as well as non-functional requirements of IoT platforms such as platform adaptation, scalability and availability. Last but not the least, it gives recommendations and compares some relevant open-source software that can speed up the development process

IIoT Data Ness: From Streaming to Added Value

In the emerging Industry 4.0 paradigm, the internet of things has been an innovation driver, allowing for environment visibility and control through sensor data analysis. However the data is of such volume and velocity that data quality cannot be assured by conventional architectures. It has been argued that the quality and observability of data are key to a project’s success, allowing users to interact with data more effectively and rapidly. In order for a project to become successful in this context, it is of imperative importance to incorporate data quality mechanisms in order to extract the most value out of data. If this goal is achieved one can expect enormous advantages that could lead to financial and innovation gains for the industry. To cope with this reality, this work presents a data mesh oriented methodology based on the state-of-the-art data management tools that exist to design a solution which leverages data quality in the Industrial Internet of Things (IIoT) space, through data contextualization. In order to achieve this goal, practices such as FAIR data principles and data observability concepts were incorporated into the solution. The result of this work allowed for the creation of an architecture that focuses on data and metadata management to elevate data context, ownership and quality.O conceito de Internet of Things (IoT) é um dos principais fatores de sucesso para a nova Indústria 4.0. Através de análise de dados sobre os valores que os sensores coletam no seu ambiente, é possível a construção uma plataforma capaz de identificar condições de sucesso e eventuais problemas antes que estes ocorram, resultando em ganho monetário relevante para as empresas. No entanto, este caso de uso não é de fácil implementação, devido à elevada quantidade e velocidade de dados proveniente de um ambiente de IIoT (Industrial Internet of Things)

A Semantic loT Early Warning System for Natural Environment Crisis Management

An early warning system (EWS) is a core type of data driven Internet of Things (IoTs) system used for environment disaster risk and effect management. The potential benefits of using a semantic-type EWS include easier sensor and data source plug-and-play, simpler, richer, and more dynamic metadata-driven data analysis and easier service interoperability and orchestration. The challenges faced during practical deployments of semantic EWSs are the need for scalable time-sensitive data exchange and processing (especially involving heterogeneous data sources) and the need for resilience to changing ICT resource constraints in crisis zones. We present a novel IoT EWS system framework that addresses these challenges, based upon a multisemantic representation model.We use lightweight semantics for metadata to enhance rich sensor data acquisition.We use heavyweight semantics for top level W3CWeb Ontology Language ontology models describing multileveled knowledge-bases and semantically driven decision support and workflow orchestration. This approach is validated through determining both system related metrics and a case study involving an advanced prototype system of the semantic EWS, integrated with a reployed EWS infrastructure

A Semantic IoT Early Warning System for Natural Environment Crisis Management

This work was supported in part by the European FP7 Funded Project TRIDEC under Grant 258723, the other project partners in helping to deliver the complete project Syste, in particular, GFZ, and the German Research Centre for Geosciences, Potsdam, Germany. The work of R. Tao was supported by the Queen Mary University of London for a Ph.D. studentship