IOT Stream Analytics Platform

The Internet of Things (IoT) is changing people’s surrounding physical world into an information ecosystem that facilitate our everyday life. Billions of smart objects become data-generating “things” that can sense environmental changes and report their sensed data. Leveraging the huge amount of sensory information is a key issue to realize the IoT solutions in many areas. Adequate technologies are required for data collection, transmission, data processing, analysis, reporting, and advanced querying. In this thesis, an IoT Stream Analytics Platform that supports IoT application and service development is proposed: it provides user applications a way to capture flowing data from multitudes of data sources and provide analytical insights in real time based on user needs. Developers can conveniently build their IoT applications on this platform without having to consider the diversity and complexity of smart devices and their underlying networks

Custom Windows Patching Methodology - Comparative Analysis

Windows Server Update Services has been a common mainstay among organizations with a heavy footprint of Windows operating systems since it was originally released as Software Update Services in 2002. While the product has grown in scope, the primary allure remains the same: WSUS offers organizations greater control over the patches that are released to their environment and saves bandwidth by allowing a centralized device to download and offer patches to internal clients rather than having each of those clients download the content they require from the Internet. Unfortunately, the product has a structural limitation in that it lacks the capacity to provide high-availability to the metadata synchronization process that must occur in order to deliver the most up-to-date patches to endpoints. WSUS metadata contains details about the individual updates, EULAs, and supersedence relationships. Due to design limitations and the growing concern of outages, a solution was developed to supplement and perhaps replace WSUS in certain scenarios. This solution, dubbed the Custom Patching Manager (CPM), is an extension of a concept originally started by Alejandro Gómez Galindo and finds middle-ground between Windows Server Update Services and Windows Update using freely available software. The solution assesses the vulnerabilities of a system or systems, determines whether or not the patches are part of an approved list, determines whether or not the content for missing updates is available locally, acquires that content depending on the previous step, and applies the patches to the endpoint. This proof-of-concept proved functional and reliable but would benefit from some optimizations that have been recommended as future works

Bridging OPC UA and DPWS for Industrial SOA

Two web-service based specifications, OPC Unified Architecture (OPC UA) and Devices Profile for Web Services (DPWS), have been proposed by various researchers and organizations as possible enabling technologies for an event-driven Service Oriented Architecture for monitoring and control in manufacturing applications. This paper aims to propose and demonstrate an approach for bridging these two technologies in a way that is applicable in existing industrial applications. A merger between OPC UA and DPWS that effectively combines their complementary strengths could help pave the path toward future industrial event-driven SOA applications, with the inherent modularity, agility, and interoperability envisioned by researchers today. A representation of DPWS devices, services, operations and events in the OPC UA data model is proposed, and a DPWS Module is developed for Ignition, a commercially available HMI/SCADA and MES platform with integrated OPC UA Server. The module discovers DPWS devices in a local network, creates the representation in the address space, and handles subscriptions, input and output parameter values, and invoking operations. A Complex Event Processing component based on Microsoft’s StreamInsight is also integrated with the system, input and output adapters exposing web service interfaces. The system prototype developed will be used as the base for a use case demonstrator in the European Commission’s Framework Package 7 Project, “Architecture for Service-Oriented Process Monitoring and Control (IMC AESOP).” The project aims to develop a system of systems approach for monitoring and control, based on SOA for very large-scale systems in the process industries

Complex Event Processing (CEP) - Using SQL Server StreamInsight for Near Real-Time Visualization and Monitoring

Atualmente as empresas têm necessidade em reagir em tempo real a eventos que ocorram durante o seu funcionamento. Estes eventos surgem sob a forma de streams de eventos que ocorrem a um dado instante de tempo.Uma forma popular de processar essas streams de eventos é utilizar a tecnologia de Complex Event Processing. Esta permite processar streams de eventos em tempo real e construir janelas temporais sobre essa stream, podendo depois aplicar agregações sobre as mesmas. Normalmente esta funcionalidade é obtida através da adição de funcionalidades à linguagem SQL por parte de um motor de CEP, permitindo que se possa utilizar SQL para processar streams através da construção de queries e criar janelas temporais sobre as mesmas.Infelizmente muitos sistemas de CEP requerem conhecimento à priori do tipo (schema) de eventos que terão de processar bem como do tipo de queries que irão ser executadas sobre eles.Pretende-se com esta dissertação implementar um sistema de CEP que possa funcionar sem ter qualquer tipo de conhecimento à priori do tipo de eventos que possam surgir, mas mantendo a capacidade de criar queries que possam executar esses eventos.O motor de CEP utilizado nesta dissertação foi o Microsoft StreamInsight.Nowadays business needs to react in real time to events that happen during their work. These events appear as streams of events, where each event has occurred during a point in time.A popular way to process those event streams, is by using Complex Event Processing. This technology allows the processing, in real time, of event streams, the creating of time windows over those streams and the use of aggregations on those windows. Usually this functionality is gained by using a CEP engine that extends the SQL language allowing the latter to process streams by constructing queries and create temporal windows on them.Unfortunately before using a CEP system, many require à priori knowledge regarding the type (i.e. schema) of events that can appear on their streams and what queries it can run. This dissertation implements a CEP system that can work without knowing the type of events that may appear, but still has the ability to create queries over those event.The CEP engine used in this dissertation was Microsoft's StreamInsight

Dynamic Complex Event Processing for Industrial Monitoring Systems

Using Complex Event Processing (CEP) as part of monitoring systems is a state-of-the-art approach in the manufacturing industry that still requires development. The industry is increasingly moving towards implementing Service Oriented Architecture (SOA) based systems to respond to increasing demands of interoperability amongst other operations in a business organisation. Complex event processors are used as part of monitoring systems but current complex event processors are usually system specific. This thesis aims to propose and demonstrate a more dynamic approach for implementing an industrial monitoring system using complex event processing. Service Oriented Architecture uses event-based messaging to communicate between different devices and systems. This creates large amounts of data in the monitored system. In order to infer important information from this vast body of data the CEP is used to query through the events. These queries are predefined and cannot be changed during runtime. The CEP holds the main logic of the monitoring system and thus dictates what the system actually monitors. Monitoring system requires the possibility to change the monitoring logic. This is why a method of dynamically adding queries will be proposed in this thesis. In order for a SOA-based monitoring system to be dynamic the CEP needs to be dynamic. This thesis proposes a CEP solution with generic implementation, dynamic query definition during runtime and the possibility to use recursive user defined functions that allow reusing query templates in different solutions. The developed CEP is tested with two different implementation use cases. First one a simulated use case that tests the monitoring system performance with large amounts of events. Second one a manufacturing line implementation to demonstrate the monitoring system in an actual manufacturing environment. Tests were run on both use cases to gain information on how the CEP performs and to demonstrate the functionality of the developed monitoring system. The developed CEP was used as a part of oil lubrication use case for IMC-AESOP project. IMC-AESOP project was an EU project researching how to apply state-of-the-art SOA-based systems to the industrial automation field

Microsoft SQL Server 2008:n ylläpito (70-433)

Insinöörityössä oli tavoitteena tehdä kurssimateriaali ja virtuaalikoneen malli Microsoftin SQL Server 2008 -tietokannan kehittäjäkurssia varten Metropolia Ammattikorkeakoululle. Kurssi on valinnainen osa tietoverkkojen koulutusohjelmaa. Työn pohjana toimi Microsoft SQL Server 2008 - Database development -itseopiskelukirjan harjoitukset ja sen mukana tullut materiaali. Työssä käydään aluksi läpi, mitä uutta SQL Server 2008:ssa on verrattuna aikaisempiin versioihin ja mikä on SQL Azure. Sen jälkeen käydään läpi, mitä kurssi pitää sisällään ja miten levykuvan luominen onnistui. Insinöörityössä tehtiin valmis levykuva, joka voidaan asentaa koulun virtuaaliympäristöön tai VirtualBoxiin. Kurssikirjan sisältämät tehtävät myös tarkistettiin, että ne toimivat syntyneellä alustalla.The purpose of this final project was to make complete course material for Microsoft SQL Server 2008 database development course. The project was done for Metropolia University of Applied Sciences. The course is optional part of information technology programme. The project was based on Microsoft's self-paced training kit called Microsoft SQL Server 2008- Database Development (70-433). First this report discusses what is new in SQL Server 2008 version, compared to previous versions and what is SQL Azure. Next chapters take a look what this course contains and how successful was making the image of the training platform that Metropolia's virtual training environment needs. The platform consists of a operating system image that can be installed on the virtualization software. After that it was made sure that the training kits exercises work in that environment

Microsoft SQL Server 2008:n ylläpito (70-433)

Insinöörityössä oli tavoitteena tehdä kurssimateriaali ja virtuaalikoneen malli Microsoftin SQL Server 2008 -tietokannan kehittäjäkurssia varten Metropolia Ammattikorkeakoululle. Kurssi on valinnainen osa tietoverkkojen koulutusohjelmaa. Työn pohjana toimi Microsoft SQL Server 2008 - Database development -itseopiskelukirjan harjoitukset ja sen mukana tullut materiaali. Työssä käydään aluksi läpi, mitä uutta SQL Server 2008:ssa on verrattuna aikaisempiin versioihin ja mikä on SQL Azure. Sen jälkeen käydään läpi, mitä kurssi pitää sisällään ja miten levykuvan luominen onnistui. Insinöörityössä tehtiin valmis levykuva, joka voidaan asentaa koulun virtuaaliympäristöön tai VirtualBoxiin. Kurssikirjan sisältämät tehtävät myös tarkistettiin, että ne toimivat syntyneellä alustalla.The purpose of this final project was to make complete course material for Microsoft SQL Server 2008 database development course. The project was done for Metropolia University of Applied Sciences. The course is optional part of information technology programme. The project was based on Microsoft's self-paced training kit called Microsoft SQL Server 2008- Database Development (70-433). First this report discusses what is new in SQL Server 2008 version, compared to previous versions and what is SQL Azure. Next chapters take a look what this course contains and how successful was making the image of the training platform that Metropolia's virtual training environment needs. The platform consists of a operating system image that can be installed on the virtualization software. After that it was made sure that the training kits exercises work in that environment

Liquid stream processing on the web: a JavaScript framework

The Web is rapidly becoming a mature platform to host distributed applications. Pervasive computing application running on the Web are now common in the era of the Web of Things, which has made it increasingly simple to integrate sensors and microcontrollers in our everyday life. Such devices are of great in- terest to Makers with basic Web development skills. With them, Makers are able to build small smart stream processing applications with sensors and actuators without spending a fortune and without knowing much about the technologies they use. Thanks to ongoing Web technology trends enabling real-time peer-to- peer communication between Web-enabled devices, Web browsers and server- side JavaScript runtimes, developers are able to implement pervasive Web ap- plications using a single programming language. These can take advantage of direct and continuous communication channels going beyond what was possible in the early stages of the Web to push data in real-time. Despite these recent advances, building stream processing applications on the Web of Things remains a challenging task. On the one hand, Web-enabled devices of different nature still have to communicate with different protocols. On the other hand, dealing with a dynamic, heterogeneous, and volatile environment like the Web requires developers to face issues like disconnections, unpredictable workload fluctuations, and device overload. To help developers deal with such issues, in this dissertation we present the Web Liquid Streams (WLS) framework, a novel streaming framework for JavaScript. Developers implement streaming operators written in JavaScript and may interactively and dynamically define a streaming topology. The framework takes care of deploying the user-defined operators on the available devices and connecting them using the appropriate data channel, removing the burden of dealing with different deployment environments from the developers. Changes in the semantic of the application and in its execution environment may be ap- plied at runtime without stopping the stream flow. Like a liquid adapts its shape to the one of its container, the Web Liquid Streams framework makes streaming topologies flow across multiple heterogeneous devices, enabling dynamic operator migration without disrupting the data flow. By constantly monitoring the execution of the topology with a hierarchical controller infrastructure, WLS takes care of parallelising the operator execution across multiple devices in case of bottlenecks and of recovering the execution of the streaming topology in case one or more devices disconnect, by restarting lost operators on other available devices

Large-Scale Indexing, Discovery, and Ranking for the Internet of Things (IoT)

Network-enabled sensing and actuation devices are key enablers to connect real-world objects to the cyber world. The Internet of Things (IoT) consists of the network-enabled devices and communication technologies that allow connectivity and integration of physical objects (Things) into the digital world (Internet). Enormous amounts of dynamic IoT data are collected from Internet-connected devices. IoT data are usually multi-variant streams that are heterogeneous, sporadic, multi-modal, and spatio-temporal. IoT data can be disseminated with different granularities and have diverse structures, types, and qualities. Dealing with the data deluge from heterogeneous IoT resources and services imposes new challenges on indexing, discovery, and ranking mechanisms that will allow building applications that require on-line access and retrieval of ad-hoc IoT data. However, the existing IoT data indexing and discovery approaches are complex or centralised, which hinders their scalability. The primary objective of this article is to provide a holistic overview of the state-of-the-art on indexing, discovery, and ranking of IoT data. The article aims to pave the way for researchers to design, develop, implement, and evaluate techniques and approaches for on-line large-scale distributed IoT applications and services