Semantic reasoning on the edge of internet of things

Abstract. The Internet of Things (IoT) is a paradigm where physical objects are connected with each other with identifying, sensing, networking and processing capabilities over the Internet. Millions of new devices will be added into IoT network thus generating huge amount of data. How to represent, store, interconnect, search, and organize information generated by IoT devices become a challenge. Semantic technologies could play an important role by encoding meaning into data to enable a computer system to possess knowledge and reasoning. The vast amount of devices and data are also challenges. Edge Computing reduces both network latency and resource consumptions by deploying services and distributing computing tasks from the core network to the edge. We recognize four challenges from IoT systems. First the centralized server may generate long latency because of physical distances. Second concern is that the resource-constrained IoT devices have limited computing ability in processing heavy tasks. Third, the data generated by heterogeneous devices can hardly be understood and utilized by other devices or systems. Our research focuses on these challenges and provide a solution based on Edge computing and semantic technologies. We utilize Edge computing and semantic reasoning into IoT. Edge computing distributes tasks to the reasoning devices, which we call the Edge nodes. They are close to the terminal devices and provide services. The newly added resources could balance the workload of the systems and improve the computing capability. We annotate meaning into the data with Resource Description Framework thus providing an approach for heterogeneous machines to understand and utilize the data. We use semantic reasoning as a general purpose intelligent processing method. The thesis work focuses on studying semantic reasoning performance in IoT system with Edge computing paradigm. We develop an Edge based IoT system with semantic technologies. The system deploys semantic reasoning services on Edge nodes. Based on IoT system, we design five experiments to evaluate the performance of the integrated IoT system. We demonstrate how could the Edge computing paradigm facilitate IoT in terms of data transforming, semantic reasoning and service experience. We analyze how to improve the performance by properly distributing the task for Cloud and Edge nodes. The thesis work result shows that the Edge computing could improve the performance of the semantic reasoning in IoT

Compact semantic representations of observational data

Das Konzept des Internet der Dinge (IoT) ist in mehreren Bereichen weit verbreitet, damit Geräte miteinander interagieren und bestimmte Aufgaben erfüllen können. IoT-Geräte umfassen verschiedene Konzepte, z.B. Sensoren, Programme, Computer und Aktoren. IoT-Geräte beobachten ihre Umgebung, um Informationen zu sammeln und miteinander zu kommunizieren, um gemeinsame Aufgaben zu erfüllen. Diese Vorrichtungen erzeugen kontinuierlich Beobachtungsdatenströme, die zu historischen Daten werden, wenn diese Beobachtungen gespeichert werden. Durch die Zunahme der Anzahl der IoT-Geräte wird eine große Menge an Streaming- und historischen Beobachtungsdaten erzeugt. Darüber hinaus wurden mehrere Ontologien, wie die Semantic Sensor Network (SSN) Ontologie, für die semantische Annotation von Beobachtungsdaten vorgeschlagen - entweder Stream oder historisch. Das Resource Description Framework (RDF) ist ein weit verbreitetes Datenmodell zur semantischen Beschreibung der Datensätze. Semantische Annotation bietet ein gemeinsames Verständnis für die Verarbeitung und Analyse von Beobachtungsdaten. Durch das Hinzufügen von Semantik wird die Datengröße jedoch weiter erhöht, insbesondere wenn die Beobachtungswerte von mehreren Geräten redundant erfasst werden. So können beispielsweise mehrere Sensoren Beobachtungen erzeugen, die den gleichen Wert für die relative Luftfeuchtigkeit in einem bestimmten Zeitstempel und einer bestimmten Stadt anzeigen. Diese Situation kann in einem RDF-Diagramm mit vier RDF-Tripel dargestellt werden, wobei Beobachtungen als Tripel dargestellt werden, die das beobachtete Phänomen, die Maßeinheit, den Zeitstempel und die Koordinaten beschreiben. Die RDF-Tripel einer Beobachtung sind mit dem gleichen Thema verbunden. Solche Beobachtungen teilen sich die gleichen Objekte in einer bestimmten Gruppe von Eigenschaften, d.h. sie entsprechen einem Sternmuster, das sich aus diesen Eigenschaften und Objekten zusammensetzt. Wenn die Anzahl dieser Subjektentitäten oder Eigenschaften in diesen Sternmustern groß ist, wird die Größe des RDF-Diagramms und der Abfrageverarbeitung negativ beeinflusst; wir bezeichnen diese Sternmuster als häufige Sternmuster. Diese Arbeit befasst sich mit dem Problem der Identifizierung von häufigen Sternenmustern in RDF-Diagrammen und entwickelt Berechnungsmethoden, um häufige Sternmuster zu identifizieren und ein faktorisiertes RDF-Diagramm zu erzeugen, bei dem die Anzahl der häufigen Sternmuster minimiert wird. Darüber hinaus wenden wir diese faktorisierten RDF-Darstellungen über historische semantische Sensordaten an, die mit der SSN-Ontologie beschrieben werden, und präsentieren tabellarische Darstellungen von faktorisierten semantischen Sensordaten, um Big Data-Frameworks auszunutzen. Darüber hinaus entwickelt diese Arbeit einen wissensbasierten Ansatz namens DESERT, der in der Lage ist, bei Bedarf Streamdaten zu faktorisieren und semantisch anzureichern (on-Demand factorizE and Semantically Enrich stReam daTa). Wir bewerten die Leistung unserer vorgeschlagenen Techniken anhand mehrerer RDF-Diagramm-Benchmarks. Die Ergebnisse zeigen, dass unsere Techniken in der Lage sind, häufige Sternmuster effektiv und effizient zu erkennen, und die Größe der RDF-Diagramme kann um bis zu 66,56% reduziert werden, während die im ursprünglichen RDF-Diagramm dargestellten Daten erhalten bleiben. Darüber hinaus sind die kompakten Darstellungen in der Lage, die Anzahl der RDF-Tripel um mindestens 53,25% in historischen Beobachtungsdaten und bis zu 94,34% in Beobachtungsdatenströmen zu reduzieren. Darüber hinaus reduzieren die Ergebnisse der Anfrageauswertung über historische Daten die Ausführungszeit der Anfrage um bis zu drei Größenordnungen. In Beobachtungsdatenströmen wird die Größe der zur Beantwortung der Anfrage benötigten Daten um 92,53% reduziert, wodurch der Speicherplatzbedarf zur Beantwortung der Anfragen reduziert wird. Diese Ergebnisse belegen, dass IoT-Daten mit den vorgeschlagenen kompakten Darstellungen effizient dargestellt werden können, wodurch die negativen Auswirkungen semantischer Annotationen auf das IoT-Datenmanagement reduziert werden.The Internet of Things (IoT) concept has been widely adopted in several domains to enable devices to interact with each other and perform certain tasks. IoT devices encompass different concepts, e.g., sensors, programs, computers, and actuators. IoT devices observe their surroundings to collect information and communicate with each other in order to perform mutual tasks. These devices continuously generate observational data streams, which become historical data when these observations are stored. Due to an increase in the number of IoT devices, a large amount of streaming and historical observational data is being produced. Moreover, several ontologies, like the Semantic Sensor Network (SSN) Ontology, have been proposed for semantic annotation of observational data-either streams or historical. Resource Description Framework (RDF) is widely adopted data model to semantically describe the datasets. Semantic annotation provides a shared understanding for processing and analysis of observational data. However, adding semantics, further increases the data size especially when the observation values are redundantly sensed by several devices. For example, several sensors can generate observations indicating the same value for relative humidity in a given timestamp and city. This situation can be represented in an RDF graph using four RDF triples where observations are represented as triples that describe the observed phenomenon, the unit of measurement, the timestamp, and the coordinates. The RDF triples of an observation are associated with the same subject. Such observations share the same objects in a certain group of properties, i.e., they match star patterns composed of these properties and objects. In case the number of these subject entities or properties in these star patterns is large, the size of the RDF graph and query processing are negatively impacted; we refer these star patterns as frequent star patterns. This thesis addresses the problem of identifying frequent star patterns in RDF graphs and develop computational methods to identify frequent star patterns and generate a factorized RDF graph where the number of frequent star patterns is minimized. Furthermore, we apply these factorized RDF representations over historical semantic sensor data described using the SSN ontology and present tabular-based representations of factorized semantic sensor data in order to exploit Big Data frameworks. In addition, this thesis devises a knowledge-driven approach named DESERT that is able to on-Demand factorizE and Semantically Enrich stReam daTa. We evaluate the performance of our proposed techniques on several RDF graph benchmarks. The outcomes show that our techniques are able to effectively and efficiently detect frequent star patterns and RDF graph size can be reduced by up to 66.56% while data represented in the original RDF graph is preserved. Moreover, the compact representations are able to reduce the number of RDF triples by at least 53.25% in historical observational data and upto 94.34% in observational data streams. Additionally, query evaluation results over historical data reduce query execution time by up to three orders of magnitude. In observational data streams the size of the data required to answer the query is reduced by 92.53% reducing the memory space requirements to answer the queries. These results provide evidence that IoT data can be efficiently represented using the proposed compact representations, reducing thus, the negative impact that semantic annotations may have on IoT data management

Ontologias para Manutenção Preditiva com Dados sensíveis ao tempo

As empresas de fabrico industrial devem assegurar um processo produtivo contínuo para serem competitivas e fornecer os produtos fabricados no prazo e com a qualidade exigida pelos clientes. A quebra da cadeia de fabrico pode ter desfechos graves, resultando numa redução da produção e na interrupção da cadeia de abastecimento. Estes processos são compostos por cadeias de máquinas que executam tarefas em etapas. Cada máquina tem uma tarefa específica a executar, e o resultado de cada etapa é fornecido à próxima etapa. Uma falha imprevista numa das máquinas tende a interromper toda a cadeia produtiva. A manutenção preventiva agendada tem como objetivo evitar a ocorrência de falhas, tendo como base o tempo médio antes da falha (MTBF), que representa a expectativa média de vida de componentes individuais com base em dados históricos. As tarefas de manutenção podem implicar um período de paralisação e a interrupção da produção. Esta manutenção é executada rotineiramente e a substituição de componentes não considera a necessidade premente da sua substituição, sendo os mesmos substituídos com base no ciclo do agendamento. É aqui que a manutenção preditiva é aplicável. Efetuando a recolha de dados de sensores dos equipamentos, é possível detetar irregularidades nos dados recolhidos, através da aplicação de processos de raciocínio e inferência, conduzindo à atempada previsão e deteção de falhas. Levando este cenário à otimização do tempo de manutenção, evitando falhas inesperadas, à redução de custos e ao aumento da produtividade em comparação com a manutenção preventiva. Os dados fornecidos pelos sensores são sensíveis ao tempo, variações e flutuações ocorrem ao longo do tempo e devem ser analisados em relação ao período em que ocorrem. Esta dissertação tem como objetivo o desenvolvimento de uma ontologia para a manutenção preditiva que descreva a sua abrangência e o campo da sua aplicação. A aplicabilidade da ontologia será demonstrada com uma ferramenta, igualmente desenvolvida, que transforma dados sensíveis ao tempo recolhidos em tempo real a partir de sensores de máquinas industriais, fornecidos por WebServices, em indivíduos dessa mesma ontologia, considerando a representação do fator temporal dos dados.Manufacturing companies must ensure a continuous production process to be competitive and supply the manufactured goods in time and with the desired quality the customers expect. Any disruption in the manufacturing chain may have disastrous consequences, representing a shortage of production and the interruption of the supply chain. The manufacturing processes are composed of a chain of industrial machines operating in stages. Each machine has a specific task to complete, and the result of each stage is forwarded to the next stage. An unpredicted malfunction of one of the machines tends to interrupt the whole production chain. Scheduled Preventive maintenance intends to avoid causes leading to faults, but relies on parameters such as Mean Time Before Failure (MTBF), which represents the average expected life span of individual components based on statistical data. A maintenance task may lead to a period of downtime and consequently to a production halt. Being the maintenance scheduled and executed routinely, the replacement of components, does not consider the effective need of its replacement, they are replaced based on the scheduling cycle. This is where predictive maintenance is applicable. By collecting sensor data of industrial equipment, anomalies can be determined through reasoning and inference processes applied to the data, leading to an early fault and time to failure prediction. This scenario leads to maintenance timing optimization, avoidance of unexpected failures, cost savings and improved productivity when compared to preventive maintenance. Data supplied by sensors is timesensitive, as variations and fluctuations occur over periods of time and must be analysed concerning the period they occur. This dissertation aims to develop an ontology for predictive maintenance that describes the scope and field of application. The applicability of the ontology will be demonstrated with a tool, also to be developed, that transforms time-sensitive data collected in real time from sensors of industrial machines, provided by a WebServices, into individuals of the same ontology, considering the representation of the temporal factor of the data

Interoperability and machine-to-machine translation model with mappings to machine learning tasks

Modern large-scale automation systems integrate thousands to hundreds of thousands of physical sensors and actuators. Demands for more flexible reconfiguration of production systems and optimization across different information models, standards and legacy systems challenge current system interoperability concepts. Automatic semantic translation across information models and standards is an increasingly important problem that needs to be addressed to fulfill these demands in a cost-efficient manner under constraints of human capacity and resources in relation to timing requirements and system complexity. Here we define a translator-based operational interoperability model for interacting cyber-physical systems in mathematical terms, which includes system identification and ontology-based translation as special cases. We present alternative mathematical definitions of the translator learning task and mappings to similar machine learning tasks and solutions based on recent developments in machine learning. Possibilities to learn translators between artefacts without a common physical context, for example in simulations of digital twins and across layers of the automation pyramid are briefly discussed.Comment: 7 pages, 2 figures, 1 table, 1 listing. Submitted to the IEEE International Conference on Industrial Informatics 2019, INDIN'1

Open Research Knowledge Graph

As we mark the fifth anniversary of the alpha release of the Open Research Knowledge Graph (ORKG), it is both timely and exhilarating to celebrate the significant strides made in this pioneering project. We designed this book as a tribute to the evolution and achievements of the ORKG and as a practical guide encapsulating its essence in a form that resonates with both the general reader and the specialist. The ORKG has opened a new era in the way scholarly knowledge is curated, managed, and disseminated. By transforming vast arrays of unstructured narrative text into structured, machine-processable knowledge, the ORKG has emerged as an essential service with sophisticated functionalities. Over the past five years, our team has developed the ORKG into a vibrant platform that enhances the accessibility and visibility of scientific research. This book serves as a non-technical guide and a comprehensive reference for new and existing users that outlines the ORKG’s approach, technologies, and its role in revolutionizing scholarly communication. By elucidating how the ORKG facilitates the collection, enhancement, and sharing of knowledge, we invite readers to appreciate the value and potential of this groundbreaking digital tool presented in a tangible form. Looking ahead, we are thrilled to announce the upcoming unveiling of promising new features and tools at the fifth-year celebration of the ORKG’s alpha release. These innovations are set to redefine the boundaries of machine assistance enabled by research knowledge graphs. Among these enhancements, you can expect more intuitive interfaces that simplify the user experience, and enhanced machine learning models that improve the automation and accuracy of data curation. We also included a glossary tailored to clarifying key terms and concepts associated with the ORKG to ensure that all readers, regardless of their technical background, can fully engage with and understand the content presented. This book transcends the boundaries of a typical technical report. We crafted this as an inspiration for future applications, a testament to the ongoing evolution in scholarly communication that invites further collaboration and innovation. Let this book serve as both your guide and invitation to explore the ORKG as it continues to grow and shape the landscape of scientific inquiry and communication

A Semantic Web approach to ontology-based system: integrating, sharing and analysing IoT health and fitness data

With the rapid development of fitness industry, Internet of Things (IoT) technology is becoming one of the most popular trends for the health and fitness areas. IoT technologies have revolutionised the fitness and the sport industry by giving users the ability to monitor their health status and keep track of their training sessions. More and more sophisticated wearable devices, fitness trackers, smart watches and health mobile applications will appear in the near future. These systems do collect data non-stop from sensors and upload them to the Cloud. However, from a data-centric perspective the landscape of IoT fitness devices and wellness appliances is characterised by a plethora of representation and serialisation formats. The high heterogeneity of IoT data representations and the lack of common accepted standards, keep data isolated within each single system, preventing users and health professionals from having an integrated view of the various information collected. Moreover, in order to fully exploit the potential of the large amounts of data, it is also necessary to enable advanced analytics over it, thus achieving actionable knowledge. Therefore, due the above situation, the aim of this thesis project is to design and implement an ontology based system to (1) allow data interoperability among heterogeneous IoT fitness and wellness devices, (2) facilitate the integration and the sharing of information and (3) enable advanced analytics over the collected data (Cognitive Computing). The novelty of the proposed solution lies in exploiting Semantic Web technologies to formally describe the meaning of the data collected by the IoT devices and define a common communication strategy for information representation and exchange

BIM-to-BRICK: Using graph modeling for IoT/BMS and spatial semantic data interoperability within digital data models of buildings

The holistic management of a building requires data from heterogeneous sources such as building management systems (BMS), Internet-of-Things (IoT) sensor networks, and building information models. Data interoperability is a key component to eliminate silos of information, and using semantic web technologies like the BRICK schema, an effort to standardize semantic descriptions of the physical, logical, and virtual assets in buildings and the relationships between them, is a suitable approach. However, current data integration processes can involve significant manual interventions. This paper presents a methodology to automatically collect, assemble, and integrate information from a building information model to a knowledge graph. The resulting application, called BIM-to-BRICK, is run on the SDE4 building located in Singapore. BIM-to-BRICK generated a bidirectional link between a BIM model of 932 instances and experimental data collected for 17 subjects into 458 BRICK objects and 1219 relationships in 17 seconds. The automation of this approach can be compared to traditional manual mapping of data types. This scientific innovation incentivizes the convergence of disparate data types and structures in built-environment applications

Sharing Human-Generated Observations by Integrating HMI and the Semantic Sensor Web

Current “Internet of Things” concepts point to a future where connected objects gather meaningful information about their environment and share it with other objects and people. In particular, objects embedding Human Machine Interaction (HMI), such as mobile devices and, increasingly, connected vehicles, home appliances, urban interactive infrastructures, etc., may not only be conceived as sources of sensor information, but, through interaction with their users, they can also produce highly valuable context-aware human-generated observations. We believe that the great promise offered by combining and sharing all of the different sources of information available can be realized through the integration of HMI and Semantic Sensor Web technologies. This paper presents a technological framework that harmonizes two of the most influential HMI and Sensor Web initiatives: the W3C’s Multimodal Architecture and Interfaces (MMI) and the Open Geospatial Consortium (OGC) Sensor Web Enablement (SWE) with its semantic extension, respectively. Although the proposed framework is general enough to be applied in a variety of connected objects integrating HMI, a particular development is presented for a connected car scenario where drivers’ observations about the traffic or their environment are shared across the Semantic Sensor Web. For implementation and evaluation purposes an on-board OSGi (Open Services Gateway Initiative) architecture was built, integrating several available HMI, Sensor Web and Semantic Web technologies. A technical performance test and a conceptual validation of the scenario with potential users are reported, with results suggesting the approach is soun