Utilising semantic technologies for decision support in dementia care

The main objective of this work is to discuss our experience in utilising semantic technologies for building decision support in Dementia care systems that are based on the non-intrusive on the non-intrusive monitoring of the patient’s behaviour. Our approach adopts context-aware modelling of the patient’s condition to facilitate the analysis of the patient’s behaviour within the inhabited environment (movement and room occupancy patterns, use of equipment, etc.) with reference to the semantic knowledge about the patient’s condition (history of present of illness, dependable behaviour patterns, etc.). The reported work especially focuses on the critical role of the semantic reasoning engine in inferring medical advice, and by means of practical experimentation and critical analysis suggests important findings related to the methodology of deploying the appropriate semantic rules systems, and the dynamics of the efficient utilisation of complex event processing technology in order to the meet the requirements of decision support for remote healthcare systems

Ami-deu : un cadre sémantique pour des applications adaptables dans des environnements intelligents

Cette thèse vise à étendre l’utilisation de l'Internet des objets (IdO) en facilitant le développement d’applications par des personnes non experts en développement logiciel. La thèse propose une nouvelle approche pour augmenter la sémantique des applications d’IdO et l’implication des experts du domaine dans le développement d’applications sensibles au contexte. Notre approche permet de gérer le contexte changeant de l’environnement et de générer des applications qui s’exécutent dans plusieurs environnements intelligents pour fournir des actions requises dans divers contextes. Notre approche est mise en œuvre dans un cadriciel (AmI-DEU) qui inclut les composants pour le développement d’applications IdO. AmI-DEU intègre les services d’environnement, favorise l’interaction de l’utilisateur et fournit les moyens de représenter le domaine d’application, le profil de l’utilisateur et les intentions de l’utilisateur. Le cadriciel permet la définition d’applications IoT avec une intention d’activité autodécrite qui contient les connaissances requises pour réaliser l’activité. Ensuite, le cadriciel génère Intention as a Context (IaaC), qui comprend une intention d’activité autodécrite avec des connaissances colligées à évaluer pour une meilleure adaptation dans des environnements intelligents. La sémantique de l’AmI-DEU est basée sur celle du ContextAA (Context-Aware Agents) – une plateforme pour fournir une connaissance du contexte dans plusieurs environnements. Le cadriciel effectue une compilation des connaissances par des règles et l'appariement sémantique pour produire des applications IdO autonomes capables de s’exécuter en ContextAA. AmI- DEU inclut également un outil de développement visuel pour le développement et le déploiement rapide d'applications sur ContextAA. L'interface graphique d’AmI-DEU adopte la métaphore du flux avec des aides visuelles pour simplifier le développement d'applications en permettant des définitions de règles étape par étape. Dans le cadre de l’expérimentation, AmI-DEU comprend un banc d’essai pour le développement d’applications IdO. Les résultats expérimentaux montrent une optimisation sémantique potentielle des ressources pour les applications IoT dynamiques dans les maisons intelligentes et les villes intelligentes. Notre approche favorise l'adoption de la technologie pour améliorer le bienêtre et la qualité de vie des personnes. Cette thèse se termine par des orientations de recherche que le cadriciel AmI-DEU dévoile pour réaliser des environnements intelligents omniprésents fournissant des adaptations appropriées pour soutenir les intentions des personnes.Abstract: This thesis aims at expanding the use of the Internet of Things (IoT) by facilitating the development of applications by people who are not experts in software development. The thesis proposes a new approach to augment IoT applications’ semantics and domain expert involvement in context-aware application development. Our approach enables us to manage the changing environment context and generate applications that run in multiple smart environments to provide required actions in diverse settings. Our approach is implemented in a framework (AmI-DEU) that includes the components for IoT application development. AmI- DEU integrates environment services, promotes end-user interaction, and provides the means to represent the application domain, end-user profile, and end-user intentions. The framework enables the definition of IoT applications with a self-described activity intention that contains the required knowledge to achieve the activity. Then, the framework generates Intention as a Context (IaaC), which includes a self-described activity intention with compiled knowledge to be assessed for augmented adaptations in smart environments. AmI-DEU framework semantics adopts ContextAA (Context-Aware Agents) – a platform to provide context-awareness in multiple environments. The framework performs a knowledge compilation by rules and semantic matching to produce autonomic IoT applications to run in ContextAA. AmI-DEU also includes a visual tool for quick application development and deployment to ContextAA. The AmI-DEU GUI adopts the flow metaphor with visual aids to simplify developing applications by allowing step-by-step rule definitions. As part of the experimentation, AmI-DEU includes a testbed for IoT application development. Experimental results show a potential semantic optimization for dynamic IoT applications in smart homes and smart cities. Our approach promotes technology adoption to improve people’s well-being and quality of life. This thesis concludes with research directions that the AmI-DEU framework uncovers to achieve pervasive smart environments providing suitable adaptations to support people’s intentions

Real-Time Sensor Observation Segmentation For Complex Activity Recognition Within Smart Environments

The file attached to this record is the author's final peer reviewed versionActivity Recognition (AR) is at the heart of any types of assistive living systems. One of the key challenges faced in AR is segmentation of the sensor events when inhabitant performs simple or composite activities of daily living (ADLs). In addition, each inhabitant may follow a particular ritual or a tradition in performing different ADLs and their patterns may change overtime. Many recent studies apply methods to segment and recognise generic ADLs performed in a composite manner. However, little has been explored in semantically distinguishing individual sensor events and directly passing it to the relevant ongoing/new atomic activities. This paper proposes to use the ontological model to capture generic knowledge of ADLs and methods which also takes inhabitant-specific preferences into considerations when segmenting sensor events. The system implementation was developed, deployed and evaluated against 84 use case scenarios. The result suggests that all sensor events were adequately segmented with 98% accuracy and the average classification time of 3971ms and 62183ms for single and composite ADL scenarios were recorded, respectively

Representação da informação incerta por meio de ontologias: um framework para smart homes

Dissertação (mestrado) - Universidade Federal de Santa Catarina, Centro de Ciências da Educação, Programa de Pós-Graduação em Ciência da Informação, Florianópolis, 2019.Nas smart homes e outros cenários da Internet das Coisas (IoT), muitas vezes, as informações coletadas estão sujeitas a interferências externas. Além disso, pode ser necessário representar situações nas quais não é possível se obter informações completas ou precisas sobre determinado fenômeno, gerando a necessidade de se lidar com a informação incerta. As ontologias apresentam um formato amplamente utilizado para a representação das informações coletadas nas smart homes. Sendo assim, atualmente existem várias abordagens não padronizadas na literatura baseadas em ontologias para a representação da informação incerta, ou ontologias incertas . Diante desse contexto, o objetivo deste trabalho é propor um framework para ser utilizado como ferramenta de referência no processo de seleção de ontologias incertas para cenários de smart homes. Para isso, foram identificadas ontologias incertas para smart homes por meio de uma Revisão Sistemática da Literatura (RSL) e foram realizadas pesquisas nos anais do International Workshop on Uncertainty Reasoning for the Semantic Web (URSW). O framework proposto é composto por dois artefatos gerados a partir de informações extraídas das ontologias incertas identificadas: a) questionário para auxiliar na identificação das necessidades de representação da informação incerta; e b) quadro de referência para ser consultado durante a seleção de uma ontologia incerta de acordo com as necessidades de representação da informação incerta. Ao todo, foram identificados 16 trabalhos que propõem ontologias incertas. Com base nestes trabalhos, elaborou-se o questionário com seis questões e diferentes opções de respostas que remetem as ontologias incertas. O quadro de referência foi elaborado contendo os 16 trabalhos identificados e as características das ontologias incertas propostas por cada trabalho. O framework foi aplicado em nove cenários de smart homes que utilizam ontologias, mas não representam a informação incerta, de modo a exemplificar o papel do framework como ferramenta de referência. Como resultado de sua aplicação, para cada cenário, exceto um, identificou-se uma ou mais opções de ontologias incertas. Isto indica que as ontologias incertas disponíveis cobrem grande parte das necessidades de representação atualmente, mas não completamente. Espera-se que o framework proposto possa ser utilizado como referência para facilitar o acesso e uso das ontologias incertas pelos profissionais interessados na construção de ontologias. Finalmente, espera-se gerar oportunidades para que sejam desenvolvidas aplicações que elevem a qualidade e capacidade dos cenários de smart homes tendo em vista principalmente as necessidades e bem-estar das pessoas.Abstract : In smart homes and other Internet of Things (IoT) scenarios, often information collected is subject to external interference. Moreover, it may be necessary to represent situations in which it is not possible to obtain complete or accurate information about a specific phenomenon, causing the need to deal with uncertain information. Ontologies provides a widespread format for representing information collected in smart homes. This way, nowadays there are many non-standard ontology-based approaches in literature focused in the task of uncertain information representation, or \"uncertain ontologies\". Given this context, the objective of this work is to propose a framework to be used as a reference tool in the process of selecting uncertain ontologies for smart home scenarios. For this purpose, uncertain ontologies for smart homes and other IoT scenarios are identified by means of a Systematic Review of Literature (RSL) and by research in proceedings from International Workshop on Uncertainty Reasoning for the Semantic Web (URSW). The proposed framework is composed by two artifacts generated from information extracted from identified uncertain ontologies: a) a survey to assist in identifying the needs for representing uncertain information; and b) a reference table which can be used for selection of uncertain ontologies according to the representation needs. Altogether, 16 uncertain ontologies proposals have been identified. Based on these proposals, the questionnaire was elaborated with six questions and different options of answers referring to uncertain ontologies. The reference table was built containing the 16 ontologies proposals and its specific features. The framework was applied in nine scenarios of smart homes which use ontologies, but do not represent the uncertain information, in order to exemplify the role of the framework as a reference tool. As a result of its application one or more uncertain ontologies options were identified for most of the work. This indicates that the available uncertain ontologies cover most of the representation needs currently, but not all. It is expected that the proposed framework will be used as a reference to ease the access and use of uncertain ontologies by professionals interested in the creation of ontologies. Finally, it is expected to generate opportunities to develop applications which raise the quality and capacity of smart home scenarios especially in view of the needs and well-being of people

Inferring Complex Activities for Context-aware Systems within Smart Environments

The rising ageing population worldwide and the prevalence of age-related conditions such as physical fragility, mental impairments and chronic diseases have significantly impacted the quality of life and caused a shortage of health and care services. Over-stretched healthcare providers are leading to a paradigm shift in public healthcare provisioning. Thus, Ambient Assisted Living (AAL) using Smart Homes (SH) technologies has been rigorously investigated to help address the aforementioned problems. Human Activity Recognition (HAR) is a critical component in AAL systems which enables applications such as just-in-time assistance, behaviour analysis, anomalies detection and emergency notifications. This thesis is aimed at investigating challenges faced in accurately recognising Activities of Daily Living (ADLs) performed by single or multiple inhabitants within smart environments. Specifically, this thesis explores five complementary research challenges in HAR. The first study contributes to knowledge by developing a semantic-enabled data segmentation approach with user-preferences. The second study takes the segmented set of sensor data to investigate and recognise human ADLs at multi-granular action level; coarse- and fine-grained action level. At the coarse-grained actions level, semantic relationships between the sensor, object and ADLs are deduced, whereas, at fine-grained action level, object usage at the satisfactory threshold with the evidence fused from multimodal sensor data is leveraged to verify the intended actions. Moreover, due to imprecise/vague interpretations of multimodal sensors and data fusion challenges, fuzzy set theory and fuzzy web ontology language (fuzzy-OWL) are leveraged. The third study focuses on incorporating uncertainties caused in HAR due to factors such as technological failure, object malfunction, and human errors. Hence, existing studies uncertainty theories and approaches are analysed and based on the findings, probabilistic ontology (PR-OWL) based HAR approach is proposed. The fourth study extends the first three studies to distinguish activities conducted by more than one inhabitant in a shared smart environment with the use of discriminative sensor-based techniques and time-series pattern analysis. The final study investigates in a suitable system architecture with a real-time smart environment tailored to AAL system and proposes microservices architecture with sensor-based off-the-shelf and bespoke sensing methods. The initial semantic-enabled data segmentation study was evaluated with 100% and 97.8% accuracy to segment sensor events under single and mixed activities scenarios. However, the average classification time taken to segment each sensor events have suffered from 3971ms and 62183ms for single and mixed activities scenarios, respectively. The second study to detect fine-grained-level user actions was evaluated with 30 and 153 fuzzy rules to detect two fine-grained movements with a pre-collected dataset from the real-time smart environment. The result of the second study indicate good average accuracy of 83.33% and 100% but with the high average duration of 24648ms and 105318ms, and posing further challenges for the scalability of fusion rule creations. The third study was evaluated by incorporating PR-OWL ontology with ADL ontologies and Semantic-Sensor-Network (SSN) ontology to define four types of uncertainties presented in the kitchen-based activity. The fourth study illustrated a case study to extended single-user AR to multi-user AR by combining RFID tags and fingerprint sensors discriminative sensors to identify and associate user actions with the aid of time-series analysis. The last study responds to the computations and performance requirements for the four studies by analysing and proposing microservices-based system architecture for AAL system. A future research investigation towards adopting fog/edge computing paradigms from cloud computing is discussed for higher availability, reduced network traffic/energy, cost, and creating a decentralised system. As a result of the five studies, this thesis develops a knowledge-driven framework to estimate and recognise multi-user activities at fine-grained level user actions. This framework integrates three complementary ontologies to conceptualise factual, fuzzy and uncertainties in the environment/ADLs, time-series analysis and discriminative sensing environment. Moreover, a distributed software architecture, multimodal sensor-based hardware prototypes, and other supportive utility tools such as simulator and synthetic ADL data generator for the experimentation were developed to support the evaluation of the proposed approaches. The distributed system is platform-independent and currently supported by an Android mobile application and web-browser based client interfaces for retrieving information such as live sensor events and HAR results

A Hybrid Context-aware Middleware for Relevant Information Delivery in Multi-Role and Multi-User Monitoring Systems: An Application to the Building Management Domain

Recent advances in information and communications technology (ICT) have greatly extended capabilities and functionalities of control and monitoring systems including Building Management Systems (BMS). Specifically, it is now possible to integrate diverse set of devices and information systems providing heterogeneous data. This data, in turn, is now available on the higher levels of the system architectures, providing more information on the matter at hand and enabling principal possibility of better-informed decisions. Furthermore, the diversity and availability of information have made control and monitoring systems more attractive to new user groups, who now have the opportunity to find needed information, which was not available before. Thus, modern control and monitoring systems are well-equipped, multi-functional systems, which incorporate great number and variety of data sources and are used by multiple users with their special tasks and information needs.In theory, the diversity and availability of new data should lead to more informed users and better decisions. In practice, it overwhelms user capacities to perceive all available information and leads to the situations, where important data is hindered and lost, therefore complicating understanding of the ongoing status. Thus, there is a need in development of new solutions, which would reduce the unnecessary information burden to the users of the system, while keeping them well informed with respect to their personal needs and responsibilities.This dissertation proposes the middleware for relevant information delivery in multi-role and multi-user BMS, which is capable of analysing ongoing situations in the environment and delivering information personalized to specific user needs. The middleware implementation is based on a novel hybrid approach, which involve semantic modelling of the contextual information and fusion of this information with runtime device data by means of Complex Event Processing (CEP). The context model is actively used at the configuration stages of the middleware, which enables flexible redirection of information flows, simplified (re)configuration of the solution, and consideration of additional information at the runtime phases. The CEP utilizes contextual information and enables temporal reasoning support in combination with runtime analysis capabilities, thus processing ongoing data from devices and delivering personalized information flows. In addition, the work proposes classification and combination principles of ongoing system notifications, which further specialize information flows in accordance to user needs and environment status.The middleware and corresponding principles (e.g. knowledge modelling, classification and combination of ongoing notifications) have been designed contemplating the building management (BM) domain. A set of experiments on real data from rehabilitation facility has been carried out demonstrating applicability of the approach with respect to delivered information and performance considerations. It is expected that with minor modifications the approach has the potential of being adopted for control and monitoring systems of discrete manufacturing domain

Modélisation formelle des systèmes de détection d'intrusions

L’écosystème de la cybersécurité évolue en permanence en termes du nombre, de la diversité, et de la complexité des attaques. De ce fait, les outils de détection deviennent inefficaces face à certaines attaques. On distingue généralement trois types de systèmes de détection d’intrusions : détection par anomalies, détection par signatures et détection hybride. La détection par anomalies est fondée sur la caractérisation du comportement habituel du système, typiquement de manière statistique. Elle permet de détecter des attaques connues ou inconnues, mais génère aussi un très grand nombre de faux positifs. La détection par signatures permet de détecter des attaques connues en définissant des règles qui décrivent le comportement connu d’un attaquant. Cela demande une bonne connaissance du comportement de l’attaquant. La détection hybride repose sur plusieurs méthodes de détection incluant celles sus-citées. Elle présente l’avantage d’être plus précise pendant la détection. Des outils tels que Snort et Zeek offrent des langages de bas niveau pour l’expression de règles de reconnaissance d’attaques. Le nombre d’attaques potentielles étant très grand, ces bases de règles deviennent rapidement difficiles à gérer et à maintenir. De plus, l’expression de règles avec état dit stateful est particulièrement ardue pour reconnaître une séquence d’événements. Dans cette thèse, nous proposons une approche stateful basée sur les diagrammes d’état-transition algébriques (ASTDs) afin d’identifier des attaques complexes. Les ASTDs permettent de représenter de façon graphique et modulaire une spécification, ce qui facilite la maintenance et la compréhension des règles. Nous étendons la notation ASTD avec de nouvelles fonctionnalités pour représenter des attaques complexes. Ensuite, nous spécifions plusieurs attaques avec la notation étendue et exécutons les spécifications obtenues sur des flots d’événements à l’aide d’un interpréteur pour identifier des attaques. Nous évaluons aussi les performances de l’interpréteur avec des outils industriels tels que Snort et Zeek. Puis, nous réalisons un compilateur afin de générer du code exécutable à partir d’une spécification ASTD, capable d’identifier de façon efficiente les séquences d’événements.Abstract : The cybersecurity ecosystem continuously evolves with the number, the diversity, and the complexity of cyber attacks. Generally, we have three types of Intrusion Detection System (IDS) : anomaly-based detection, signature-based detection, and hybrid detection. Anomaly detection is based on the usual behavior description of the system, typically in a static manner. It enables detecting known or unknown attacks but also generating a large number of false positives. Signature based detection enables detecting known attacks by defining rules that describe known attacker’s behavior. It needs a good knowledge of attacker behavior. Hybrid detection relies on several detection methods including the previous ones. It has the advantage of being more precise during detection. Tools like Snort and Zeek offer low level languages to represent rules for detecting attacks. The number of potential attacks being large, these rule bases become quickly hard to manage and maintain. Moreover, the representation of stateful rules to recognize a sequence of events is particularly arduous. In this thesis, we propose a stateful approach based on algebraic state-transition diagrams (ASTDs) to identify complex attacks. ASTDs allow a graphical and modular representation of a specification, that facilitates maintenance and understanding of rules. We extend the ASTD notation with new features to represent complex attacks. Next, we specify several attacks with the extended notation and run the resulting specifications on event streams using an interpreter to identify attacks. We also evaluate the performance of the interpreter with industrial tools such as Snort and Zeek. Then, we build a compiler in order to generate executable code from an ASTD specification, able to efficiently identify sequences of events