Inferring Complex Activities for Context-aware Systems within Smart Environments

The rising ageing population worldwide and the prevalence of age-related conditions such as physical fragility, mental impairments and chronic diseases have significantly impacted the quality of life and caused a shortage of health and care services. Over-stretched healthcare providers are leading to a paradigm shift in public healthcare provisioning. Thus, Ambient Assisted Living (AAL) using Smart Homes (SH) technologies has been rigorously investigated to help address the aforementioned problems. Human Activity Recognition (HAR) is a critical component in AAL systems which enables applications such as just-in-time assistance, behaviour analysis, anomalies detection and emergency notifications. This thesis is aimed at investigating challenges faced in accurately recognising Activities of Daily Living (ADLs) performed by single or multiple inhabitants within smart environments. Specifically, this thesis explores five complementary research challenges in HAR. The first study contributes to knowledge by developing a semantic-enabled data segmentation approach with user-preferences. The second study takes the segmented set of sensor data to investigate and recognise human ADLs at multi-granular action level; coarse- and fine-grained action level. At the coarse-grained actions level, semantic relationships between the sensor, object and ADLs are deduced, whereas, at fine-grained action level, object usage at the satisfactory threshold with the evidence fused from multimodal sensor data is leveraged to verify the intended actions. Moreover, due to imprecise/vague interpretations of multimodal sensors and data fusion challenges, fuzzy set theory and fuzzy web ontology language (fuzzy-OWL) are leveraged. The third study focuses on incorporating uncertainties caused in HAR due to factors such as technological failure, object malfunction, and human errors. Hence, existing studies uncertainty theories and approaches are analysed and based on the findings, probabilistic ontology (PR-OWL) based HAR approach is proposed. The fourth study extends the first three studies to distinguish activities conducted by more than one inhabitant in a shared smart environment with the use of discriminative sensor-based techniques and time-series pattern analysis. The final study investigates in a suitable system architecture with a real-time smart environment tailored to AAL system and proposes microservices architecture with sensor-based off-the-shelf and bespoke sensing methods. The initial semantic-enabled data segmentation study was evaluated with 100% and 97.8% accuracy to segment sensor events under single and mixed activities scenarios. However, the average classification time taken to segment each sensor events have suffered from 3971ms and 62183ms for single and mixed activities scenarios, respectively. The second study to detect fine-grained-level user actions was evaluated with 30 and 153 fuzzy rules to detect two fine-grained movements with a pre-collected dataset from the real-time smart environment. The result of the second study indicate good average accuracy of 83.33% and 100% but with the high average duration of 24648ms and 105318ms, and posing further challenges for the scalability of fusion rule creations. The third study was evaluated by incorporating PR-OWL ontology with ADL ontologies and Semantic-Sensor-Network (SSN) ontology to define four types of uncertainties presented in the kitchen-based activity. The fourth study illustrated a case study to extended single-user AR to multi-user AR by combining RFID tags and fingerprint sensors discriminative sensors to identify and associate user actions with the aid of time-series analysis. The last study responds to the computations and performance requirements for the four studies by analysing and proposing microservices-based system architecture for AAL system. A future research investigation towards adopting fog/edge computing paradigms from cloud computing is discussed for higher availability, reduced network traffic/energy, cost, and creating a decentralised system. As a result of the five studies, this thesis develops a knowledge-driven framework to estimate and recognise multi-user activities at fine-grained level user actions. This framework integrates three complementary ontologies to conceptualise factual, fuzzy and uncertainties in the environment/ADLs, time-series analysis and discriminative sensing environment. Moreover, a distributed software architecture, multimodal sensor-based hardware prototypes, and other supportive utility tools such as simulator and synthetic ADL data generator for the experimentation were developed to support the evaluation of the proposed approaches. The distributed system is platform-independent and currently supported by an Android mobile application and web-browser based client interfaces for retrieving information such as live sensor events and HAR results

Goal Lifecycles and Ontological Models for Intention Based Assistive Living within Smart Environments

Current ambient assistive living solutions have adopted a traditional sensor-centric approach, involving data analysis and activity recognition to provide assistance to individuals. The reliance on sensors and activity recognition in this approach introduces issues with scalability and ability to model activity variations. This study introduces a novel approach to assistive living which intends to address these issues via a paradigm shift from a sensor centric approach to a goal-oriented one. The goal-oriented approach focuses on identification of user goals in order to pro-actively offer assistance by either pre-defined or dynamically constructed instructions. This paper introduces the architecture of this goal-oriented approach and describes an ontological goal model to serve as its basis. The use of this approach is illustrated in a case study which focuses on assisting a user with activities of daily living

From Activity Recognition to Intention Recognition for Assisted Living Within Smart Homes

The file attached to this record is the author's final peer reviewed version. The Publisher's final version can be found by following the DOI link.The global population is aging; projections show that by 2050, more than 20% of the population will be aged over 64. This will lead to an increase in aging related illness, a decrease in informal support, and ultimately issues with providing care for these individuals. Assistive smart homes provide a promising solution to some of these issues. Nevertheless, they currently have issues hindering their adoption. To help address some of these issues, this study introduces a novel approach to implementing assistive smart homes. The devised approach is based upon an intention recognition mechanism incorporated into an intelligent agent architecture. This approach is detailed and evaluated. Evaluation was performed across three scenarios. Scenario 1 involved a web interface, focusing on testing the intention recognition mechanism. Scenarios 2 and 3 involved retrofitting a home with sensors and providing assistance with activities over a period of 3 months. The average accuracy for these three scenarios was 100%, 64.4%, and 83.3%, respectively. Future will extend and further evaluate this approach by implementing advanced sensor-filtering rules and evaluating more complex activities

Representação da informação incerta por meio de ontologias: um framework para smart homes

Dissertação (mestrado) - Universidade Federal de Santa Catarina, Centro de Ciências da Educação, Programa de Pós-Graduação em Ciência da Informação, Florianópolis, 2019.Nas smart homes e outros cenários da Internet das Coisas (IoT), muitas vezes, as informações coletadas estão sujeitas a interferências externas. Além disso, pode ser necessário representar situações nas quais não é possível se obter informações completas ou precisas sobre determinado fenômeno, gerando a necessidade de se lidar com a informação incerta. As ontologias apresentam um formato amplamente utilizado para a representação das informações coletadas nas smart homes. Sendo assim, atualmente existem várias abordagens não padronizadas na literatura baseadas em ontologias para a representação da informação incerta, ou ontologias incertas . Diante desse contexto, o objetivo deste trabalho é propor um framework para ser utilizado como ferramenta de referência no processo de seleção de ontologias incertas para cenários de smart homes. Para isso, foram identificadas ontologias incertas para smart homes por meio de uma Revisão Sistemática da Literatura (RSL) e foram realizadas pesquisas nos anais do International Workshop on Uncertainty Reasoning for the Semantic Web (URSW). O framework proposto é composto por dois artefatos gerados a partir de informações extraídas das ontologias incertas identificadas: a) questionário para auxiliar na identificação das necessidades de representação da informação incerta; e b) quadro de referência para ser consultado durante a seleção de uma ontologia incerta de acordo com as necessidades de representação da informação incerta. Ao todo, foram identificados 16 trabalhos que propõem ontologias incertas. Com base nestes trabalhos, elaborou-se o questionário com seis questões e diferentes opções de respostas que remetem as ontologias incertas. O quadro de referência foi elaborado contendo os 16 trabalhos identificados e as características das ontologias incertas propostas por cada trabalho. O framework foi aplicado em nove cenários de smart homes que utilizam ontologias, mas não representam a informação incerta, de modo a exemplificar o papel do framework como ferramenta de referência. Como resultado de sua aplicação, para cada cenário, exceto um, identificou-se uma ou mais opções de ontologias incertas. Isto indica que as ontologias incertas disponíveis cobrem grande parte das necessidades de representação atualmente, mas não completamente. Espera-se que o framework proposto possa ser utilizado como referência para facilitar o acesso e uso das ontologias incertas pelos profissionais interessados na construção de ontologias. Finalmente, espera-se gerar oportunidades para que sejam desenvolvidas aplicações que elevem a qualidade e capacidade dos cenários de smart homes tendo em vista principalmente as necessidades e bem-estar das pessoas.Abstract : In smart homes and other Internet of Things (IoT) scenarios, often information collected is subject to external interference. Moreover, it may be necessary to represent situations in which it is not possible to obtain complete or accurate information about a specific phenomenon, causing the need to deal with uncertain information. Ontologies provides a widespread format for representing information collected in smart homes. This way, nowadays there are many non-standard ontology-based approaches in literature focused in the task of uncertain information representation, or \"uncertain ontologies\". Given this context, the objective of this work is to propose a framework to be used as a reference tool in the process of selecting uncertain ontologies for smart home scenarios. For this purpose, uncertain ontologies for smart homes and other IoT scenarios are identified by means of a Systematic Review of Literature (RSL) and by research in proceedings from International Workshop on Uncertainty Reasoning for the Semantic Web (URSW). The proposed framework is composed by two artifacts generated from information extracted from identified uncertain ontologies: a) a survey to assist in identifying the needs for representing uncertain information; and b) a reference table which can be used for selection of uncertain ontologies according to the representation needs. Altogether, 16 uncertain ontologies proposals have been identified. Based on these proposals, the questionnaire was elaborated with six questions and different options of answers referring to uncertain ontologies. The reference table was built containing the 16 ontologies proposals and its specific features. The framework was applied in nine scenarios of smart homes which use ontologies, but do not represent the uncertain information, in order to exemplify the role of the framework as a reference tool. As a result of its application one or more uncertain ontologies options were identified for most of the work. This indicates that the available uncertain ontologies cover most of the representation needs currently, but not all. It is expected that the proposed framework will be used as a reference to ease the access and use of uncertain ontologies by professionals interested in the creation of ontologies. Finally, it is expected to generate opportunities to develop applications which raise the quality and capacity of smart home scenarios especially in view of the needs and well-being of people

Rover-II: A Context-Aware Middleware for Pervasive Computing Environment

It is well recognized that context plays a significant role in all human endeavors. All decisions are based on information which has to be interpreted in context. By making information systems context-aware we can have systems that significantly enhance human capabilities to make critical decisions. A major challenge of context-aware systems is to balance usability with generality and extensibility. The relevant context changes depending on the particular application. The model used to represent the context and its relationship to entities must be general enough to allow additions of context categories without redesign while remaining usable across many applications. Also, while efforts are put in by application designers and developers to make applications context-aware, these efforts are customized to specific needs of the target application, and only certain common contexts like location and time are taken into account. Therefore, a general framework is called for that can (i) efficiently maintain, represent and integrate contextual information, (ii) act as an integration platform where different applications can share contexts and (iii) provide relevant services to make efficient use of the contextual information. This dissertation presents: * a generic and effective context model - Rover Context Model (RoCoM) that is structured around four primitives: entities, events, relationships, and activities; and practically usable through the concept of templates, * a flexible, extensible and generic ontology - Rover Context Model Ontology (RoCoMO) supporting the model, that addresses the shortcomings of existing ontologies, * an effective mechanism of modeling the context of a situation, through the concept of relevant context, with the help of situation graph, efficiently handling and making best use of context information, * a context middleware - Rover-II, which serves as a framework for contextual information integration, that could be used not just to store and compile the contextual information, but also integrate relevant services to enhance the context information; and more importantly, enable sharing of context among the applications subscribed to it, * the initial design and implementation of a distributed architecture for Rover-II, following a P2P arrangement inspired from Tapestry, The above concepts are illustrated through M-Urgency, a context-aware public safety system that has been deployed at the University of Maryland Police Department

Semantic adaptability for the systems interoperability

In the current global and competitive business context, it is essential that enterprises adapt their knowledge resources in order to smoothly interact and collaborate with others. However, due to the existent multiculturalism of people and enterprises, there are different representation views of business processes or products, even inside a same domain. Consequently, one of the main problems found in the interoperability between enterprise systems and applications is related to semantics. The integration and sharing of enterprises knowledge to build a common lexicon, plays an important role to the semantic adaptability of the information systems. The author proposes a framework to support the development of systems to manage dynamic semantic adaptability resolution. It allows different organisations to participate in a common knowledge base building, letting at the same time maintain their own views of the domain, without compromising the integration between them. Thus, systems are able to be aware of new knowledge, and have the capacity to learn from it and to manage its semantic interoperability in a dynamic and adaptable way. The author endorses the vision that in the near future, the semantic adaptability skills of the enterprise systems will be the booster to enterprises collaboration and the appearance of new business opportunities

Semantics-Empowered Big Data Processing with Applications

We discuss the nature of Big Data and address the role of semantics in analyzing and processing Big Data that arises in the context of Physical-Cyber-Social Systems. We organize our research around the Five Vs of Big Data, where four of the Vs are harnessed to produce the fifth V - value. To handle the challenge of Volume, we advocate semantic perception that can convert low-level observational data to higher-level abstractions more suitable for decision-making. To handle the challenge of Variety, we resort to the use of semantic models and annotations of data so that much of the intelligent processing can be done at a level independent of heterogeneity of data formats and media. To handle the challenge of Velocity, we seek to use continuous semantics capability to dynamically create event or situation specific models and recognize relevant new concepts, entities and facts. To handle Veracity, we explore the formalization of trust models and approaches to glean trustworthiness. The above four Vs of Big Data are harnessed by the semantics-empowered analytics to derive value for supporting practical applications transcending physical-cyber-social continuum

Stratégies pour le raisonnement sur le contexte dans les environnements d assistance pour les personnes âgées

Tirant parti de notre expérience avec une approche traditionnelle des environnements d'assistance ambiante (AAL) qui repose sur l'utilisation de nombreuses technologies hétérogènes dans les déploiements, cette thèse étudie la possibilité d'une approche simplifiée et complémentaire, ou seul un sous-ensemble hardware réduit est déployé, initiant un transfert de complexité vers le côté logiciel. Axé sur les aspects de raisonnement dans les systèmes AAL, ce travail a permis à la proposition d'un moteur d'inférence sémantique adapté à l'utilisation particulière à ces systèmes, répondant ainsi à un besoin de la communauté scientifique. Prenant en compte la grossière granularité des données situationnelles disponible avec une telle approche, un ensemble de règles dédiées avec des stratégies d'inférence adaptées est proposé, implémenté et validé en utilisant ce moteur. Un mécanisme de raisonnement sémantique novateur est proposé sur la base d'une architecture de raisonnement inspiré du système cognitif. Enfin, le système de raisonnement est intégré dans un framework de provision de services sensible au contexte, se chargeant de l'intelligence vis-à-vis des données contextuelles en effectuant un traitement des événements en direct par des manipulations ontologiques complexes. L ensemble du système est validé par des déploiements in-situ dans une maison de retraite ainsi que dans des maisons privées, ce qui en soi est remarquable dans un domaine de recherche principalement cantonné aux laboratoiresLeveraging our experience with the traditional approach to ambient assisted living (AAL) which relies on a large spread of heterogeneous technologies in deployments, this thesis studies the possibility of a more stripped down and complementary approach, where only a reduced hardware subset is deployed, probing a transfer of complexity towards the software side, and enhancing the large scale deployability of the solution. Focused on the reasoning aspects in AAL systems, this work has allowed the finding of a suitable semantic inference engine for the peculiar use in these systems, responding to a need in this scientific community. Considering the coarse granularity of situational data available, dedicated rule-sets with adapted inference strategies are proposed, implemented, and validated using this engine. A novel semantic reasoning mechanism is proposed based on a cognitively inspired reasoning architecture. Finally, the whole reasoning system is integrated in a fully featured context-aware service framework, powering its context awareness by performing live event processing through complex ontological manipulation. the overall system is validated through in-situ deployments in a nursing home as well as private homes over a few months period, which itself is noticeable in a mainly laboratory-bound research domainEVRY-INT (912282302) / SudocSudocFranceF

Viewpoints on emergent semantics

Authors include:Philippe Cudr´e-Mauroux, and Karl Aberer (editors), Alia I. Abdelmoty, Tiziana Catarci, Ernesto Damiani, Arantxa Illaramendi, Robert Meersman, Erich J. Neuhold, Christine Parent, Kai-Uwe Sattler, Monica Scannapieco, Stefano Spaccapietra, Peter Spyns, and Guy De Tr´eWe introduce a novel view on how to deal with the problems of semantic interoperability in distributed systems. This view is based on the concept of emergent semantics, which sees both the representation of semantics and the discovery of the proper interpretation of symbols as the result of a self-organizing process performed by distributed agents exchanging symbols and having utilities dependent on the proper interpretation of the symbols. This is a complex systems perspective on the problem of dealing with semantics. We highlight some of the distinctive features of our vision and point out preliminary examples of its applicatio