Integration of multisensor hybrid reasoners to support personal autonomy in the smart home.

The deployment of the Ambient Intelligence (AmI) paradigm requires designing and integrating user-centered smart environments to assist people in their daily life activities. This research paper details an integration and validation of multiple heterogeneous sensors with hybrid reasoners that support decision making in order to monitor personal and environmental data at a smart home in a private way. The results innovate on knowledge-based platforms, distributed sensors, connected objects, accessibility and authentication methods to promote independent living for elderly people. TALISMAN+, the AmI framework deployed, integrates four subsystems in the smart home: (i) a mobile biomedical telemonitoring platform to provide elderly patients with continuous disease management; (ii) an integration middleware that allows context capture from heterogeneous sensors to program environment¿s reaction; (iii) a vision system for intelligent monitoring of daily activities in the home; and (iv) an ontologies-based integrated reasoning platform to trigger local actions and manage private information in the smart home. The framework was integrated in two real running environments, the UPM Accessible Digital Home and MetalTIC house, and successfully validated by five experts in home care, elderly people and personal autonomy

Ontology-based Activity Recognition Framework and Services

This paper introduces an ontology-based integrated framework for activity modeling, activity recognition and activity model evolution. Central to the framework is ontological activity modeling and semantic-based activity recognition, which is supported by an iterative process that incrementally improves the completeness and accuracy of activity models. In addition, the paper presents a service-oriented architecture for the realization of the proposed framework which can provide activity context-aware services in a scalable distributed manner. The paper further describes and discusses the implementation and testing experience of the framework and services in the context of smart home based assistive living

Inferring Complex Activities for Context-aware Systems within Smart Environments

The rising ageing population worldwide and the prevalence of age-related conditions such as physical fragility, mental impairments and chronic diseases have significantly impacted the quality of life and caused a shortage of health and care services. Over-stretched healthcare providers are leading to a paradigm shift in public healthcare provisioning. Thus, Ambient Assisted Living (AAL) using Smart Homes (SH) technologies has been rigorously investigated to help address the aforementioned problems. Human Activity Recognition (HAR) is a critical component in AAL systems which enables applications such as just-in-time assistance, behaviour analysis, anomalies detection and emergency notifications. This thesis is aimed at investigating challenges faced in accurately recognising Activities of Daily Living (ADLs) performed by single or multiple inhabitants within smart environments. Specifically, this thesis explores five complementary research challenges in HAR. The first study contributes to knowledge by developing a semantic-enabled data segmentation approach with user-preferences. The second study takes the segmented set of sensor data to investigate and recognise human ADLs at multi-granular action level; coarse- and fine-grained action level. At the coarse-grained actions level, semantic relationships between the sensor, object and ADLs are deduced, whereas, at fine-grained action level, object usage at the satisfactory threshold with the evidence fused from multimodal sensor data is leveraged to verify the intended actions. Moreover, due to imprecise/vague interpretations of multimodal sensors and data fusion challenges, fuzzy set theory and fuzzy web ontology language (fuzzy-OWL) are leveraged. The third study focuses on incorporating uncertainties caused in HAR due to factors such as technological failure, object malfunction, and human errors. Hence, existing studies uncertainty theories and approaches are analysed and based on the findings, probabilistic ontology (PR-OWL) based HAR approach is proposed. The fourth study extends the first three studies to distinguish activities conducted by more than one inhabitant in a shared smart environment with the use of discriminative sensor-based techniques and time-series pattern analysis. The final study investigates in a suitable system architecture with a real-time smart environment tailored to AAL system and proposes microservices architecture with sensor-based off-the-shelf and bespoke sensing methods. The initial semantic-enabled data segmentation study was evaluated with 100% and 97.8% accuracy to segment sensor events under single and mixed activities scenarios. However, the average classification time taken to segment each sensor events have suffered from 3971ms and 62183ms for single and mixed activities scenarios, respectively. The second study to detect fine-grained-level user actions was evaluated with 30 and 153 fuzzy rules to detect two fine-grained movements with a pre-collected dataset from the real-time smart environment. The result of the second study indicate good average accuracy of 83.33% and 100% but with the high average duration of 24648ms and 105318ms, and posing further challenges for the scalability of fusion rule creations. The third study was evaluated by incorporating PR-OWL ontology with ADL ontologies and Semantic-Sensor-Network (SSN) ontology to define four types of uncertainties presented in the kitchen-based activity. The fourth study illustrated a case study to extended single-user AR to multi-user AR by combining RFID tags and fingerprint sensors discriminative sensors to identify and associate user actions with the aid of time-series analysis. The last study responds to the computations and performance requirements for the four studies by analysing and proposing microservices-based system architecture for AAL system. A future research investigation towards adopting fog/edge computing paradigms from cloud computing is discussed for higher availability, reduced network traffic/energy, cost, and creating a decentralised system. As a result of the five studies, this thesis develops a knowledge-driven framework to estimate and recognise multi-user activities at fine-grained level user actions. This framework integrates three complementary ontologies to conceptualise factual, fuzzy and uncertainties in the environment/ADLs, time-series analysis and discriminative sensing environment. Moreover, a distributed software architecture, multimodal sensor-based hardware prototypes, and other supportive utility tools such as simulator and synthetic ADL data generator for the experimentation were developed to support the evaluation of the proposed approaches. The distributed system is platform-independent and currently supported by an Android mobile application and web-browser based client interfaces for retrieving information such as live sensor events and HAR results

Handling Real-World Context Awareness, Uncertainty and Vagueness in Real-Time Human Activity Tracking and Recognition with a Fuzzy Ontology-Based Hybrid Method

Human activity recognition is a key task in ambient intelligence applications to achieve proper ambient assisted living. There has been remarkable progress in this domain, but some challenges still remain to obtain robust methods. Our goal in this work is to provide a system that allows the modeling and recognition of a set of complex activities in real life scenarios involving interaction with the environment. The proposed framework is a hybrid model that comprises two main modules: a low level sub-activity recognizer, based on data-driven methods, and a high-level activity recognizer, implemented with a fuzzy ontology to include the semantic interpretation of actions performed by users. The fuzzy ontology is fed by the sub-activities recognized by the low level data-driven component and provides fuzzy ontological reasoning to recognize both the activities and their influence in the environment with semantics. An additional benefit of the approach is the ability to handle vagueness and uncertainty in the knowledge-based module, which substantially outperforms the treatment of incomplete and/or imprecise data with respect to classic crisp ontologies. We validate these advantages with the public CAD-120 dataset (Cornell Activity Dataset), achieving an accuracy of 90.1% and 91.07% for low-level and high-level activities, respectively. This entails an improvement over fully data-driven or ontology-based approaches.This work was funded by TUCS (Turku Centre for Computer Science), Finnish Cultural Foundation, Nokia Foundation, Google Anita Borg Scholarship, CEI BioTIC Project CEI2013-P-3, Contrato-Programa of Faculty of Education, Economy and Technology of Ceuta and Project TIN2012-30939 from National I+D Research Program (Spain). We also thank Fernando Bobillo for his support with FuzzyOWL and FuzzyDL tools

Ontology-driven monitoring of patient's vital signs enabling personalized medical detection and alert

A major challenge related to caring for patients with chronic conditions is the early detection of exacerbations of the disease. Medical personnel should be contacted immediately in order to intervene in time before an acute state is reached, ensuring patient safety. This paper proposes an approach to an ambient intelligence (AmI) framework supporting real-time remote monitoring of patients diagnosed with congestive heart failure (CHF). Its novelty is the integration of: (i) personalized monitoring of the patients health status and risk stage; (ii) intelligent alerting of the dedicated physician through the construction of medical workflows on-the-fly; and (iii) dynamic adaptation of the vital signs' monitoring environment on any available device or smart phone located in close proximity to the physician depending on new medical measurements, additional disease specifications or the failure of the infrastructure. The intelligence lies in the adoption of semantics providing for a personalized and automated emergency alerting that smoothly interacts with the physician, regardless of his location, ensuring timely intervention during an emergency. It is evaluated on a medical emergency scenario, where in the case of exceeded patient thresholds, medical personnel are localized and contacted, presenting ad hoc information on the patient's condition on the most suited device within the physician's reach

Environnement logiciel pour l assistance à l autonomie à domicile (gestion de la dynamique et de l incertitude pour la fourniture sémantique en temps réel de services d assistance)

L hétérogénéité des environnements ainsi que la diversité des profils et des besoins des patients représentent des contraintes majeures qui remettent en question l utilisation à grande échelle des systèmes d assistance à l autonomie à domicile (AAL). En effet, afin de répondre à l évolution de l état des patients et de leurs besoins humains, les environnements AAL sont en évolution continue par l introduction ou la disparition de capteurs, de dispositifs d interaction et de services d assistance. Par conséquent, une plateforme générique et dynamique capable de s adapter à différents environnements et d intégrer de nouveaux capteurs, dispositifs d interaction et services d assistance est requise. La mise en œuvre d un tel aspect dynamique peut produire une situation d incertitude dérivée des problèmes techniques liés à la fiabilité des capteurs ou à des problèmes de réseau. Par conséquent, la notion d incertitude doit être introduite dans la représentation de contexte et la prise de décision afin de faire face à ce problème. Au cours de cette thèse, j ai développé une plateforme dynamique et extensible capable de s adapter à différents environnements et aux besoins des patients. Ceci a été réalisé sur la base de l approche Plug&Play sémantique que j ai proposé. Afin de traiter le problème d incertitude de l information lié à des problèmes techniques, j ai proposé une approche de mesure d incertitude en utilisant les caractéristiques intrinsèques des capteurs et leurs comportements fonctionnels. J ai aussi fourni un modèle de représentation sémantique et de raisonnement avec incertitude associé avec la théorie de Dempster-Shafer (DST) pour la prise de décisionThe heterogeneity of the environments as well as the diversity of patients needs and profiles are major constraints that challenge the spread of ambient assistive living (AAL) systems. AAL environments are usually evolving by the introduction or the disappearance of sensors, devices and assistive services to respond to the evolution of patients conditions and human needs. Therefore, a generic framework that is able to adapt to such dynamic environments and to integrate new sensors, devices and assistive services at runtime is required. Implementing such a dynamic aspect may produce an uncertainty derived from technical problems related to sensors reliability or network problems. Therefore, a notion of uncertain should be introduced in context representation and decision making in order to deal with this problem. During this thesis, I have developed a dynamic and extendible framework able to adapt to different environments and patients needs. This was achieved based on my proposed approach of semantic Plug&Play mechanism. In order to handle the problem of uncertain information related to technical problems, I have proposed an approach for uncertainty measurement based on intrinsic characteristics of the sensors and their functional behaviors, then I have provided a model of semantic representation and reasoning under uncertainty coupled with the Dempster-Shafer Theory of evidence (DST) for decision makingEVRY-INT (912282302) / SudocSudocFranceF

Stratégies pour le raisonnement sur le contexte dans les environnements d assistance pour les personnes âgées

Tirant parti de notre expérience avec une approche traditionnelle des environnements d'assistance ambiante (AAL) qui repose sur l'utilisation de nombreuses technologies hétérogènes dans les déploiements, cette thèse étudie la possibilité d'une approche simplifiée et complémentaire, ou seul un sous-ensemble hardware réduit est déployé, initiant un transfert de complexité vers le côté logiciel. Axé sur les aspects de raisonnement dans les systèmes AAL, ce travail a permis à la proposition d'un moteur d'inférence sémantique adapté à l'utilisation particulière à ces systèmes, répondant ainsi à un besoin de la communauté scientifique. Prenant en compte la grossière granularité des données situationnelles disponible avec une telle approche, un ensemble de règles dédiées avec des stratégies d'inférence adaptées est proposé, implémenté et validé en utilisant ce moteur. Un mécanisme de raisonnement sémantique novateur est proposé sur la base d'une architecture de raisonnement inspiré du système cognitif. Enfin, le système de raisonnement est intégré dans un framework de provision de services sensible au contexte, se chargeant de l'intelligence vis-à-vis des données contextuelles en effectuant un traitement des événements en direct par des manipulations ontologiques complexes. L ensemble du système est validé par des déploiements in-situ dans une maison de retraite ainsi que dans des maisons privées, ce qui en soi est remarquable dans un domaine de recherche principalement cantonné aux laboratoiresLeveraging our experience with the traditional approach to ambient assisted living (AAL) which relies on a large spread of heterogeneous technologies in deployments, this thesis studies the possibility of a more stripped down and complementary approach, where only a reduced hardware subset is deployed, probing a transfer of complexity towards the software side, and enhancing the large scale deployability of the solution. Focused on the reasoning aspects in AAL systems, this work has allowed the finding of a suitable semantic inference engine for the peculiar use in these systems, responding to a need in this scientific community. Considering the coarse granularity of situational data available, dedicated rule-sets with adapted inference strategies are proposed, implemented, and validated using this engine. A novel semantic reasoning mechanism is proposed based on a cognitively inspired reasoning architecture. Finally, the whole reasoning system is integrated in a fully featured context-aware service framework, powering its context awareness by performing live event processing through complex ontological manipulation. the overall system is validated through in-situ deployments in a nursing home as well as private homes over a few months period, which itself is noticeable in a mainly laboratory-bound research domainEVRY-INT (912282302) / SudocSudocFranceF

Ambient-aware continuous care through semantic context dissemination

Background: The ultimate ambient-intelligent care room contains numerous sensors and devices to monitor the patient, sense and adjust the environment and support the staff. This sensor-based approach results in a large amount of data, which can be processed by current and future applications, e. g., task management and alerting systems. Today, nurses are responsible for coordinating all these applications and supplied information, which reduces the added value and slows down the adoption rate. The aim of the presented research is the design of a pervasive and scalable framework that is able to optimize continuous care processes by intelligently reasoning on the large amount of heterogeneous care data. Methods: The developed Ontology-based Care Platform (OCarePlatform) consists of modular components that perform a specific reasoning task. Consequently, they can easily be replicated and distributed. Complex reasoning is achieved by combining the results of different components. To ensure that the components only receive information, which is of interest to them at that time, they are able to dynamically generate and register filter rules with a Semantic Communication Bus (SCB). This SCB semantically filters all the heterogeneous care data according to the registered rules by using a continuous care ontology. The SCB can be distributed and a cache can be employed to ensure scalability. Results: A prototype implementation is presented consisting of a new-generation nurse call system supported by a localization and a home automation component. The amount of data that is filtered and the performance of the SCB are evaluated by testing the prototype in a living lab. The delay introduced by processing the filter rules is negligible when 10 or fewer rules are registered. Conclusions: The OCarePlatform allows disseminating relevant care data for the different applications and additionally supports composing complex applications from a set of smaller independent components. This way, the platform significantly reduces the amount of information that needs to be processed by the nurses. The delay resulting from processing the filter rules is linear in the amount of rules. Distributed deployment of the SCB and using a cache allows further improvement of these performance results