Inferring Complex Activities for Context-aware Systems within Smart Environments

The rising ageing population worldwide and the prevalence of age-related conditions such as physical fragility, mental impairments and chronic diseases have significantly impacted the quality of life and caused a shortage of health and care services. Over-stretched healthcare providers are leading to a paradigm shift in public healthcare provisioning. Thus, Ambient Assisted Living (AAL) using Smart Homes (SH) technologies has been rigorously investigated to help address the aforementioned problems. Human Activity Recognition (HAR) is a critical component in AAL systems which enables applications such as just-in-time assistance, behaviour analysis, anomalies detection and emergency notifications. This thesis is aimed at investigating challenges faced in accurately recognising Activities of Daily Living (ADLs) performed by single or multiple inhabitants within smart environments. Specifically, this thesis explores five complementary research challenges in HAR. The first study contributes to knowledge by developing a semantic-enabled data segmentation approach with user-preferences. The second study takes the segmented set of sensor data to investigate and recognise human ADLs at multi-granular action level; coarse- and fine-grained action level. At the coarse-grained actions level, semantic relationships between the sensor, object and ADLs are deduced, whereas, at fine-grained action level, object usage at the satisfactory threshold with the evidence fused from multimodal sensor data is leveraged to verify the intended actions. Moreover, due to imprecise/vague interpretations of multimodal sensors and data fusion challenges, fuzzy set theory and fuzzy web ontology language (fuzzy-OWL) are leveraged. The third study focuses on incorporating uncertainties caused in HAR due to factors such as technological failure, object malfunction, and human errors. Hence, existing studies uncertainty theories and approaches are analysed and based on the findings, probabilistic ontology (PR-OWL) based HAR approach is proposed. The fourth study extends the first three studies to distinguish activities conducted by more than one inhabitant in a shared smart environment with the use of discriminative sensor-based techniques and time-series pattern analysis. The final study investigates in a suitable system architecture with a real-time smart environment tailored to AAL system and proposes microservices architecture with sensor-based off-the-shelf and bespoke sensing methods. The initial semantic-enabled data segmentation study was evaluated with 100% and 97.8% accuracy to segment sensor events under single and mixed activities scenarios. However, the average classification time taken to segment each sensor events have suffered from 3971ms and 62183ms for single and mixed activities scenarios, respectively. The second study to detect fine-grained-level user actions was evaluated with 30 and 153 fuzzy rules to detect two fine-grained movements with a pre-collected dataset from the real-time smart environment. The result of the second study indicate good average accuracy of 83.33% and 100% but with the high average duration of 24648ms and 105318ms, and posing further challenges for the scalability of fusion rule creations. The third study was evaluated by incorporating PR-OWL ontology with ADL ontologies and Semantic-Sensor-Network (SSN) ontology to define four types of uncertainties presented in the kitchen-based activity. The fourth study illustrated a case study to extended single-user AR to multi-user AR by combining RFID tags and fingerprint sensors discriminative sensors to identify and associate user actions with the aid of time-series analysis. The last study responds to the computations and performance requirements for the four studies by analysing and proposing microservices-based system architecture for AAL system. A future research investigation towards adopting fog/edge computing paradigms from cloud computing is discussed for higher availability, reduced network traffic/energy, cost, and creating a decentralised system. As a result of the five studies, this thesis develops a knowledge-driven framework to estimate and recognise multi-user activities at fine-grained level user actions. This framework integrates three complementary ontologies to conceptualise factual, fuzzy and uncertainties in the environment/ADLs, time-series analysis and discriminative sensing environment. Moreover, a distributed software architecture, multimodal sensor-based hardware prototypes, and other supportive utility tools such as simulator and synthetic ADL data generator for the experimentation were developed to support the evaluation of the proposed approaches. The distributed system is platform-independent and currently supported by an Android mobile application and web-browser based client interfaces for retrieving information such as live sensor events and HAR results

First Responders' Localization and Health Monitoring During Rescue Operations

Currently, first responders’ coordination and decision-making during res-cue, firefighting or police operations is performed via radio/GSM channels with some support of video streaming. In unknown premises, officers have no global situational awareness on operation status, which reduces coordination efficiency and increases decision making mistakes. This paper pro-poses a solution enabling the situational awareness by introducing an integrated operation workflow for actors localization and health monitoring. The solution will provide global situational awareness to both coordinators and actors, thereby increasing efficiency of coordination, reducing mistakes in decision making and diminishing risks of unexpected situations to appear. This will result in faster operation progress, lower number of human casualties and financial losses and, the most important, saved human lives in calamity situations

A review of smart homes in healthcare

The technology of Smart Homes (SH), as an instance of ambient assisted living technologies, is designed to assist the homes’ residents accomplishing their daily-living activities and thus having a better quality of life while preserving their privacy. A SH system is usually equipped with a collection of inter-related software and hardware components to monitor the living space by capturing the behaviour of the resident and understanding his activities. By doing so the system can inform about risky situations and take actions on behalf of the resident to his satisfaction. The present survey will address technologies and analysis methods and bring examples of the state of the art research studies in order to provide background for the research community. In particular, the survey will expose infrastructure technologies such as sensors and communication platforms along with artificial intelligence techniques used for modeling and recognizing activities. A brief overview of approaches used to develop Human–Computer interfaces for SH systems is given. The survey also highlights the challenges and research trends in this area

A tailored smart home for dementia care

Dementia refers to a group of chronic conditions that cause the permanent and gradual cognitive decline. Therefore, a Person with Dementia (PwD) requires constant care from various types of caregivers (e.g., informal, social and formal). It is commonly accepted that utilising Smart Homes (SH), as an instance of Ambient Assisted Living (AAL) technologies, for dementia care could potentially facilitate the care and consequently improve the quality of PwDs’ well-being. Nevertheless, most of the studies view dementia care as a straight application of standard SH technology without accommodating the specific requirements of dementia care. A consequence of this approach is the inadequacy and unacceptability of generic SH systems for the stakeholders of dementia care. This work considers the specific requirements of PwDs and their care circle in all development steps of an SH, such as design, implementation, and evaluation. It investigates how utilising novel design and computing approaches can enhance the quality of SHs for dementia care and consequently improve healthcare and wellbeing of PwDs. To do so, the thesis first studies the existing SHs for healthcare and identifies their drawbacks. Then, the requirements of dementia care stakeholders will be collected, analysed and reflected on in an SH system design. Extensions and adaptation of existing frameworks and technologies will be proposed to implement a prototype based on the design. Finally, a series of thorough evaluations and validation of the prototype will be carried out

Qualitative spatial reasoning for activity recognition using tools of ambient intelligence

The aging population represents a growing concern of governments due to the extent that it will take in the coming decades and the speed of its evolution. This problem will result in increasing number of people affected by many diseases associated with aging such as the various types of dementia, including the sadly famous Alzheimer's disease. People with Alzheimer's must be assisted at all time during their everyday life. Technological assistance inside what is called a smart home could bring an affordable solution to solve this concern. One of the key issues to smart home assistance is to recognize the ongoing activities of everyday life made by the patient in order to be able to provide useful services at an appropriate moment. To do so, we must build a structured knowledge base of activities from which one or many intelligent agents (communicating with each other) would use information extracted from the various sensors to take a decision on what the inhabitant could be currently doing. The best way to build such an algorithm is to exploit constraints of different natures (logical, temporal, etc.) in order to circumscribe a library of activities. Many authors have emphasized the importance of the fundamental spatial aspect in activity recognition. However, only few works exist, and they are tested in a limited way that does not allow discerning the importance of dealing with space. Important spatial criterions, such as distance between objects, could help to reduce the number of hypotheses. Moreover, many errors can be detected only by using the spatial reasoning such as position problems (inappropriate objects are brought into the activity zone) or orientation of object issue (cup of coffee is upside down when pouring coffee). This thesis provides potential solutions to the problem outlined, which deals with spatial recognition of activities of daily living of a person with Alzheimer's disease. It proposes to adapt a theory of spatial reasoning, developed by Egenhofer, to a new model for recognition of activities. This new model allows identifying the ongoing activity using only qualitative spatial criterions which we demonstrate through the text that some could not have been identified otherwise. It also allows detection of new abnormalities related to the behavior of an individual in loss of autonomy. Finally, the model has been implemented and validated in carrying out activities in a smart home on the cutting edge of technology. These activities were derived from a clinical study with normal and mild to moderate Alzheimer subjects. The results were analyzed and compared with existing approaches to measure the contribution of this thesis. Le vieillissement de la population représente une préoccupation croissante des gouvernements en raison de l'ampleur qu'il prendra dans les prochaines décennies et la rapidité de son évolution. Ce problème se traduira par l'augmentation du nombre de personnes touchées par de nombreuses maladies liées au vieillissement telles que les différents types de démence, y compris la tristement célèbre maladie d'Alzheimer. Les personnes atteintes de la maladie d'Alzheimer doivent être assistées en tout temps dans leur vie quotidienne. L'assistance technologique à l'intérieur de ce qu'on appelle une maison intelligente pourrait apporter une solution abordable pour cette tâche. Une des questions clés inhérentes à ce type d'assistance est de reconnaître les activités courantes de la vie quotidienne faite par le patient afin d'être en mesure de fournir des services utiles au moment le plus opportun. Pour ce faire, nous devons construire une base de connaissances structurée à partir de laquelle un ou plusieurs agents intelligents utilisant l'information extraite des divers capteurs pour émettre une hypothèse ciblée concernant l'activité en cours de l'habitant. La meilleure façon de construire un tel algorithme est d'exploiter les contraintes de natures différentes (logique, temporelle, etc.) afin de circonscrire une bibliothèque d'activités. De nombreux auteurs ont souligné l'importance de l'aspect spatial fondamental dans la reconnaissance d'activité. Cependant, seuls quelques travaux existent, et ils sont testés de façon limitée qui ne permet pas de voir l'importance de considérer l'espace. Néanmoins, plusieurs critères spatiaux tels que la distance entre les objets pourraient aider à réduire le nombre d'hypothèses d'activités. Par ailleurs, de nombreuses erreurs peuvent être détectées uniquement en utilisant le raisonnement spatial, tel que les problèmes de type position ou d'orientation. Cette thèse fournit des pistes de solutions aux problèmes décrits, qui traitent de la reconnaissance spatiale des activités de la vie quotidienne d'une personne avec la maladie d'Alzheimer. Elle propose d'adapter une théorie du raisonnement spatial, développé par Egenhofer, à un nouveau modèle pour la reconnaissance des activités. Ce nouveau modèle permet d'identifier les activités en cours en utilisant uniquement les critères spatiaux. Nous démontrons à travers le texte que certaines activités ne pourraient pas avoir été identifiées autrement. Le modèle permet également la détection de nouvelles anomalies liées au comportement d'un individu en perte d'autonomie. Enfin, le modèle a été implémenté et validé en réalisant des activités dans un habitat intelligent à la fine pointe de la technologie. Ces activités ont été tirées d'une étude clinique avec des sujets normaux et Alzheimer. Les résultats ont été analysés et comparés avec les approches existantes pour évaluer la contribution de ce modèle

Probabilistic modelling and inference of human behaviour from mobile phone time series

With an estimated 4.1 billion subscribers around the world, the mobile phone offers a unique opportunity to sense and understand human behaviour from location, co-presence and communication data. While the benefit of modelling this unprecedented amount of data is widely recognised, a number of challenges impede the development of accurate behaviour models. In this thesis, we identify and address two modelling problems and show that their consideration improves the accuracy of behaviour inference. We first examine the modelling of long-range dependencies in human behaviour. Human behaviour models only take into account short-range dependencies in mobile phone time series. Using information theory, we quantify long-range dependencies in mobile phone time series for the first time, demonstrate that they exhibit periodic oscillations and introduce novel tools to analyse them. We further show that considering what the user did 24 hours earlier improves accuracy when predicting user behaviour five hours or longer in advance. The second problem that we address is the modelling of temporal variations in human behaviour. The time spent by a user on an activity varies from one day to the next. In order to recognise behaviour patterns despite temporal variations, we establish a methodological connection between human behaviour modelling and biological sequence alignment. This connection allows us to compare, cluster and model behaviour sequences and introduce novel features for behaviour recognition which improve its accuracy. The experiments presented in this thesis have been conducted on the largest publicly available mobile phone dataset labelled in an unsupervised fashion and are entirely repeatable. Furthermore, our techniques only require cellular data which can easily be recorded by today's mobile phones and could benefit a wide range of applications including life logging, health monitoring, customer profiling and large-scale surveillance

Data Science and Knowledge Discovery

Data Science (DS) is gaining significant importance in the decision process due to a mix of various areas, including Computer Science, Machine Learning, Math and Statistics, domain/business knowledge, software development, and traditional research. In the business field, DS's application allows using scientific methods, processes, algorithms, and systems to extract knowledge and insights from structured and unstructured data to support the decision process. After collecting the data, it is crucial to discover the knowledge. In this step, Knowledge Discovery (KD) tasks are used to create knowledge from structured and unstructured sources (e.g., text, data, and images). The output needs to be in a readable and interpretable format. It must represent knowledge in a manner that facilitates inferencing. KD is applied in several areas, such as education, health, accounting, energy, and public administration. This book includes fourteen excellent articles which discuss this trending topic and present innovative solutions to show the importance of Data Science and Knowledge Discovery to researchers, managers, industry, society, and other communities. The chapters address several topics like Data mining, Deep Learning, Data Visualization and Analytics, Semantic data, Geospatial and Spatio-Temporal Data, Data Augmentation and Text Mining