Inferring Complex Activities for Context-aware Systems within Smart Environments

The rising ageing population worldwide and the prevalence of age-related conditions such as physical fragility, mental impairments and chronic diseases have significantly impacted the quality of life and caused a shortage of health and care services. Over-stretched healthcare providers are leading to a paradigm shift in public healthcare provisioning. Thus, Ambient Assisted Living (AAL) using Smart Homes (SH) technologies has been rigorously investigated to help address the aforementioned problems. Human Activity Recognition (HAR) is a critical component in AAL systems which enables applications such as just-in-time assistance, behaviour analysis, anomalies detection and emergency notifications. This thesis is aimed at investigating challenges faced in accurately recognising Activities of Daily Living (ADLs) performed by single or multiple inhabitants within smart environments. Specifically, this thesis explores five complementary research challenges in HAR. The first study contributes to knowledge by developing a semantic-enabled data segmentation approach with user-preferences. The second study takes the segmented set of sensor data to investigate and recognise human ADLs at multi-granular action level; coarse- and fine-grained action level. At the coarse-grained actions level, semantic relationships between the sensor, object and ADLs are deduced, whereas, at fine-grained action level, object usage at the satisfactory threshold with the evidence fused from multimodal sensor data is leveraged to verify the intended actions. Moreover, due to imprecise/vague interpretations of multimodal sensors and data fusion challenges, fuzzy set theory and fuzzy web ontology language (fuzzy-OWL) are leveraged. The third study focuses on incorporating uncertainties caused in HAR due to factors such as technological failure, object malfunction, and human errors. Hence, existing studies uncertainty theories and approaches are analysed and based on the findings, probabilistic ontology (PR-OWL) based HAR approach is proposed. The fourth study extends the first three studies to distinguish activities conducted by more than one inhabitant in a shared smart environment with the use of discriminative sensor-based techniques and time-series pattern analysis. The final study investigates in a suitable system architecture with a real-time smart environment tailored to AAL system and proposes microservices architecture with sensor-based off-the-shelf and bespoke sensing methods. The initial semantic-enabled data segmentation study was evaluated with 100% and 97.8% accuracy to segment sensor events under single and mixed activities scenarios. However, the average classification time taken to segment each sensor events have suffered from 3971ms and 62183ms for single and mixed activities scenarios, respectively. The second study to detect fine-grained-level user actions was evaluated with 30 and 153 fuzzy rules to detect two fine-grained movements with a pre-collected dataset from the real-time smart environment. The result of the second study indicate good average accuracy of 83.33% and 100% but with the high average duration of 24648ms and 105318ms, and posing further challenges for the scalability of fusion rule creations. The third study was evaluated by incorporating PR-OWL ontology with ADL ontologies and Semantic-Sensor-Network (SSN) ontology to define four types of uncertainties presented in the kitchen-based activity. The fourth study illustrated a case study to extended single-user AR to multi-user AR by combining RFID tags and fingerprint sensors discriminative sensors to identify and associate user actions with the aid of time-series analysis. The last study responds to the computations and performance requirements for the four studies by analysing and proposing microservices-based system architecture for AAL system. A future research investigation towards adopting fog/edge computing paradigms from cloud computing is discussed for higher availability, reduced network traffic/energy, cost, and creating a decentralised system. As a result of the five studies, this thesis develops a knowledge-driven framework to estimate and recognise multi-user activities at fine-grained level user actions. This framework integrates three complementary ontologies to conceptualise factual, fuzzy and uncertainties in the environment/ADLs, time-series analysis and discriminative sensing environment. Moreover, a distributed software architecture, multimodal sensor-based hardware prototypes, and other supportive utility tools such as simulator and synthetic ADL data generator for the experimentation were developed to support the evaluation of the proposed approaches. The distributed system is platform-independent and currently supported by an Android mobile application and web-browser based client interfaces for retrieving information such as live sensor events and HAR results

A general cognitive framework for context-aware systems: extensions and applications for high level information fusion approaches

Mención Internacional en el título de doctorContext-aware systems aims at the development of computational systems that process data acquired from different datasources and adapt their behaviour in order to provide the 'right' information, at the 'right' time, in the 'right' place, in the 'right' way to the 'right' person (Fischer, 2012). Traditionally computational research has tried to answer these needs by means of low-level algorithms. In the last years the combination of numeric and symbolic approaches has offered the opportunity to create systems to deal with these issues. However, although the performance of algorithms and the quality of the data directly provided by computers and devices has quickly improved, symbolic models used to represent the resulting knowledge have not yet been adapted to smart environments. This lack of representation does not allow to take advantage of the semantic quality of the information provided by new sensors. This dissertation proposes a set of extensions and applications focused on a cognitive framework for the implementation of context-aware systems based on a general model inspired by the Information Fusion paradigm. This model is stepped in several abstraction levels from low-level raw data to high level scene interpretation whose structure is determined by a set of ontologies. Each ontology level provides a skeleton that includes general concepts and relations to describe entities and their connections. This structure has been designed to promote extensibility and modularity, and might be refined to apply this model in specific domains. This framework combines a priori context knowledge represented with ontologies with real data coming from sensors to support logic-based high-level interpretation of the current situation and to automatically generate feedback recommendations to adjust data acquisition procedures. This work advocates for the introduction of general purpose cognitive layers in order to obtain a closer representation to the human cognition, generate additional knowledge and improve the high-level interpretation. Extensibility and adaptability of the basic ontology levels is demonstrated with the introduction of these traverse semantic layers which are able to be present and represent information at several granularity levels of knowledge using a common formalism. Context-based system must be able to reason about uncertainty. However the reasoning associated to ontologies has been limited to classical description logic mechanisms. This research also tackle the problem of reasoning under uncertainty circumstances through a logic-based paradigm for abductive reasoning: the Belief-Argumentation System. The main contribution of this dissertation is the adaptation of the general architecture and the theoretical proposals to several context-aware application areas such as Ambient Intelligence, Social Signal Processing and surveillance systems. The implementation of prototypes and examples for these areas are explained along this dissertation to progressively illustrate the improvements and extensions in the framework. To initially depict the general model, its components and the basic reasoning mechanisms a video-based Ambient Intelligence application is presented. The advantages and features of the framework extensions through traverse cognitive layers are demonstrated in a Social Signal Processing case for the elaboration of automatic market researches. Finally, the functioning of the system under uncertainty circumstances is illustrated with several examples to support decision makers in the detection of potential threats in common harbor scenarios.Programa Oficial de Doctorado en Ciencia y Tecnología InformáticaPresidente: José Manuel Molina López.- Secretario: Ángel Arroyo.- Vocal: Nayat Sánchez P

Semantic Smart Homes: Towards Knowledge Rich Assisted Living Environments

International audienceThe complexity of the Emergency Supply Chains makes its management very difficult. Hence, we present in this article a comprehensive view of the French emergency supply chain (ESC), we propose an ad hoc relationship model between actors, and a GRAI grid-based model to initiate a new approach for controlling the ESC deficiencies, especially related to decision making. Throughout the article, we discuss the interest of the use of enterprise modelling to model the ESC. We discuss too, the characterization of the different issues related to the steering of the ESC. A literature review based on the GRAI grid model is proposed and discussed too. The GRAI method is used here because it presents the advantage of using the theory of complex systems, and it provides a dynamic model of an organization by focusing on decision-making and decisions communication

Integration of discriminative and generative models for activity recognition in smart homes

Activity recognition in smart homes enables the remote monitoring of elderly and patients. In healthcare systems, reliability of a recognition model is of high importance. Limited amount of training data and imbalanced number of activity instances result in over-fitting thus making recognition models inconsistent. In this paper, we propose an activity recognition approach that integrates the distance minimization (DM) and probability estimation (PE) approaches to improve the reliability of recognitions. DM uses distances of instances from the mean representation of each activity class for label assignment. DM is useful in avoiding decision biasing towards the activity class with majority instances; however, DM can result in over-fitting. PE on the other hand has good generalization abilities. PE measures the probability of correct assignments from the obtained distances, while it requires a large amount of data for training. We apply data oversampling to improve the representation of classes with less number of instances. Support vector machine (SVM) is applied to combine the outputs of both DM and PE, since SVM performs better with imbalanced data and further improves the generalization ability of the approach. The proposed approach is evaluated using five publicly available smart home datasets. The results demonstrate better performance of the proposed approach compared to the state-of-the-art activity recognition approaches