A Survey on Multi-Resident Activity Recognition in Smart Environments

Human activity recognition (HAR) is a rapidly growing field that utilizes smart devices, sensors, and algorithms to automatically classify and identify the actions of individuals within a given environment. These systems have a wide range of applications, including assisting with caring tasks, increasing security, and improving energy efficiency. However, there are several challenges that must be addressed in order to effectively utilize HAR systems in multi-resident environments. One of the key challenges is accurately associating sensor observations with the identities of the individuals involved, which can be particularly difficult when residents are engaging in complex and collaborative activities. This paper provides a brief overview of the design and implementation of HAR systems, including a summary of the various data collection devices and approaches used for human activity identification. It also reviews previous research on the use of these systems in multi-resident environments and offers conclusions on the current state of the art in the field.Comment: 16 pages, to appear in Evolution of Information, Communication and Computing Systems (EICCS) Book Serie

Multioccupant Activity Recognition in Pervasive Smart Home Environments

been the center of lot of research for many years now. The aim is to recognize the sequence of actions by a specific person using sensor readings. Most of the research has been devoted to activity recognition of single occupants in the environment. However, living environments are usually inhabited by more than one person and possibly with pets. Hence, human activity recognition in the context of multi-occupancy is more general, but also more challenging. The difficulty comes from mainly two aspects: resident identification, known as data association, and diversity of human activities. The present survey paper provides an overview of existing approaches and current practices for activity recognition in multi-occupant smart homes. It presents the latest developments and highlights the open issues in this field

Federated learning and genetic mutation for multi-resident activity recognition

Multi-Resident activity recognition refers to the task of recognizing activities performed by multiple individuals living in the same residence. It involves using sensors or other monitoring devices to capture data about the activities taking place in the living space, and then using Machine Learning (ML) or Deep Learning (DL) algorithms to analyze and classify these activities. Federated Learning (FL) is a technique that enables multiple devices to collaboratively train a model without sharing their data with each other, while Genetic Mutation (GM) is a technique used in evolutionary algorithms to introduce random changes to the genetic code of individuals in a population. Our proposed framework involves the use FL and GM for Human Activity Recognition (HAR). The approach was evaluated on the ARAS dataset, collected from two houses with different activity patterns. Two Recurrent Neural Network (RNN) models, Gated Recurrent Unit (GRU) and Long-Short Term Memory (LSTM), were employed for the activity classification task and a genetic mutation operator was applied to the weights of the models before federated averaging. The results indicate that FL is suitable for privacy preserving activity recognition, it can help with early deployment and even improve the performance of the models in some cases

Privacy-preserving human mobility and activity modelling

The exponential proliferation of digital trends and worldwide responses to the COVID-19 pandemic thrust the world into digitalization and interconnectedness, pushing increasingly new technologies/devices/applications into the market. More and more intimate data of users are collected for positive analysis purposes of improving living well-being but shared with/without the user's consent, emphasizing the importance of making human mobility and activity models inclusive, private, and fair. In this thesis, I develop and implement advanced methods/algorithms to model human mobility and activity in terms of temporal-context dynamics, multi-occupancy impacts, privacy protection, and fair analysis. The following research questions have been thoroughly investigated: i) whether the temporal information integrated into the deep learning networks can improve the prediction accuracy in both predicting the next activity and its timing; ii) how is the trade-off between cost and performance when optimizing the sensor network for multiple-occupancy smart homes; iii) whether the malicious purposes such as user re-identification in human mobility modelling could be mitigated by adversarial learning; iv) whether the fairness implications of mobility models and whether privacy-preserving techniques perform equally for different groups of users. To answer these research questions, I develop different architectures to model human activity and mobility. I first clarify the temporal-context dynamics in human activity modelling and achieve better prediction accuracy by appropriately using the temporal information. I then design a framework MoSen to simulate the interaction dynamics among residents and intelligent environments and generate an effective sensor network strategy. To relieve users' privacy concerns, I design Mo-PAE and show that the privacy of mobility traces attains decent protection at the marginal utility cost. Last but not least, I investigate the relations between fairness and privacy and conclude that while the privacy-aware model guarantees group fairness, it violates the individual fairness criteria.Open Acces

Adaptive learning approaches for smart home environments with a simulator implementation

Smart home solutions are utilized in a different way by each user according to the user’s unique needs and preferences over his unique home setting. User – device interactions themselves carry some implicit characteristics of the context the smart devices are used. In order to better exploit Internet of Things technologies in smart home environments, the user’s interaction history can be leveraged to generate knowledge of his behavior habits. With a purpose of achieving personalized decision making of smart lighting environments, this thesis presents three learning model approaches, which are based solely on the individual’s interaction habits with the devices, adaptive to changes in inhabitant’s device usage behavior, and do not require preset data for initialization. Individual interactions with smart light devices are contextualized based on the timestamp of the interaction, and ambient light intensity reading of the room at that instant. Moreover, the thesis introduces a Java simulator, which interactively demonstrates the behavior of a personalized smart home setting with the integrated learning models with a feedback mechanism. The simulator leads up to a way of evaluating adaptive learning models in smart home environments, reducing the immediate need of testing their behavior in a real life smart home, which is both hard and costly to construct and maintain. The learning approaches, namely K-Nearest Neighbor and Softmax Regression with batch learning and online learning algorithms, are evaluated with different datasets representing different scenarios, and the results show that all three methods are able to perfectly capture the usage pattern when the interactions with distinct “things”, smart light devices, are separable in terms of their corresponding context. For more complex datasets, which have overlaps between the usage context of distinct devices and big changes in user behavior over time, the online learning algorithm needs more data in order to catch the performance of KNN and Softmax Regression with batch learning algorithms.Electrical and Computer Engineerin

Modeling Interaction in Multi-Resident Activities

In this paper we investigate the problem of modeling multi-resident activities. Specifically, we explore different approaches based on Hidden Markov Models (HMMs) to deal with parallel activities and cooperative activities. We propose an HMM-based method, called CL-HMM, where activity labels as well as observation labels of different residents are combined to generate the corresponding sequence of activities as well as the corresponding sequence of observations on which a conventional HMM is applied. We also propose a Linked HMM (LHMM) in which activities of all residents are linked at each time step. We compare these two models to baseline models which are Coupled HMM (CHMM) and Parallel HMM (PHMM). The experimental results show that the proposed models outperform CHMM and PHMM when tested on parallel and cooperative activities