Deep learning for multi-resident activity recognition in ambient sensing smart homes

Advances in smart home technology and IoT devices has enabled us for monitoring of human activities for their health status and efficient energy consumption. Machine learning has been a great tool for the prediction of human activities. However, Multi-resident activity recognition is still a challenge as there is no direct correlation between sensor values and resident activities. In this paper, we have displayed the state of art deep learning algorithms on the real-world ARAS multi-resident dataset, which consists of data from two houses each with two residents. We have used different variations of RNN on the dataset and measured their performance with fewer data and more data and with data generated with GAN

A Survey on Multi-Resident Activity Recognition in Smart Environments

Human activity recognition (HAR) is a rapidly growing field that utilizes smart devices, sensors, and algorithms to automatically classify and identify the actions of individuals within a given environment. These systems have a wide range of applications, including assisting with caring tasks, increasing security, and improving energy efficiency. However, there are several challenges that must be addressed in order to effectively utilize HAR systems in multi-resident environments. One of the key challenges is accurately associating sensor observations with the identities of the individuals involved, which can be particularly difficult when residents are engaging in complex and collaborative activities. This paper provides a brief overview of the design and implementation of HAR systems, including a summary of the various data collection devices and approaches used for human activity identification. It also reviews previous research on the use of these systems in multi-resident environments and offers conclusions on the current state of the art in the field.Comment: 16 pages, to appear in Evolution of Information, Communication and Computing Systems (EICCS) Book Serie

Overview of Human Activity Recognition Using Sensor Data

Human activity recognition (HAR) is an essential research field that has been used in different applications including home and workplace automation, security and surveillance as well as healthcare. Starting from conventional machine learning methods to the recently developing deep learning techniques and the Internet of things, significant contributions have been shown in the HAR area in the last decade. Even though several review and survey studies have been published, there is a lack of sensor-based HAR overview studies focusing on summarising the usage of wearable sensors and smart home sensors data as well as applications of HAR and deep learning techniques. Hence, we overview sensor-based HAR, discuss several important applications that rely on HAR, and highlight the most common machine learning methods that have been used for HAR. Finally, several challenges of HAR are explored that should be addressed to further improve the robustness of HAR

Employing multi-modal sensors for personalised smart home health monitoring.

Smart home systems are employed worldwide for a variety of automated monitoring tasks. FITsense is a system that performs personalised smart home health monitoring using sensor data. In this thesis, we expand upon this system by identifying the limits of health monitoring using simple IoT sensors, and establishing deployable solutions for new rich sensing technologies. The FITsense system collects data from FitHomes and generates behavioural insights for health monitoring. To allow the system to expand to arbitrary home layouts, sensing applications must be delivered while relying on sparse "ground truth" data. An enhanced data representation was tested for improving activity recognition performance by encoding observed temporal dependencies. Experiments showed an improvement in activity recognition accuracy over baseline data representations with standard classifiers. Channel State Information (CSI) was chosen as our rich sensing technology for its ambient nature and potential deployability. We developed a novel Python toolkit, called CSIKit, to handle various CSI software implementations, including automatic detection for off-the-shelf CSI formats. Previous researchers proposed a method to address AGC effects on COTS CSI hardware, which we tested and found to improve correlation with a baseline without AGC. This implementation was included in the public release of CSIKit. Two sensing applications were delivered using CSIKit to demonstrate its functionality. Our statistical approach to motion detection with CSI data showed a 32% increase in accuracy over an infrared sensor-based solution using data from 2 unique environments. We also demonstrated the first CSI activity recognition application on a Raspberry Pi 4, which achieved an accuracy of 92% with 11 activity classes. An application was then trained to support movement detection using data from all COTS CSI hardware. This was combined with our signal divider implementation to compare CSI wireless and sensing performance characteristics. The IWL5300 exhibited the most consistent wireless performance, while the ESP32 was found to produce viable CSI data for sensing applications. This establishes the ESP32 as a low-cost high-value hardware solution for CSI sensing. To complete this work, an in-home study was performed using real-world sensor data. An ESP32-based CSI sensor was developed to be integrated into our IoT network. This sensor was tested in a FitHome environment to identify how the data from our existing simple sensors could aid sensor development. We performed an experiment to demonstrate that annotations for CSI data could be gathered with infrared motion sensors. Results showed that our new CSI sensor collected real-world data of similar utility to that collected manually in a controlled environment

A survey of user-centred approaches for smart home transfer learning and new user home automation adaptation

Recent smart home applications enhance the quality of people's home experiences by detecting their daily activities and providing them services that make their daily life more comfortable and safe. Human activity recognition is one of the fundamental tasks that a smart home should accomplish. However, there are still several challenges for such recognition in smart homes, with the target home adaptation process being one of the most critical, since new home environments do not have sufficient data to initiate the necessary activity recognition process. The transfer learning approach is considered the solution to this challenge, due to its ability to improve the adaptation process. This paper endeavours to provide a concrete review of user-centred smart homes along with the recent advancements in transfer learning for activity recognition. Furthermore, the paper proposes an integrated, personalised system that is able to create a dataset for target homes using both survey and transfer learning approaches, providing a personalised dataset based on user preferences and feedback

A survey of user-centred approaches for smart home transfer learning and new user home automation adaptation

Recent smart home applications enhance the quality of people's home experiences by detecting their daily activities and providing them services that make their daily life more comfortable and safe. Human activity recognition is one of the fundamental tasks that a smart home should accomplish. However, there are still several challenges for such recognition in smart homes, with the target home adaptation process being one of the most critical, since new home environments do not have sufficient data to initiate the necessary activity recognition process. The transfer learning approach is considered the solution to this challenge, due to its ability to improve the adaptation process. This paper endeavours to provide a concrete review of user-centred smart homes along with the recent advancements in transfer learning for activity recognition. Furthermore, the paper proposes an integrated, personalised system that is able to create a dataset for target homes using both survey and transfer learning approaches, providing a personalised dataset based on user preferences and feedback

Privacy-preserving human mobility and activity modelling

The exponential proliferation of digital trends and worldwide responses to the COVID-19 pandemic thrust the world into digitalization and interconnectedness, pushing increasingly new technologies/devices/applications into the market. More and more intimate data of users are collected for positive analysis purposes of improving living well-being but shared with/without the user's consent, emphasizing the importance of making human mobility and activity models inclusive, private, and fair. In this thesis, I develop and implement advanced methods/algorithms to model human mobility and activity in terms of temporal-context dynamics, multi-occupancy impacts, privacy protection, and fair analysis. The following research questions have been thoroughly investigated: i) whether the temporal information integrated into the deep learning networks can improve the prediction accuracy in both predicting the next activity and its timing; ii) how is the trade-off between cost and performance when optimizing the sensor network for multiple-occupancy smart homes; iii) whether the malicious purposes such as user re-identification in human mobility modelling could be mitigated by adversarial learning; iv) whether the fairness implications of mobility models and whether privacy-preserving techniques perform equally for different groups of users. To answer these research questions, I develop different architectures to model human activity and mobility. I first clarify the temporal-context dynamics in human activity modelling and achieve better prediction accuracy by appropriately using the temporal information. I then design a framework MoSen to simulate the interaction dynamics among residents and intelligent environments and generate an effective sensor network strategy. To relieve users' privacy concerns, I design Mo-PAE and show that the privacy of mobility traces attains decent protection at the marginal utility cost. Last but not least, I investigate the relations between fairness and privacy and conclude that while the privacy-aware model guarantees group fairness, it violates the individual fairness criteria.Open Acces