Discovering activity patterns in office environment using a network of low-resolution visual sensors

Understanding activity patterns in office environments is important in order to increase workers’ comfort and productivity. This paper proposes an automated system for discovering activity patterns of multiple persons in a work environment using a network of cheap low-resolution visual sensors (900 pixels). Firstly, the users’ locations are obtained from a robust people tracker based on recursive maximum likelihood principles. Secondly, based on the users’ mobility tracks, the high density positions are found using a bivariate kernel density estimation. Then, the hotspots are detected using a confidence region estimation. Thirdly, we analyze the individual’s tracks to find the starting and ending hotspots. The starting and ending hotspots form an observation sequence, where the user’s presence and absence are detected using three powerful Probabilistic Graphical Models (PGMs). We describe two approaches to identify the user’s status: a single model approach and a two-model mining approach. We evaluate both approaches on video sequences captured in a real work environment, where the persons’ daily routines are recorded over 5 months. We show how the second approach achieves a better performance than the first approach. Routines dominating the entire group’s activities are identified with a methodology based on the Latent Dirichlet Allocation topic model. We also detect routines which are characteristic of persons. More specifically, we perform various analysis to determine regions with high variations, which may correspond to specific events

High accuracy context recovery using clustering mechanisms

This paper examines the recovery of user context in indoor environmnents with existing wireless infrastructures to enable assistive systems. We present a novel approach to the extraction of user context, casting the problem of context recovery as an unsupervised, clustering problem. A well known density-based clustering technique, DBSCAN, is adapted to recover user context that includes user motion state, and significant places the user visits from WiFi observations consisting of access point id and signal strength. Furthermore, user rhythms or sequences of places the user visits periodically are derived from the above low level contexts by employing state-of-the-art probabilistic clustering technique, the Latent Dirichiet Allocation (LDA), to enable a variety of application services. Experimental results with real data are presented to validate the proposed unsupervised learning approach and demonstrate its applicability.<br /

Non-intrusive load monitoring techniques for activity of daily living recognition

Esta tesis nace con la motivación de afrontar dos grandes problemas de nuestra era: la falta de recursos energéticos y el envejecimiento de la población. Respecto al primer problema, nace en la primera década de este siglo el concepto de Smart Grids con el objetivo de alcanzar la eficiencia energética. Numerosos países comienzan a realizar despliegues masivos de contadores inteligentes ("Smart Meters"), lo que despierta el interés de investigadores que comienzan a desarrollar nuevas técnicas para predecir la demanda. Así, los sistemas NILM (Non-Intrusive Load Monitoring) tratan de predecir el consumo individual de los dispositivos conectados a partir de un único sensor: el contador inteligente. Por otra parte, los grandes avances en la medicina moderna han permitido que nuestra esperanza de vida aumente considerablemente. No obstante, esta longevidad, junto con la baja fertilidad en los países desarrollados, tiene un efecto secundario: el envejecimiento de la población. Unos de los grandes avances es la incorporación de la tecnología en la vida cotidiana, lo que ayuda a los más mayores a llevar una vida independiente. El despliegue de una red de sensores dentro de la vivienda permite su monitorización y asistencia en las tareas cotidianas. Sin embargo, son intrusivos, no escalables y, en algunas ocasiones, de alto coste, por lo que no están preparados para hacer frente al incremento de la demanda de esta comunidad. Esta tesis doctoral nace de la motivación de afrontar estos problemas y tiene dos objetivos principales: lograr un modelo de monitorización sostenible para personas mayores y, a su vez, dar un valor añadido a los sistemas NILM que despierte el interés del usuario final. Con este objetivo, se presentan nuevas técnicas de monitorización basadas en NILM, aunando lo mejor de ambos campos. Esto supone un ahorro considerable de recursos en la monitorización, ya que únicamente se necesita un sensor: el contador inteligente; lo cual da escalabilidad a estos sistemas. Las contribuciones de esta tesis se dividen en dos bloques principales. En el primero se proponen nuevas técnicas NILM optimizadas para la detección de la actividad humana. Así, se desarrolla una propuesta basada en detección de eventos (conexiones de dispositivos) en tiempo real y su clasificación a un dispositivo. Con el objetivo de que pueda integrarse en contadores inteligentes. Cabe destacar que el clasificador se basa en modelos generalizados de dispositivos y no necesita conocimiento específico de la vivienda. El segundo bloque presenta tres nuevas técnicas de monitorización de personas mayores basadas en NILM. El objetivo es proporcionar una monitorización básica pero eficiente y altamente escalable, ahorrando en recursos. Los procesos Cox, log Gaussian Cox Processes (LGCP), monitorizan un único dispositivo si la rutina está estrechamente ligada a este. Así, se propone un sistema de alarmas si se detectan cambios en el comportamiento. LGCP tiene la ventaja de poder modelar periodicidades e incertidumbres propias del comportamiento humano. Cuando la rutina no depende de un único dispositivo, se proponen dos técnicas: una basada en gaussianas mixtas, Gaussian Mixture Models (GMM); y la otra basada en la Teoría de la Evidencia de Dempster-Shafer (DST). Ambas monitorizan y detectan deterioros en la actividad, causados por enfermedades como la demencia y el alzhéimer. Únicamente DST usa incertidumbres que simulan mejor el comportamiento humano y, por tanto, permite alarmas en caso de un repentino desvío. Finalmente, todas las propuestas han sido validadas mediante la evaluación de métricas y la obtención de resultados experimentales. Para ello, se han usado medidas de escenarios reales que han sido recopiladas en bases de datos. Los resultados obtenidos han sido satisfactorios, demostrando que este tipo de monitorización es posible y muy beneficioso para nuestra sociedad. Además, se ha dado a lugar nuevas propuestas que serán desarrolladas en el futuro. Códigos UNESCO: 120320 - sistemas de control medico, 332201 – distribución de la energía, 120701 – análisis de actividades, 120304 – inteligencia artificial, 120807 – plausibilidad, 221402 – patrones

An Unsupervised Framework for Online Spatiotemporal Detection of Activities of Daily Living by Hierarchical Activity Models

International audienceAutomatic detection and analysis of human activities captured by various sensors (e.g. 1 sequence of images captured by RGB camera) play an essential role in various research fields in order 2 to understand the semantic content of a captured scene. The main focus of the earlier studies has 3 been widely on supervised classification problem, where a label is assigned for a given short clip. 4 Nevertheless, in real-world scenarios, such as in Activities of Daily Living (ADL), the challenge is 5 to automatically browse long-term (days and weeks) stream of videos to identify segments with 6 semantics corresponding to the model activities and their temporal boundaries. This paper proposes 7 an unsupervised solution to address this problem by generating hierarchical models that combine 8 global trajectory information with local dynamics of the human body. Global information helps in 9 modeling the spatiotemporal evolution of long-term activities and hence, their spatial and temporal 10 localization. Moreover, the local dynamic information incorporates complex local motion patterns of 11 daily activities into the models. Our proposed method is evaluated using realistic datasets captured 12 from observation rooms in hospitals and nursing homes. The experimental data on a variety of 13 monitoring scenarios in hospital settings reveals how this framework can be exploited to provide 14 timely diagnose and medical interventions for cognitive disorders such as Alzheimer's disease. The 15 obtained results show that our framework is a promising attempt capable of generating activity 16 models without any supervision. 1

Continual machine learning for non-stationary data analysis

Although deep learning models have achieved significant successes in various fields, most of them have limited capacity in learning multiple tasks sequentially. The issue of forgetting the previously learned tasks in continual learning is known as catastrophic forgetting or interference. When the input data or the goal of learning changes, a conventional machine learning model will learn and adapt to the new status. However, the model will not remember or recognise any revisits to the previous states. This causes performance reduction and re-training curves in dealing with periodic or irregularly reoccurring changes in the data or goals. Without continual learning ability, one cannot deploy an adaptive machine learning model in a changing environment. This thesis investigates the continual learning and mitigating the catastrophic forgetting problem in neural networks. We assume non-stationary data contains multiple different tasks which are coming in sequence and will not be stored. We propose a regularisation method, which is to identify and penalise the changes of important parameters of previous tasks while learning a new one. However, when the number of tasks is sufficiently large, this method cannot preserve all the previously learned knowledge, or it impedes the integration of new knowledge. This is also known as the stability-plasticity dilemma. To solve this problem, we proposed a replay method based on Generative Adversarial Networks (GANs). Different from other replay methods, the proposed model is not bounded by the fitting capacity of the generator. However, the number of parameters increases rapidly as the number of learned tasks grows. Therefore, we propose a continual learning model based on Bayesian neural networks and a Mixture of Experts (MoE) framework. The proposed model integrates different experts which are responsible for different tasks into a giant model. Previously knowledge is preserved, and new tasks can be efficiently learned by assigning new experts. Based on Monte-Carlo Sampling, the performance is not satisfied. To address this issue, we propose a Probabilistic Neural Network (PNN) and integrate it with a conventional neural network. The PNN can produce the likelihood given input and be used in a variety of fields. To apply continual learning methods to real-world applications, we then propose a semi-supervised learning model to analyse healthcare datasets. The proposed framework extracts the general features from unlabelled data. We integrate the PNN into the framework to classify the data, which includes a smaller set of labelled samples and continually learn the new cases. The proposed model has been tested on benchmark datasets and also a real-world clinical dataset. The results showed that our proposed model outperforms the state-of-the-art models without requiring prior knowledge of the tasks and overall accuracy of the continual learning. The experiments on the real-world clinical data were designed to identify the risk of Urinary Tract Infections (UTIs) using in-home monitoring data. The UTI risk analysis model has been deployed in a digital platform and is currently part of the on-going Minder clinical study at the UK Dementia Research Institute (UK DRI). An earlier version of the model was deployed as a part of a Class-I CE marked medical device. The UK DRI Minder platform and the deployed machine learning models, including the UTI risk analysis model developed in this research, are in the process to be accredited as a Class-IIa medical device. Overall, this PhD research tackles theoretical and applied challenges of continuous learning models in dealing with real-world data. We evaluate the proposed continual learning methods in a variety of benchmarks with comprehensive analysis and show their effectiveness. Furthermore, we have applied the proposed methods in real-world applications and demonstrated the applicability of the models to real-world settings and clinical problems.Open Acces

Unsupervised training methods for non-intrusive appliance load monitoring from smart meter data

Non-intrusive appliance load monitoring (NIALM) is the process of disaggregating a household’s total electricity consumption into its contributing appliances. Smart meters are currently being deployed on national scales, providing a platform to collect aggregate household electricity consumption data. Existing approaches to NIALM require a manual training phase in which either sub-metered appliance data is collected or appliance usage is manually labelled. This training data is used to build models of the house- hold appliances, which are subsequently used to disaggregate the household’s electricity data. Due to the requirement of such a training phase, existing approaches do not scale automatically to the national scales of smart meter data currently being collected.In this thesis we propose an unsupervised training method which, unlike existing approaches, does not require a manual training phase. Instead, our approach combines general appliance knowledge with just aggregate smart meter data from the household to perform disaggregation. To do so, we address the following three problems: (i) how to generalise the behaviour of multiple appliances of the same type, (ii) how to tune general knowledge of appliances to the specific appliances within a single household using only smart meter data, and (iii) how to provide actionable energy saving advice based on the tuned appliance knowledge.First, we propose an approach to the appliance generalisation problem, which uses the Tracebase data set to build probabilistic models of household appliances. We take a Bayesian approach to modelling appliances using hidden Markov models, and empirically evaluate the extent to which they generalise to previously unseen appliances through cross validation. We show that learning using multiple appliances vastly outperforms learning from a single appliance by 61–99% when attempting to generalise to a previously unseen appliance, and furthermore that such general models can be learned from only 2–6 appliances.Second, we propose an unsupervised solution to the model tuning problem, which uses only smart meter data to learn the behaviour of the specific appliances in a given house-hold. Our approach uses general appliance models to extract appliance signatures from ?a household’s smart meter data, which are then used to refine the general appliance models. We evaluate the benefit of this process using the Reference Energy Disaggregation Data set, and show that the tuned appliance models more accurately represent the energy consumption behaviour of a given household’s appliances compared to when general appliance models are used, and furthermore that such general models can per- form comparably to when sub-metered data is used for model training. We also show that our tuning approach outperforms the current state of the art, which uses a factorial hidden Markov model to tune the general appliance models.Third, we apply both of these approaches to infer the energy efficiency of refrigerators and freezers in a data set of 117 households. We evaluate the accuracy of our approach, and show that it is able to successfully infer the energy efficiency of combined fridge freezers. We then propose an extension to our model tuning process using factorial hidden semi-Markov models to model households with a separate fridge and freezer. Finally, we show that through this extension our approach is able to simultaneously tune the appliance models of both appliances.The above contributions provide a solution which satisfies the requirements of a NIALM training method which is both unsupervised (no manual interaction required during training) and uses only smart meter data (no installation of additional hardware is required). When combined, the contributions presented in this thesis represent an advancement in the state of the art in the field of non-intrusive appliance load monitoring, and a step towards increasing the efficiency of energy consumption within households

Application of data fusion techniques and technologies for wearable health monitoring

Technological advances in sensors and communications have enabled discrete integration into everyday objects, both in the home and about the person. Information gathered by monitoring physiological, behavioural, and social aspects of our lives, can be used to achieve a positive impact on quality of life, health, and well-being. Wearable sensors are at the cusp of becoming truly pervasive, and could be woven into the clothes and accessories that we wear such that they become ubiquitous and transparent. To interpret the complex multidimensional information provided by these sensors, data fusion techniques are employed to provide a meaningful representation of the sensor outputs. This paper is intended to provide a short overview of data fusion techniques and algorithms that can be used to interpret wearable sensor data in the context of health monitoring applications. The application of these techniques are then described in the context of healthcare including activity and ambulatory monitoring, gait analysis, fall detection, and biometric monitoring. A snap-shot of current commercially available sensors is also provided, focusing on their sensing capability, and a commentary on the gaps that need to be bridged to bring research to market

Using topic models to detect behaviour patterns for healthcare monitoring

Healthcare systems worldwide are facing growing demands on their resources due to an ageing population and increase in prevalence of chronic diseases. Innovative residential healthcare monitoring systems, using a variety of sensors are being developed to help address these needs. Interpreting the vast wealth of data generated is key to fully exploiting the benefits offered by a monitoring system. This thesis presents the application of topic models, a machine learning algorithm, to detect behaviour patterns in different types of data produced by a monitoring system. Latent Dirichlet Allocation was applied to real world activity data with corresponding ground truth labels of daily routines. The results from an existing dataset and a novel dataset collected using a custom mobile phone app, demonstrated that the patterns found are equivalent of routines. Long term monitoring can identify changes that could indicate an alteration in health status. Dynamic topic models were applied to simulated long term activity datasets to detect changes in the structure of daily routines. It was shown that the changes occurring in the simulated data can successfully be detected. This result suggests potential for dynamic topic models to identify changes in routines that could aid early diagnosis of chronic diseases. Furthermore, chronic conditions, such as diabetes and obesity, are related to quality of diet. Current research findings on the association between eating behaviours, especially snacking, and the impact on diet quality and health are often conflicting. One problem is the lack of consistent definitions for different types of eating event. The novel application of Latent Dirichlet Allocation to three nutrition datasets is described. The results demonstrated that combinations of food groups representative of eating event types can be detected. Moreover, labels assigned to these combinations showed good agreement with alternative methods for labelling eating event types