A Survey on Multi-Resident Activity Recognition in Smart Environments

Human activity recognition (HAR) is a rapidly growing field that utilizes smart devices, sensors, and algorithms to automatically classify and identify the actions of individuals within a given environment. These systems have a wide range of applications, including assisting with caring tasks, increasing security, and improving energy efficiency. However, there are several challenges that must be addressed in order to effectively utilize HAR systems in multi-resident environments. One of the key challenges is accurately associating sensor observations with the identities of the individuals involved, which can be particularly difficult when residents are engaging in complex and collaborative activities. This paper provides a brief overview of the design and implementation of HAR systems, including a summary of the various data collection devices and approaches used for human activity identification. It also reviews previous research on the use of these systems in multi-resident environments and offers conclusions on the current state of the art in the field.Comment: 16 pages, to appear in Evolution of Information, Communication and Computing Systems (EICCS) Book Serie

Non-intrusive load monitoring techniques for activity of daily living recognition

Esta tesis nace con la motivación de afrontar dos grandes problemas de nuestra era: la falta de recursos energéticos y el envejecimiento de la población. Respecto al primer problema, nace en la primera década de este siglo el concepto de Smart Grids con el objetivo de alcanzar la eficiencia energética. Numerosos países comienzan a realizar despliegues masivos de contadores inteligentes ("Smart Meters"), lo que despierta el interés de investigadores que comienzan a desarrollar nuevas técnicas para predecir la demanda. Así, los sistemas NILM (Non-Intrusive Load Monitoring) tratan de predecir el consumo individual de los dispositivos conectados a partir de un único sensor: el contador inteligente. Por otra parte, los grandes avances en la medicina moderna han permitido que nuestra esperanza de vida aumente considerablemente. No obstante, esta longevidad, junto con la baja fertilidad en los países desarrollados, tiene un efecto secundario: el envejecimiento de la población. Unos de los grandes avances es la incorporación de la tecnología en la vida cotidiana, lo que ayuda a los más mayores a llevar una vida independiente. El despliegue de una red de sensores dentro de la vivienda permite su monitorización y asistencia en las tareas cotidianas. Sin embargo, son intrusivos, no escalables y, en algunas ocasiones, de alto coste, por lo que no están preparados para hacer frente al incremento de la demanda de esta comunidad. Esta tesis doctoral nace de la motivación de afrontar estos problemas y tiene dos objetivos principales: lograr un modelo de monitorización sostenible para personas mayores y, a su vez, dar un valor añadido a los sistemas NILM que despierte el interés del usuario final. Con este objetivo, se presentan nuevas técnicas de monitorización basadas en NILM, aunando lo mejor de ambos campos. Esto supone un ahorro considerable de recursos en la monitorización, ya que únicamente se necesita un sensor: el contador inteligente; lo cual da escalabilidad a estos sistemas. Las contribuciones de esta tesis se dividen en dos bloques principales. En el primero se proponen nuevas técnicas NILM optimizadas para la detección de la actividad humana. Así, se desarrolla una propuesta basada en detección de eventos (conexiones de dispositivos) en tiempo real y su clasificación a un dispositivo. Con el objetivo de que pueda integrarse en contadores inteligentes. Cabe destacar que el clasificador se basa en modelos generalizados de dispositivos y no necesita conocimiento específico de la vivienda. El segundo bloque presenta tres nuevas técnicas de monitorización de personas mayores basadas en NILM. El objetivo es proporcionar una monitorización básica pero eficiente y altamente escalable, ahorrando en recursos. Los procesos Cox, log Gaussian Cox Processes (LGCP), monitorizan un único dispositivo si la rutina está estrechamente ligada a este. Así, se propone un sistema de alarmas si se detectan cambios en el comportamiento. LGCP tiene la ventaja de poder modelar periodicidades e incertidumbres propias del comportamiento humano. Cuando la rutina no depende de un único dispositivo, se proponen dos técnicas: una basada en gaussianas mixtas, Gaussian Mixture Models (GMM); y la otra basada en la Teoría de la Evidencia de Dempster-Shafer (DST). Ambas monitorizan y detectan deterioros en la actividad, causados por enfermedades como la demencia y el alzhéimer. Únicamente DST usa incertidumbres que simulan mejor el comportamiento humano y, por tanto, permite alarmas en caso de un repentino desvío. Finalmente, todas las propuestas han sido validadas mediante la evaluación de métricas y la obtención de resultados experimentales. Para ello, se han usado medidas de escenarios reales que han sido recopiladas en bases de datos. Los resultados obtenidos han sido satisfactorios, demostrando que este tipo de monitorización es posible y muy beneficioso para nuestra sociedad. Además, se ha dado a lugar nuevas propuestas que serán desarrolladas en el futuro. Códigos UNESCO: 120320 - sistemas de control medico, 332201 – distribución de la energía, 120701 – análisis de actividades, 120304 – inteligencia artificial, 120807 – plausibilidad, 221402 – patrones

Identification and prediction of abnormal behaviour activities of daily living in intelligent environments

The aim of this research is to investigate efficient mining of useful information from a sensor network forming an Ambient Intelligence (AmI) environment. In this thesis, we investigate methods for supporting independent living of the elderly (and specifically patients who are suffering from dementia) by means of equipping their home with a simple sensor network to monitor their behaviour and identify their Activities of Daily Living (ADL). Dementia is considered to be one of the most important causes of disability in the elderly. Mostpatients would prefer to use non-intrusive technology to help them tomaintain their independence. Such monitoring and prediction would allow the caregiver to see any trend in the behaviour of the elderly person and to be informed of any abnormal behaviour

Residential Sector Energy Consumption at the Spotlight: From Data to Knowledge

Energy consumption is at the core of economic development, but its severe impacts on resources depletion and climate change have justified a call for its general reduction across all economic activities. Lowering households’ energy demand is a key factor to achieve carbon dioxide emission reductions as it has an important energy-saving potential. Households in the European Union (EU28) countries have a significant weight (25%) in the total final energy consumption. However, a wide range of variation is observed within the residential sector from 7.6 to 37.4 GJ per capita/annum, with the lowest consumption indicator observed in Southern EU countries. Energy consumption in the residential sector is a complex issue, explained by a combination of different factors. To pinpoint how to reduce energy consumption effectively while deliver energy services, we need to look not just at technology, but also to the factors that drive how and in what extent people consume energy, including the way they interact with technology (i.e., energy efficiency). The main objective of this research is to understand the differences in energy consumption arising from different socio-demographic, technologic, behavioral and economic characteristics of residential households. This research brings to the spotlight the needs and benefits of looking deeper into residential sector energy consumption in a southern European country. Portugal and the municipality of Évora, in particular, were selected as case studies. Residential sector consumption is a moving target, which increase the complexity of adequate policies and instruments that have to address the bottleneck between increase demand for e.g. climatization due to current lack of thermal comfort and to comply with objectives of increased energy efficiency which ultimately intend to reduce energy consumption. This calls for different levels of knowledge to feed multiscale policies. This dissertation expands the understanding of energy consumption patterns at households, consumers’ role in energy consumption profiles, indoor thermal comfort, and the levels of satisfaction from energy services demand. In a country potentially highly impacted by climate change, with low levels of income and significant lower energy consumption per capita compared to the EU28 average, looking into these issues gains even more importance. The work combines detailed analysis at different spatial (national, city and consumers level) and time scales (hour to annual) taking advantage of diverse methods and datasets including smart meters’ data, door to door surveys and energy simulation and optimization modelling. The results identify (i) ten distinct residential sector consumer groups (e.g., under fuel poverty); (ii) daily and annual consumption patterns (W, U and flat); (iii) major energy consumption determinants such as the physical characteristics of dwellings, particularly the year of construction and floor area; climatization equipment ownership and use, and occupants’ profiles (mainly number and monthly income). It is (iv) recognized that inhabitants try to actively control space heating, but without achieving indoor thermal comfort levels. The results also show (v) that technology can overweight the impact of practices and lifestyle changes for some end-uses as space heating and lighting. Nevertheless, important focus should be given to the evolution in the future of uncertain parameters related with consumer behavior, especially those on climatization, related to thermal comfort and equipment’s use. Furthermore, the research work presents a (vi) bottom-up methodology to project detailed energy end-uses demand, and (vii) an integrated framework for city energy planning. This work sets the ground for the definition of tailor-made policy recommendations for targeted consumer groups (e.g., vulnerable consumers) and climatization behavior/practices to reduce peak demand, social support policies, energy efficiency instruments and measures, renewable energy sources integration, and energy systems planning

Adaptive learning approaches for smart home environments with a simulator implementation

Smart home solutions are utilized in a different way by each user according to the user’s unique needs and preferences over his unique home setting. User – device interactions themselves carry some implicit characteristics of the context the smart devices are used. In order to better exploit Internet of Things technologies in smart home environments, the user’s interaction history can be leveraged to generate knowledge of his behavior habits. With a purpose of achieving personalized decision making of smart lighting environments, this thesis presents three learning model approaches, which are based solely on the individual’s interaction habits with the devices, adaptive to changes in inhabitant’s device usage behavior, and do not require preset data for initialization. Individual interactions with smart light devices are contextualized based on the timestamp of the interaction, and ambient light intensity reading of the room at that instant. Moreover, the thesis introduces a Java simulator, which interactively demonstrates the behavior of a personalized smart home setting with the integrated learning models with a feedback mechanism. The simulator leads up to a way of evaluating adaptive learning models in smart home environments, reducing the immediate need of testing their behavior in a real life smart home, which is both hard and costly to construct and maintain. The learning approaches, namely K-Nearest Neighbor and Softmax Regression with batch learning and online learning algorithms, are evaluated with different datasets representing different scenarios, and the results show that all three methods are able to perfectly capture the usage pattern when the interactions with distinct “things”, smart light devices, are separable in terms of their corresponding context. For more complex datasets, which have overlaps between the usage context of distinct devices and big changes in user behavior over time, the online learning algorithm needs more data in order to catch the performance of KNN and Softmax Regression with batch learning algorithms.Electrical and Computer Engineerin

Recognition of cooking activities through air quality sensor data for supporting food journaling

Abstract Unhealthy behaviors regarding nutrition are a global risk for health. Therefore, the healthiness of an individual's nutrition should be monitored in the medium and long term. A powerful tool for monitoring nutrition is a food diary; i.e., a daily list of food taken by the individual, together with portion information. Unfortunately, frail people such as the elderly have a hard time filling food diaries on a continuous basis due to forgetfulness or physical issues. Existing solutions based on mobile apps also require user's effort and are rarely used in the long term, especially by elderly people. For these reasons, in this paper we propose a novel architecture to automatically recognize the preparation of food at home in a privacy-preserving and unobtrusive way, by means of air quality data acquired from a commercial sensor. In particular, we devised statistical features to represent the trend of several air parameters, and a deep neural network for recognizing cooking activities based on those data. We collected a large corpus of annotated sensor data gathered over a period of 8 months from different individuals in different homes, and performed extensive experiments. Moreover, we developed an initial prototype of an interactive system for acquiring food information from the user when a cooking activity is detected by the neural network. To the best of our knowledge, this is the first work that adopts air quality sensor data for cooking activity recognition

Sensor-based early activity recognition inside buildings to support energy and comfort management systems

Building Energy and Comfort Management (BECM) systems have the potential to considerably reduce costs related to energy consumption and improve the efficiency of resource exploitation, by implementing strategies for resource management and control and policies for Demand-Side Management (DSM). One of the main requirements for such systems is to be able to adapt their management decisions to the users’ specific habits and preferences, even when they change over time. This feature is fundamental to prevent users’ disaffection and the gradual abandonment of the system. In this paper, a sensor-based system for analysis of user habits and early detection and prediction of user activities is presented. To improve the resulting accuracy, the system incorporates statistics related to other relevant external conditions that have been observed to be correlated (e.g., time of the day). Performance evaluation on a real use case proves that the proposed system enables early recognition of activities after only 10 sensor events with an accuracy of 81%. Furthermore, the correlation between activities can be used to predict the next activity with an accuracy of about 60%

Machine learning techniques for sensor-based household activity recognition and forecasting

Thanks to the recent development of cheap and unobtrusive smart-home sensors, ambient assisted living tools promise to offer innovative solutions to support the users in carrying out their everyday activities in a smoother and more sustainable way. To be effective, these solutions need to constantly monitor and forecast the activities of daily living carried out by the inhabitants. The Machine Learning field has seen significant advancements in the development of new techniques, especially regarding deep learning algorithms. Such techniques can be successfully applied to household activity signal data to benefit the user in several applications. This thesis therefore aims to produce a contribution that artificial intelligence can make in the field of activity recognition and energy consumption. The effective recognition of common actions or the use of high-consumption appliances would lead to user profiling, thus enabling the optimisation of energy consumption in favour of the user himself or the energy community in general. Avoiding wasting electricity and optimising its consumption is one of the main objectives of the community. This work is therefore intended as a forerunner for future studies that will allow, through the results in this thesis, the creation of increasingly intelligent systems capable of making the best use of the user's resources for everyday life actions. Namely, this thesis focuses on signals from sensors installed in a house: data from position sensors, door sensors, smartphones or smart meters, and investigates the use of advanced machine learning algorithms to recognize and forecast inhabitant activities, including the use of appliances and the power consumption. The thesis is structured into four main chapters, each of which represents a contribution regarding Machine Learning or Deep Learning techniques for addressing challenges related to the aforementioned data from different sources. The first contribution highlights the importance of exploiting dimensionality reduction techniques that can simplify a Machine Learning model and increase its efficiency by identifying and retaining only the most informative and predictive features for activity recognition. In more detail, it is presented an extensive experimental study involving several feature selection algorithms and multiple Human Activity Recognition benchmarks containing mobile sensor data. In the second contribution, we propose a machine learning approach to forecast future energy consumption considering not only past consumption data, but also context data such as inhabitants’ actions and activities, use of household appliances, interaction with furniture and doors, and environmental data. We performed an experimental evaluation with real-world data acquired in an instrumented environment from a large user group. Finally, the last two contributions address the Non-Intrusive-Load-Monitoring problem. In one case, the aim is to identify the operating state (on/off) and the precise energy consumption of individual electrical loads, considering only the aggregate consumption of these loads as input. We use a Deep Learning method to disaggregate the low-frequency energy signal generated directly by the new generation smart meters being deployed in Italy, without the need for additional specific hardware. In the other case, driven by the need to build intelligent non-intrusive algorithms for disaggregating electrical signals, the work aims to recognize which appliance is activated by analyzing energy measurements and classifying appliances through Machine Learning techniques. Namely, we present a new way of approaching the problem by unifying Single Label (single active appliance recognition) and Multi Label (multiple active appliance recognition) learning paradigms. This combined approach, supplemented with an event detector, which suggests the instants of activation, would allow the development of an end-to-end NILM approach

New hybrid housing scenarios

Tese de doutoramento em Arquitectura, Universidade Lusíada de Lisboa, 2020Exame público realizado em 31 de Maio de 2021Living and working in a contemporary metropolis nowadays means adapting to the socioeconomic, working and community dynamics dictated by the forma urbis itself. Dwelling represents its founding element, not only for its archetypal role as a "domestic hearth" and for its ethno-identity component, but also because it is now also part of the production process itself, thanks to the spread of remote working. Furthermore, the ways of living, in the broadest sense of the term itself, vary according to personal housing practices as well as consolidated cultural legacies. Within this framework, starting from the specific case of Dubai, chosen as a field of investigation for its contextual specificity, the social and spatial paradigms of living and working in such multicultural context were identified. Its forma urbis, today, is no longer representative of the environmental and cultural context of the resident communities, as it was until a few decades ago, but instead the result of globalization and city branding. The salient and specific characteristics of this recent past have therefore been identified, which have led to the formulation of the theoretical framework of the present Research. The new settlement paradigm identified is based on an incremental and time-varying urban density housing system, based on the synergy of two constituent elements defined as: "Combinatory Urban Matrix" and "Unit(y) of Living". Their application reveals new hybrid living and working scenarios in Dubai that will allow to overcome the current applied models. From the synergy of these two factors together with the Internet-network, social media and digital technologies, such as Augmented Reality and Virtual Reality, urban spaces will acquire osmotic properties. The diaphragmatic features of the "Combinatory Urban Matrix" will constitute the physical-virtual support for the re-establishment of spaces for social and community interaction, as well as the physical support for the "Unit(y) of Living". The latter, thanks to its planimetric and aggregative flexibility, will also be responsible of interpreting the housing identity of the various resident communities. As well as to provide the necessary ductility to the urban settlement system itself, in order to proceeding beyond cultural barriers, in the name of integration and social equit

Realistic Multi-Scale Modelling of Household Electricity Behaviours

To improve the management and reliability of power distribution networks, there is a strong demand for models simulating energy loads in a realistic way. In this paper, we present a novel multi-scale model to generate realistic residential load profiles at different spatial-temporal resolutions. By taking advantage of information from Census and national surveys, we generate statistically consistent populations of heterogeneous families with their respective appliances. Exploiting a Bottom-up approach based on Monte Carlo Non Homogeneous Semi-Markov, we provide household end-user behaviours and realistic households load profiles on a daily as well as on a weekly basis, for either weekdays and weekends. The proposed approach overcomes limitations of state-of-art solutions that do not consider neither the time-dependency of the probability of performing specific activities in a house, nor their duration, or are limited in the type of probability distributions they can model. On top of that, it provides outcomes that are not limited on a per-day basis. The range of available space and time resolutions span from single household to district and from second to year, respectively, featuring multi-level aggregation of the simulation outcomes. To demonstrate the accuracy of our model, we present experimental results obtained simulating realistic populations in a period covering a whole calendar year and analyse our model’s outcome at different scales. Then, we compare such results with three different data-sets that provide real load consumption at household, national and European levels, respectively