Universal Non-Intrusive Load Monitoring (UNILM) Using Filter Pipelines, Probabilistic Knapsack, and Labelled Partition Maps

Being able to track appliances energy usage without the need of sensors can help occupants reduce their energy consumption to help save the environment all while saving money. Non-intrusive load monitoring (NILM) tries to do just that. One of the hardest problems NILM faces is the ability to run unsupervised -- discovering appliances without prior knowledge -- and to run independent of the differences in appliance mixes and operational characteristics found in various countries and regions. We propose a solution that can do this with the use of an advanced filter pipeline to preprocess the data, a Gaussian appliance model with a probabilistic knapsack algorithm to disaggregate the aggregate smart meter signal, and partition maps to label which appliances were found and how much energy they use no matter the country/region. Experimental results show that relatively complex appliance signals can be tracked accounting for 93.7% of the total aggregate energy consumed

Desagregação de consumos energéticos usando Machine Learning

Nowadays, we are surrounded by electric appliances. Either at home by the washing machine, kettle, or oven, or work by the computer, cellphone, or printer. Such devices help us daily, but their popularization increased the energy consumption to concerning values. In an attempt to reduce energy consumption, governments started enforcing policies regarding energy education to teach homeowners how to reduce energy wastage on the demand side. One of those policies was the deployment of smart meters, which allow the consumer to know how much energy is being consumed at any given time through a display on the household energy meter. Even though this measure was well received, the studies show that the best results in energy conservation are obtained through real-time appliance level feedback. To get such feedback, one can either measure every outlet in a household, which is unviable for a broad deployment solution, or disaggregate the energy recorded by the smart meter. NILM or Non-Intrusive Load Monitoring is the name we give to the second option where we use the aggregated readings of a household to find the energy consumed by each appliance. There were many proposals to solve NILM ranging from HMMs to GSP, where deep learning models showed remarkable results, obtaining state-of-the-art results. With the intent to create a complete NILM solution, Withus partnered with the University of Aveiro and proposed this dissertation. The initial objective was to develop a machine learning model to solve NILM. Still, during the background analysis, we found the need to create a new dataset which led to the expansion of the initial proposal to include the dataset preprocessing and conversion. Regarding NILM, we proposed three new deep learning models: a convolutional neural network with residual blocks, a recurrent neural network, and a multilayer perceptron that uses discrete wavelet transforms as features. These models went through multiple iterations, being evaluated first in the simpler ON/OFF classification task and later modified and evaluated for the disaggregation task. We compared our models to the state-of-the-art ones proposed in NILMTK, where they presented better results than the real-time alternative, dAE, reducing the NRMSE on average by 49%. We also got close to the best option that classified with a 30 min delay, Seq2Point, increasing the error on average by 17%. Besides that, we also analyze the best models from the previous comparison on the benefit of transfer learning between datasets, where the results show a marginal performance improvement when using transfer learning. This document presents the solution outline definition, the multiple options considered for dataset processing and the best solution, the models’ evolution and results, and the comparison with the state-of-the-art models regarding generalization to different houses and under transfer learning.Hoje em dia estamos rodeados de dispositivos elétricos. Quer seja em casa, pela máquina de lavar, o microondas ou o forno ou no emprego pelo computador, o telemóvel ou a impressora. Estes dispositivos ajudam-nos diariamente, mas com a sua popularização o consumo energético atingiu valores preocupantes. Numa tentativa de reduzir o consumo energético, os governos começaram a introduzir políticas de educação energética para ensinar os consumidores a reduzir o desperdício energético. Uma das medidas foi a implementação generalizada de smart meters, que permitem ao consumidor saber quanta energia está a ser consumida a qualquer altura através de um ecrã no contador da casa. Mesmo sendo bem recebida, esta medida não é suficiente uma vez que os estudos indicam que os melhores resultados são obtidos através de feedback ao nível do dispositivo em tempo real. Para obtermos este feedback existem duas formas, podemos medir cada tomada numa dada casa, o que é inviável para uma implementação em larga escala, ou desagregar a energia registrada pelo smart meter que já está presente na casa. NILM ou Non-Intrusive Load Monitoring é o nome dado à segunda opção onde a energia agregada da casa é usada para descobrirmos a energia consumida por cada dispositivo elétrico. Para resolver este problema foram propostas várias alternativas, desde HMMs a GSP, onde os modelos de deep learning obtiveram resultados notáveis sendo agora o estado da arte. Com o objetivo de produzir um sistema NILM completo, a Withus juntou-se à Universidade de Aveiro e juntos propuseram esta dissertação. O objetivo inicial era o desenvolvimento de um modelo de machine learning para desagregar consumos elétricos. Contudo, durante análise do estado da arte, deparamo-nos com a necessidade de criar um novo dataset, o que levou à extensão da proposta inicial para incluir também o pré-processamento e conversão do dataset. Para desagregação de consumos elétricos propusemos três modelos: uma rede neuronal convolucional com blocos residuais, uma rede neuronal recorrente e um multilayer perceptron que usa discrete wavelet transforms como features. Estes modelos passaram por diversas iterações, sendo avaliados primeiro na tarefa de classificação ON/OFF e depois modificados e avaliados para desagregação. Os modelos foram ainda comparados com os do estado da arte presentes no NILMTK, onde apresentaram melhores resultados que a alternativa real-time, dAE, diminuindo o NRMSE em média 49% ficando próximos da melhor alternativa que classifica com atraso, Seq2Point, apresentando um erro pior, em média, de 17%. Para além disso, também analisamos os melhores modelos da experiência anterior no benefício de usar transfer learning entre datasets, onde os resultados mostram uma melhoria marginal quando usamos transfer learning. Este documento apresenta a definição do esboço da solução, as múltiplas opções consideradas para processamento de dataset e qual a melhor, a evolução dos modelos, os seus resultados e a comparação com os modelos do estado da arte na capacidade de generalização entre diferentes casas e de transfer learning entre datasets.Mestrado em Engenharia Informátic

Energy Data Analytics for Smart Meter Data

The principal advantage of smart electricity meters is their ability to transfer digitized electricity consumption data to remote processing systems. The data collected by these devices make the realization of many novel use cases possible, providing benefits to electricity providers and customers alike. This book includes 14 research articles that explore and exploit the information content of smart meter data, and provides insights into the realization of new digital solutions and services that support the transition towards a sustainable energy system. This volume has been edited by Andreas Reinhardt, head of the Energy Informatics research group at Technische Universität Clausthal, Germany, and Lucas Pereira, research fellow at Técnico Lisboa, Portugal

Machine learning techniques for sensor-based household activity recognition and forecasting

Thanks to the recent development of cheap and unobtrusive smart-home sensors, ambient assisted living tools promise to offer innovative solutions to support the users in carrying out their everyday activities in a smoother and more sustainable way. To be effective, these solutions need to constantly monitor and forecast the activities of daily living carried out by the inhabitants. The Machine Learning field has seen significant advancements in the development of new techniques, especially regarding deep learning algorithms. Such techniques can be successfully applied to household activity signal data to benefit the user in several applications. This thesis therefore aims to produce a contribution that artificial intelligence can make in the field of activity recognition and energy consumption. The effective recognition of common actions or the use of high-consumption appliances would lead to user profiling, thus enabling the optimisation of energy consumption in favour of the user himself or the energy community in general. Avoiding wasting electricity and optimising its consumption is one of the main objectives of the community. This work is therefore intended as a forerunner for future studies that will allow, through the results in this thesis, the creation of increasingly intelligent systems capable of making the best use of the user's resources for everyday life actions. Namely, this thesis focuses on signals from sensors installed in a house: data from position sensors, door sensors, smartphones or smart meters, and investigates the use of advanced machine learning algorithms to recognize and forecast inhabitant activities, including the use of appliances and the power consumption. The thesis is structured into four main chapters, each of which represents a contribution regarding Machine Learning or Deep Learning techniques for addressing challenges related to the aforementioned data from different sources. The first contribution highlights the importance of exploiting dimensionality reduction techniques that can simplify a Machine Learning model and increase its efficiency by identifying and retaining only the most informative and predictive features for activity recognition. In more detail, it is presented an extensive experimental study involving several feature selection algorithms and multiple Human Activity Recognition benchmarks containing mobile sensor data. In the second contribution, we propose a machine learning approach to forecast future energy consumption considering not only past consumption data, but also context data such as inhabitants’ actions and activities, use of household appliances, interaction with furniture and doors, and environmental data. We performed an experimental evaluation with real-world data acquired in an instrumented environment from a large user group. Finally, the last two contributions address the Non-Intrusive-Load-Monitoring problem. In one case, the aim is to identify the operating state (on/off) and the precise energy consumption of individual electrical loads, considering only the aggregate consumption of these loads as input. We use a Deep Learning method to disaggregate the low-frequency energy signal generated directly by the new generation smart meters being deployed in Italy, without the need for additional specific hardware. In the other case, driven by the need to build intelligent non-intrusive algorithms for disaggregating electrical signals, the work aims to recognize which appliance is activated by analyzing energy measurements and classifying appliances through Machine Learning techniques. Namely, we present a new way of approaching the problem by unifying Single Label (single active appliance recognition) and Multi Label (multiple active appliance recognition) learning paradigms. This combined approach, supplemented with an event detector, which suggests the instants of activation, would allow the development of an end-to-end NILM approach

Energy-Use Feedback Engineering - Technology and Information Design for Residential Users

The research presented in this study covers a first design iteration of energy feedback for residential users. This research contributes with a framework and new insights into the study of energy-use information for residential users, which exemplifies the challenges and potential of integrating information technology in this part of the energy system

Non-intrusive load monitoring: Use of low resolution steady state features to disaggregate household appliances

This work focuses on understanding capability and limits of low resolution steady state features to disaggregate domestic appliances. Features are active and eventually reactive power with sampling period of one or few seconds. Three algorithms have been used: Hart, Weiss and EICCA - NILM algorithms. This work details results for several appliance groups. Time of occurrence has been investigated as a possible feature, exposing results and limitations

Robust event-based non-intrusive appliance recognition using multi-scale wavelet packet tree and ensemble bagging tree

open access articleProviding the user with appliance-level consumption data is the core of each energy efficiency system. To that end, non-intrusive load monitoring is employed for extracting appliance specific consumption data at a low cost without the need of installing separate submeters for each electrical device. In this context, we propose in this paper a novel non-intrusive appliance recognition system based on (i) detecting events in the aggregated power signal using a novel and powerful scheme, (ii) applying multiscale wavelet packet tree to collect comprehensive energy consumption features, and (iii) adopting an ensemble bagging tree classifier along with comparing its performance with various machine learning schemes. Moreover, to validate the proposed model, an empirical investigation is conducted on two real and public energy consumption datasets, namely, the GREEND and REDD, in which consumption readings are collected at low-frequencies. In addition, a comprehensive review of recent non-intrusive load monitoring approaches has been conducted and presented, in which their characteristics, performances and limitations are described. The proposed non-intrusive load monitoring system shows a high appliance recognition performance in terms of the accuracy, F1 score and low time complexity when it has been applied to different households from the GREEND and REDD repositories, in which every house includes various domestic appliances. Obtained results have described, e.g., that average accuracies of 97.01% and 96.36% have been reached on the GREEND and REDD datasets, respectively, which outperformed almost existing solutions considered in this framework

Enhancing the efficiency of electricity utilization through home energy management systems within the smart grid framework

The concept behind smart grids is the aggregation of “intelligence” into the grid, whether through communication systems technologies that allow broadcast/data reception in real-time, or through monitoring and systems control in an autonomous way. With respect to the technological advancements, in recent years there has been a significant increment in devices and new strategies for the implementation of smart buildings/homes, due to the growing awareness of society in relation to environmental concerns and higher energy costs, so that energy efficiency improvements can provide real gains within modern society. In this perspective, the end-users are seen as active players with the ability to manage their energy resources, for example, microproduction units, domestic loads, electric vehicles and their participation in demand response events. This thesis is focused on identifying application areas where such technologies could bring benefits for their applicability, such as the case of wireless networks, considering the positive and negative points of each protocol available in the market. Moreover, this thesis provides an evaluation of dynamic prices of electricity and peak power, using as an example a system with electric vehicles and energy storage, supported by mixed-integer linear programming, within residential energy management. This thesis will also develop a power measuring prototype designed to process and determine the main electrical measurements and quantify the electrical load connected to a low voltage alternating current system. Finally, two cases studies are proposed regarding the application of model predictive control and thermal regulation for domestic applications with cooling requirements, allowing to minimize energy consumption, considering the restrictions of demand, load and acclimatization in the system