Energy Data Analytics for Smart Meter Data

The principal advantage of smart electricity meters is their ability to transfer digitized electricity consumption data to remote processing systems. The data collected by these devices make the realization of many novel use cases possible, providing benefits to electricity providers and customers alike. This book includes 14 research articles that explore and exploit the information content of smart meter data, and provides insights into the realization of new digital solutions and services that support the transition towards a sustainable energy system. This volume has been edited by Andreas Reinhardt, head of the Energy Informatics research group at Technische Universität Clausthal, Germany, and Lucas Pereira, research fellow at Técnico Lisboa, Portugal

Contribuitions and developments on nonintrusive load monitoring

Energy efficiency is a key subject in our present world agenda, not only because of greenhouse gas emissions, which contribute to global warming, but also because of possible supply interruptions. In Brazil, energy wastage in the residential market is estimated to be around 15%. Previous studies have indicated that the most savings were achieved with specific appliance, electricity consumption feedback, which caused behavioral changes and encouraged consumers to pursue energy conservation. Nonintrusive Load Monitoring (NILM) is a relatively new term. It aims to disaggregate global consumption at an appliance level, using only a single point of measurement. Various methods have been suggested to infer when appliances are turned on and off, using the analysis of current and voltage aggregated waveforms. Within this context, we aim to provide a methodology for NILM to determine which sets of electrical features and feature extraction rates, obtained from aggregated household data, are essential to preserve equivalent levels of accuracy; thus reducing the amount of data that needs to be transferred to, and stored on, cloud servers. As an addendum to this thesis, a Brazilian appliance dataset, sampled from real appliances, was developed for future NILM developments and research. Beyond that, a low-cost NILM smart meter was developed to encourage consumers to change their habits to more sustainable methods.Eficiência energética é um assunto essencial na agenda mundial. No Brasil, o desperdício de energia no setor residencial é estimado em 15%. Estudos indicaram que maiores ganhos em eficiência são conseguidos quando o usuário recebe as informações de consumo detalhadas por cada aparelho, provocando mudanças comportamentais e incentivando os consumidores na conservação de energia. Monitoramento não intrusivo de cargas (NILM da sigla em inglês) é um termo relativamente novo. A sua finalidade é inferir o consumo de um ambiente até observar os consumos individualizados de cada equipamento utilizando-se de apenas um único ponto de medição. Métodos sofisticados têm sido propostos para inferir quando os aparelhos são ligados e desligados em um ambiente. Dentro deste contexto, este trabalho apresenta uma metodologia para a definição de um conjunto mínimo de características elétricas e sua taxa de extração que reduz a quantidade de dados a serem transmitidos e armazenados em servidores de processamento de dados, preservando níveis equivalentes de acurácia. São utilizadas diferentes técnicas de aprendizado de máquina visando à caracterização e solução do problema. Como adendo ao trabalho, apresenta-se um banco de dados de eletrodomésticos brasileiros, com amostras de equipamentos nacionais para desenvolvimentos futuros em NILM, além de um medidor inteligente de baixo custo para desagregação de cargas, visando tornar o consumo de energia mais sustentável

Machine learning techniques for sensor-based household activity recognition and forecasting

Thanks to the recent development of cheap and unobtrusive smart-home sensors, ambient assisted living tools promise to offer innovative solutions to support the users in carrying out their everyday activities in a smoother and more sustainable way. To be effective, these solutions need to constantly monitor and forecast the activities of daily living carried out by the inhabitants. The Machine Learning field has seen significant advancements in the development of new techniques, especially regarding deep learning algorithms. Such techniques can be successfully applied to household activity signal data to benefit the user in several applications. This thesis therefore aims to produce a contribution that artificial intelligence can make in the field of activity recognition and energy consumption. The effective recognition of common actions or the use of high-consumption appliances would lead to user profiling, thus enabling the optimisation of energy consumption in favour of the user himself or the energy community in general. Avoiding wasting electricity and optimising its consumption is one of the main objectives of the community. This work is therefore intended as a forerunner for future studies that will allow, through the results in this thesis, the creation of increasingly intelligent systems capable of making the best use of the user's resources for everyday life actions. Namely, this thesis focuses on signals from sensors installed in a house: data from position sensors, door sensors, smartphones or smart meters, and investigates the use of advanced machine learning algorithms to recognize and forecast inhabitant activities, including the use of appliances and the power consumption. The thesis is structured into four main chapters, each of which represents a contribution regarding Machine Learning or Deep Learning techniques for addressing challenges related to the aforementioned data from different sources. The first contribution highlights the importance of exploiting dimensionality reduction techniques that can simplify a Machine Learning model and increase its efficiency by identifying and retaining only the most informative and predictive features for activity recognition. In more detail, it is presented an extensive experimental study involving several feature selection algorithms and multiple Human Activity Recognition benchmarks containing mobile sensor data. In the second contribution, we propose a machine learning approach to forecast future energy consumption considering not only past consumption data, but also context data such as inhabitants’ actions and activities, use of household appliances, interaction with furniture and doors, and environmental data. We performed an experimental evaluation with real-world data acquired in an instrumented environment from a large user group. Finally, the last two contributions address the Non-Intrusive-Load-Monitoring problem. In one case, the aim is to identify the operating state (on/off) and the precise energy consumption of individual electrical loads, considering only the aggregate consumption of these loads as input. We use a Deep Learning method to disaggregate the low-frequency energy signal generated directly by the new generation smart meters being deployed in Italy, without the need for additional specific hardware. In the other case, driven by the need to build intelligent non-intrusive algorithms for disaggregating electrical signals, the work aims to recognize which appliance is activated by analyzing energy measurements and classifying appliances through Machine Learning techniques. Namely, we present a new way of approaching the problem by unifying Single Label (single active appliance recognition) and Multi Label (multiple active appliance recognition) learning paradigms. This combined approach, supplemented with an event detector, which suggests the instants of activation, would allow the development of an end-to-end NILM approach

Non-intrusive load disaggregation using graph signal processing

With the large-scale roll-out of smart metering worldwide, there is a growing need to account for the individual contribution of appliances to the load demand. In this paper, we design a Graph signal processing (GSP)-based approach for non-intrusive appliance load monitoring (NILM), i.e., disaggregation of total energy consumption down to individual appliances used. Leveraging piecewise smoothness of the power load signal, two GSP-based NILM approaches are proposed. The first approach, based on total graph variation minimization, searches for a smooth graph signal under known label constraints. The second approach uses the total graph variation minimizer as a starting point for further refinement via simulated annealing. The proposed GSP-based NILM approach aims to address the large training overhead and associated complexity of conventional graph-based methods through a novel event-based graph approach. Simulation results using two datasets of real house measurements demonstrate the competitive performance of the GSP-based approaches with respect to traditionally used Hidden Markov Model-based and Decision Tree-based approaches

A new method for residential side non-intrusive load monitoring

This thesis proposes a new non-intrusive method for residential load monitoring. The proposed method can detect appliance switching events, separate appliance electric features, and identify appliance types. Compared with other non-intrusive monitoring methods, the proposed method improves the monitoring accuracy and decreases the monitoring response time. Firstly, the monitoring hardware was designed and constructed to sample and acquire the aggregated electric data of one residential area. Secondly, the sampled data were processed and analysed with the proposed method, which achieves the monitoring of individual appliance running conditions and power consumption in this area in a non-intrusive way. The data analysis process includes three steps, 1) the appliance switching event is detected by the Heuristic detection method. 2) the working current of the switched appliance is separated according to the difference method, 3) the type of switched appliance is identified with the K-nearest neighbour method according to the appliance’s current harmonic components, and the identification result is modified and corrected according to appliance operation pattern with the aid of a Back Propagation Neural Network. Thirdly, the proposed NILM method was tested through offline and online applications. The offline application involves three days of pre-recorded data which were processed and analysed. The online application consists of two parts. The first part is a direct application for four domestic homes during one day (24 hours). As for the second part, the proposed monitoring method was applied to one domestic home for ninety days. All the online and offline tests, the running conditions and the power consumption of appliances were monitored and recorded. Due to the test results, the proposed method is reliable and offers a powerful monitoring method for demand side management.This thesis proposes a new non-intrusive method for residential load monitoring. The proposed method can detect appliance switching events, separate appliance electric features, and identify appliance types. Compared with other non-intrusive monitoring methods, the proposed method improves the monitoring accuracy and decreases the monitoring response time. Firstly, the monitoring hardware was designed and constructed to sample and acquire the aggregated electric data of one residential area. Secondly, the sampled data were processed and analysed with the proposed method, which achieves the monitoring of individual appliance running conditions and power consumption in this area in a non-intrusive way. The data analysis process includes three steps, 1) the appliance switching event is detected by the Heuristic detection method. 2) the working current of the switched appliance is separated according to the difference method, 3) the type of switched appliance is identified with the K-nearest neighbour method according to the appliance’s current harmonic components, and the identification result is modified and corrected according to appliance operation pattern with the aid of a Back Propagation Neural Network. Thirdly, the proposed NILM method was tested through offline and online applications. The offline application involves three days of pre-recorded data which were processed and analysed. The online application consists of two parts. The first part is a direct application for four domestic homes during one day (24 hours). As for the second part, the proposed monitoring method was applied to one domestic home for ninety days. All the online and offline tests, the running conditions and the power consumption of appliances were monitored and recorded. Due to the test results, the proposed method is reliable and offers a powerful monitoring method for demand side management

Lockdown impacts on residential electricity demand in India: A data-driven and non-intrusive load monitoring study using Gaussian mixture models.

This study evaluates the effect of complete nationwide lockdown in 2020 on residential electricity demand across 13 Indian cities and the role of digitalisation using a public smart meter dataset. We undertake a data-driven approach to explore the energy impacts of work-from-home norms across five dwelling typologies. Our methodology includes climate correction, dimensionality reduction and machine learning-based clustering using Gaussian Mixture Models of daily load curves. Results show that during the lockdown, maximum daily peak demand increased by 150-200% as compared to 2018 and 2019 levels for one room-units (RM1), one bedroom-units (BR1) and two bedroom-units (BR2) which are typical for low- and middle-income families. While the upper-middle- and higher-income dwelling units (i.e., three (3BR) and more-than-three bedroom-units (M3BR)) saw night-time demand rise by almost 44% in 2020, as compared to 2018 and 2019 levels. Our results also showed that new peak demand emerged for the lockdown period for RM1, BR1 and BR2 dwelling typologies. We found that the lack of supporting socioeconomic and climatic data can restrict a comprehensive analysis of demand shocks using similar public datasets, which informed policy implications for India's digitalisation. We further emphasised improving the data quality and reliability for effective data-centric policymaking

Performance Evaluation of Three Virtual Metering Methods to Estimate Zone-level Perimeter Heater Energy Requirement

Virtual metering provides a cost-effective alternative to physical meters to monitor building energy performance and capture unmetered energy flows at the zone-level. Virtual metering accuracy depends on the modelling method and its ability to represent the heating and cooling processes at a building thermal zone. This paper employs three virtual metering methods to estimate the heating energy of zone-level perimeter heaters: a steady-state modelling method, a transient modelling method, and a load disaggregation modelling method. Inverse models representing these three virtual metering methods are trained using data obtained from seven perimeter offices in an academic building in Ottawa, Canada. Model parameters are identified using the genetic algorithm and used for creating virtual meters that estimate the energy requirement of zone-level perimeter heaters. The virtual meters\u27 accuracy is assessed by comparing the results to measured heating energy obtained from physical meters installed in the seven offices. The three virtual metering methods\u27 performance is evaluated through illustrative examples in terms of modelling assumptions, data requirements, and virtual metering accuracy. The results indicate that the three virtual metering methods can estimate the daily heating energy supplied by perimeter heaters at a normalized root-mean-square error between 13% and 23%

Comprehensive feature selection for appliance classification in NILM

Since the inception of non-intrusive appliance load monitoring (NILM), extensive research has focused on identifying an effective set of features that allows to form a unique appliance signature to discriminate various loads. Although an abundance of features are reported in literature, most works use only a limited subset of them. A systematic comparison and combination of the available features in terms of their effectiveness is still missing. This paper, as its first contribution, offers a concise and updated review of the features reported in literature for the purpose of load identification. As a second contribution, a systematic feature elimination process is proposed to identify the most effective feature set. The analysis is validated on a large benchmark dataset and shows that the proposed feature elimination process improves the appliance classification accuracy for all the appliances in the dataset compared to using all the features or randomly chosen subsets of features. (C) 2017 Elsevier B.V. All rights reserved

Hardware and software platforms to deploy and evaluate non-intrusive load monitoring systems

The work in this PhD thesis addresses the practical implications of deploying and testing Non-Intrusive Load Monitoring (NILM) and eco-feedback solutions in real-world scenarios. The contributions to this topic are centered around the design and development of NILM frameworks that have been deployed in the wild, supporting long-term research in ecofeedback and also serving the purpose of producing real-world datasets and furthering the state of the art regarding the performance metrics used to evaluate NILM algorithms. This thesis consists of three main parts: i) the development of tools and datasets for NILM and eco-feedback research, ii) the design, implementation and deployment of NILM and eco-feedback technologies in real world scenarios, and iii) an experimental comparison of performance metrics for event detection and event classification algorithms. In the first part we describe the Energy Monitoring and Disaggregation Data Format (EMD-DF) and the SustData and SustDataED public datasets. In second part we discuss the development and deployment of two hardware and software platforms in real households, to support eco-feedback research. We then report on more than five years of experience in deploying and maintaining such platforms. Our findings suggest that the main practical issues can be divided in two categories, technological (e.g., system installation) and social (e.g., maintaining a steady sample throughout the whole study). In the final part of this thesis we analyze experimentally the behavior of a number of performance metrics for event detection and event classification, identifying clusters and relationships between the different measures. Our results evidence some considerable differences in the behavior of the performance metrics when applied to the different problems.O trabalho desenvolvido nesta tese de doutoramento aborda as implicações praticas da instalação e avaliação de soluções de monitorização não intrusiva de cargas elétricas (NILM) e eco-feedback em cenários reais. As contribuições para este tópico estão centradas em torno da concepção e desenvolvimento de plataformas NILM que foram instaladas em ambientes não controlados, suportando a pesquisa de longo termo em eco-feedback e servindo também o propósito de produzir conjuntos de dados científicos, bem como promover o avanço do estado da arte acerca das métricas de desempenho utilizadas para avaliar algoritmos NILM. Esta tese é constituída por três partes principais: i) o desenvolvimento de ferramentas e conjuntos de dados científicos para investigação em NILM e eco-feedback, ii) a concepção, desenho e instalação de tecnologias NILM e eco-feedback em cenários reais, e iii) uma comparação experimental de métricas de desempenho para algoritmos de detecção e de classificação de eventos. Na primeira parte descrevemos o Energy Monitoring and Disaggregation Data Format (EMD-DF) e os conjuntos de dados científicos SustData e SustDataED. Na segunda parte discutimos o desenvolvimento e instalação de duas plataformas de hardware e software em residências atuais com a finalidade de suportar a investigação em eco-feedback. Aqui, reportamos sobre mais de cinco anos de experiência na instalação e manutenção destes sistemas. Os nossos resultados sugerem que as principais implicações práticas podem ser divididas em duas categorias, físicas (e.g., instalação do sistema) e sociais (e.g., manter uma amostra constante ao longo de todo o estudo). Na terceira parte analisamos experimentalmente o comportamento de uma série de métricas de desempenho quando estas são utilizadas para avaliar algoritmos de detecção e de classificação de eventos. Calculamos as correlações lineares e não lineares entre os vários pares de métricas, e com base nesses valores procuramos agrupar as métricas que evidenciam um comportamento semelhante. Os nossos resultados sugerem a existência de diferenças evidentes no comportamento das métricas quando aplicadas a ambos dos problemas.Fundação para a Ciência e a Tecnologi

Load classification and appliance fingerprinting for residential load monitoring system

Previous work on residential load monitoring has attempted to address different requirements including the systematic collection of information about electric power consumption for load research purpose, the provision of a detailed consumption report to facilitate energy conservation practices and the monitoring of critical loads for fault diagnostics. This work focuses on developing methods for appliance fingerprinting that is foreseen to be an integral part of an automatic residential load monitoring system. Various approaches outlined in previous research form the basis for the concepts developed in this thesis. In addition, an extensive series of measurement work was performed on several household appliances in order to acquire the necessary operation data for building the technique and also to explore the extent up to which residential loads can be categorized into distinct groups. The fingerprinting process proposed in this work employs three main phases: feature extraction of electrical attributes, event detection and pattern recognition. Test results obtained at different stages of the work using the measurement data are also discussed in detail. Such studies are necessary to enable utilities to manage their networks reliably and efficiently, and also to encourage the active participation of consumers in energy conservation programs