NILMTK: An Open Source Toolkit for Non-intrusive Load Monitoring

Non-intrusive load monitoring, or energy disaggregation, aims to separate household energy consumption data collected from a single point of measurement into appliance-level consumption data. In recent years, the field has rapidly expanded due to increased interest as national deployments of smart meters have begun in many countries. However, empirically comparing disaggregation algorithms is currently virtually impossible. This is due to the different data sets used, the lack of reference implementations of these algorithms and the variety of accuracy metrics employed. To address this challenge, we present the Non-intrusive Load Monitoring Toolkit (NILMTK); an open source toolkit designed specifically to enable the comparison of energy disaggregation algorithms in a reproducible manner. This work is the first research to compare multiple disaggregation approaches across multiple publicly available data sets. Our toolkit includes parsers for a range of existing data sets, a collection of preprocessing algorithms, a set of statistics for describing data sets, two reference benchmark disaggregation algorithms and a suite of accuracy metrics. We demonstrate the range of reproducible analyses which are made possible by our toolkit, including the analysis of six publicly available data sets and the evaluation of both benchmark disaggregation algorithms across such data sets.Comment: To appear in the fifth International Conference on Future Energy Systems (ACM e-Energy), Cambridge, UK. 201

The future of computing beyond Moore's Law.

Moore's Law is a techno-economic model that has enabled the information technology industry to double the performance and functionality of digital electronics roughly every 2 years within a fixed cost, power and area. Advances in silicon lithography have enabled this exponential miniaturization of electronics, but, as transistors reach atomic scale and fabrication costs continue to rise, the classical technological driver that has underpinned Moore's Law for 50 years is failing and is anticipated to flatten by 2025. This article provides an updated view of what a post-exascale system will look like and the challenges ahead, based on our most recent understanding of technology roadmaps. It also discusses the tapering of historical improvements, and how it affects options available to continue scaling of successors to the first exascale machine. Lastly, this article covers the many different opportunities and strategies available to continue computing performance improvements in the absence of historical technology drivers. This article is part of a discussion meeting issue 'Numerical algorithms for high-performance computational science'

Desagregação de consumos energéticos usando Machine Learning

Nowadays, we are surrounded by electric appliances. Either at home by the washing machine, kettle, or oven, or work by the computer, cellphone, or printer. Such devices help us daily, but their popularization increased the energy consumption to concerning values. In an attempt to reduce energy consumption, governments started enforcing policies regarding energy education to teach homeowners how to reduce energy wastage on the demand side. One of those policies was the deployment of smart meters, which allow the consumer to know how much energy is being consumed at any given time through a display on the household energy meter. Even though this measure was well received, the studies show that the best results in energy conservation are obtained through real-time appliance level feedback. To get such feedback, one can either measure every outlet in a household, which is unviable for a broad deployment solution, or disaggregate the energy recorded by the smart meter. NILM or Non-Intrusive Load Monitoring is the name we give to the second option where we use the aggregated readings of a household to find the energy consumed by each appliance. There were many proposals to solve NILM ranging from HMMs to GSP, where deep learning models showed remarkable results, obtaining state-of-the-art results. With the intent to create a complete NILM solution, Withus partnered with the University of Aveiro and proposed this dissertation. The initial objective was to develop a machine learning model to solve NILM. Still, during the background analysis, we found the need to create a new dataset which led to the expansion of the initial proposal to include the dataset preprocessing and conversion. Regarding NILM, we proposed three new deep learning models: a convolutional neural network with residual blocks, a recurrent neural network, and a multilayer perceptron that uses discrete wavelet transforms as features. These models went through multiple iterations, being evaluated first in the simpler ON/OFF classification task and later modified and evaluated for the disaggregation task. We compared our models to the state-of-the-art ones proposed in NILMTK, where they presented better results than the real-time alternative, dAE, reducing the NRMSE on average by 49%. We also got close to the best option that classified with a 30 min delay, Seq2Point, increasing the error on average by 17%. Besides that, we also analyze the best models from the previous comparison on the benefit of transfer learning between datasets, where the results show a marginal performance improvement when using transfer learning. This document presents the solution outline definition, the multiple options considered for dataset processing and the best solution, the models’ evolution and results, and the comparison with the state-of-the-art models regarding generalization to different houses and under transfer learning.Hoje em dia estamos rodeados de dispositivos elétricos. Quer seja em casa, pela máquina de lavar, o microondas ou o forno ou no emprego pelo computador, o telemóvel ou a impressora. Estes dispositivos ajudam-nos diariamente, mas com a sua popularização o consumo energético atingiu valores preocupantes. Numa tentativa de reduzir o consumo energético, os governos começaram a introduzir políticas de educação energética para ensinar os consumidores a reduzir o desperdício energético. Uma das medidas foi a implementação generalizada de smart meters, que permitem ao consumidor saber quanta energia está a ser consumida a qualquer altura através de um ecrã no contador da casa. Mesmo sendo bem recebida, esta medida não é suficiente uma vez que os estudos indicam que os melhores resultados são obtidos através de feedback ao nível do dispositivo em tempo real. Para obtermos este feedback existem duas formas, podemos medir cada tomada numa dada casa, o que é inviável para uma implementação em larga escala, ou desagregar a energia registrada pelo smart meter que já está presente na casa. NILM ou Non-Intrusive Load Monitoring é o nome dado à segunda opção onde a energia agregada da casa é usada para descobrirmos a energia consumida por cada dispositivo elétrico. Para resolver este problema foram propostas várias alternativas, desde HMMs a GSP, onde os modelos de deep learning obtiveram resultados notáveis sendo agora o estado da arte. Com o objetivo de produzir um sistema NILM completo, a Withus juntou-se à Universidade de Aveiro e juntos propuseram esta dissertação. O objetivo inicial era o desenvolvimento de um modelo de machine learning para desagregar consumos elétricos. Contudo, durante análise do estado da arte, deparamo-nos com a necessidade de criar um novo dataset, o que levou à extensão da proposta inicial para incluir também o pré-processamento e conversão do dataset. Para desagregação de consumos elétricos propusemos três modelos: uma rede neuronal convolucional com blocos residuais, uma rede neuronal recorrente e um multilayer perceptron que usa discrete wavelet transforms como features. Estes modelos passaram por diversas iterações, sendo avaliados primeiro na tarefa de classificação ON/OFF e depois modificados e avaliados para desagregação. Os modelos foram ainda comparados com os do estado da arte presentes no NILMTK, onde apresentaram melhores resultados que a alternativa real-time, dAE, diminuindo o NRMSE em média 49% ficando próximos da melhor alternativa que classifica com atraso, Seq2Point, apresentando um erro pior, em média, de 17%. Para além disso, também analisamos os melhores modelos da experiência anterior no benefício de usar transfer learning entre datasets, onde os resultados mostram uma melhoria marginal quando usamos transfer learning. Este documento apresenta a definição do esboço da solução, as múltiplas opções consideradas para processamento de dataset e qual a melhor, a evolução dos modelos, os seus resultados e a comparação com os modelos do estado da arte na capacidade de generalização entre diferentes casas e de transfer learning entre datasets.Mestrado em Engenharia Informátic

M-sequenze based ultra-wideband radar and its application to crack detection in salt mines

Die vorliegende Dissertation beschreibt einen innovativen ultra-breitband (UWB)elektromagnetischen Sensor basierend auf einem Pseudo-Rauschverfahren.Der Sensor wurde für zerstörungsfreies Testen in zivilen Anwendungen entwickelt.Zerstörungsfreies Testen entwickelt sich zu einem immer wichtiger werdenden Bereich in Forschung und Entwicklung. Neben unzähligen weiteren Anwendungen und Technologien, besteht ein primäres Aufgabenfeld in der Überwachung und Untersuchung von Bauwerken und Baumaterialien durch berührungslose Messung aus der Ferne.Diese Arbeit konzentriert sich auf das Beispiel der Auflockerungszone im Salzgestein.Der Hintergrund und die Notwendigkeit, den Zustand der oberflächennahen Salzschichten in Salzminen kennen zu müssen, werden beleuchtet und die Messaufgabe anhand einfacher theoretischer Überlegungen beschrieben. Daraus werden die Anforderungen für geeignete UWB Sensoren abgeleitet. Die wichtigsten Eigenschaften sind eine sehr hohe Messband breite sowie eine sehr saubere Systemimpulsantwort frei von systematischen Gerätefehlern. Beide Eigenschaften sind notwendig, um die schwachen Rückstreuungen der Auflockerungen trotz der unvermeidlichen starken Oberflächenreflexion detektieren zu können.Da systematische Fehler bei UWB Geräten technisch nicht von vorne herein komplett vermeidbar sind, muss der Sensor eine Gerätekalibrierung erlauben, um solche Fehler möglichst gut zu unterdrücken.Aufgrund der genannten Anforderungen und den Nebenbedingungen der Messumgebung unter Tage, wurde aus den verschiedenen UWB-Technologien ein Prinzip ausgewählt, welches pseudozufällige Maximalfolgen als Anregungssignal benutzt. Das M-Sequenzkonzept dient als Ausgangpunkt für zahlreiche Weiterentwicklungen. Ein neues Sendemodul erweitert dabei die Messbandbreite auf 12~GHz. Die äquivalente Abtastrate wird um den Faktor vier auf 36~GHz erhöht, ohne den geringen Abtastjitter des ursprünglichen Konzepts zu vergrössern.Weiterhin wird die Umsetzung eines Zweitormesskopfes zur Erfassung von S-Parametern sowie einer automatische Kalibriereinheit beschrieben. Etablierte Kalibrierverfahren aus dem Bereich der Netzwerkanalyse werden kurz rekapituliert und die Adaption des 8-Term Verfahrens mit unbekanntem Transmissionsnormal für das M-Sequenzsystem beschrieben. Dabei werden Kennwerte vorgeschlagen, die dem Bediener unter Tage einfach erlauben, die Kalibrierqualität einzuschätzen und Hinweise auf mögliche Gerätefehler oder andere Probleme zu bekommen. Die Kalibriergenauigkeit des neuen Sensors im Labor wird mit der eines Netzwerkanalysators verglichen. Beide Geräte erreichen eine störungsfreie Dynamik von mehr als 60~dB in den Systemimpulsantworten für Reflexion und Transmission.Der neu entwickelte UWB Sensor wurde in zahlreichen Messungen in Salzminen in Deutschland getestet. Zwei Messbeispiele werden vorgestellt - ein sehr alter, kreisrunder Tunnel sowie ein ovaler Tunnelstumpf, welcher kurz vor den Messungen erst aufgefahren wurde. Messaufbauten und Datenverarbeitung werden beschrieben. Schließlich werden Schlussfolgerungen und Vorschläge für zukünftige Arbeiten mit dem neuen M-Sequenzsensor sowie der Messung von Auflockerungen im Salzgestein diskutiert.This dissertation describes an innovative ultra-wideband (UWB) electromagnetic sensor device based on a pseudo-noise principle developed in the context of non-destructive testing in civil engineering.Non-destructive testing is becoming a more and more important fieldfor researchers and engineers alike. Besides the vast field of possibleapplications and testing technologies, a prime and therefore typical topic is the inspection and monitoringof constructions and materials by means of contactless remote sensing techniques.This work focuses on one example the assessment of the disaggregation zone in salt rock tunnels.The background and relevance of knowing the state of salt rock layers near a tunnel's surface are explainedand simple theoretical considerations for requirements of suitable UWB sensor devices are shown. The most important sensor parameters are a very large measurement bandwidth and a very clean impulse response. The latterparameter translates into the mandatory application of calibration techniques to remove systematic errors of the sensor system itself. This enables detection of weak scattering responses from near-surface disaggregation despite the presence of a strong surface reflection.According to the mentioned requirements and other side conditions in salt mine environments an UWB sensor principlebased on pseudo-noise stimuli namely M-Sequences is selected as a starting point for system development. A newtransmitter frontend for extending the stimulus bandwidth up to 12~GHz is presented. Furthermore, a technique for increasing the (equivalent) sampling rate while keeping the stable and low-jitter sampling regime of the M-Sequencesapproach is introduced and its implementation is shown. Moreover, an automatic calibration unit for full two-port coaxial calibration of the new UWB sensor has been developed. Common calibration techniques from the area of vector network analysers are shortly reviewed and a reasonablealgorithm the 8-term method with an unknown line standard - is selected for the M-Sequences device. The 8-term method is defined in the frequency domain and is adapted for use with time domain devices. Some performance figures and comparisonwith calibration results from network analysers are discussed to show the effectiveness of the calibration.A spurious-free dynamic range of the time domain impulse responses in excess of 60~dB has been achieved for reflection as well as transmission measurements.The new UWB sensor was used in various real world measurements in different salt mines throughout Germany. Two measurementexamples are described and results from the disaggregation zone of a very old and a freshly cut tunnel will be presented. Measurement setup and data processing are discussed and finally some conclusions for future work on this topic are drawn

Thresholding methods in non-intrusive load monitoring

Non-intrusive load monitoring (NILM) is the problem of predicting the status or consumption of individual domestic appliances only from the knowledge of the aggregated power load. NILM is often formulated as a classifcation (ON/OFF) problem for each device. However, the training datasets gathered by smart meters do not contain these labels, but only the electric consumption at every time interval. This paper addresses a fundamental methodological problem in how a NILM problem is posed, namely how the diferent possible thresholding methods lead to diferent classifcation problems. Standard datasets and NILM deep learning models are used to illustrate how the choice of thresholding method afects the output results. Some criteria that should be considered for the choice of such methods are also proposed. Finally, we propose a slight modifcation to current deep learning models for multi-tasking, i.e. tackling the classifcation and regression problems simultaneously. Transfer learning between both problems might improve performance on each of them.Funding for open access publishing: Universidad de Cádiz/CBUA. This research has been financed in part by the Spanish Agencia Estatal de Investigación under grants PID2021-122154NB-I00 and TED2021-129455B-I00, and by a 2021 BBVA Foundation project for research in Mathematics. He also acknowledges support from the EU under the 2014-2020 ERDF Operational Programme and the Department of Economy, Knowledge, Business and University of the Regional Government of Andalusia (FEDER-UCA18-108393)

Deep Learning Applications in non-intrusive load monitoring

Within the frame of the project Non-Intrusive Load Monitoring for Intelligent Home Energy Management Systems, this work will present a deep learning application in non-intrusive load monitoring on a case study in a residential home in in Gambelas, Faro in the Algarve region south of Portugal. This work has for a goal to detect type 2 appliances in different houses. For the sake of this study, two models will be trained: - Convolutional Neural Network - Long Short-term Memory Recurrent Neural Network on three datasets: - UKDale - REDD - Data from the Portuguese private residential house from the project NILM for IHEMS.No âmbito do projeto Monitorização de Carga Não Intrusiva para Sistemas Inteligentes de Gestão de Energia Doméstica, este trabalho apresentará uma aplicação de aprendizagem profunda na monitorização de carga não intrusiva num estudo de caso numa casa residencial em Gambelas, Faro na região sul do Algarve de Portugal. Este trabalho tem por objetivo detectar eletrodomésticos tipo 2 em diferentes residências. Para fins deste estudo, dois modelos serão treinados: - Rede Neural Convolucional - Rede Neural Recorrente de Memória Longa de Curto Prazo em três conjuntos de dados: - UKDale - REDD - Dados da habitação privada portuguesa do projecto NILM para IHEMS

Artificial Intelligence based Anomaly Detection of Energy Consumption in Buildings: A Review, Current Trends and New Perspectives

Enormous amounts of data are being produced everyday by sub-meters and smart sensors installed in residential buildings. If leveraged properly, that data could assist end-users, energy producers and utility companies in detecting anomalous power consumption and understanding the causes of each anomaly. Therefore, anomaly detection could stop a minor problem becoming overwhelming. Moreover, it will aid in better decision-making to reduce wasted energy and promote sustainable and energy efficient behavior. In this regard, this paper is an in-depth review of existing anomaly detection frameworks for building energy consumption based on artificial intelligence. Specifically, an extensive survey is presented, in which a comprehensive taxonomy is introduced to classify existing algorithms based on different modules and parameters adopted, such as machine learning algorithms, feature extraction approaches, anomaly detection levels, computing platforms and application scenarios. To the best of the authors' knowledge, this is the first review article that discusses anomaly detection in building energy consumption. Moving forward, important findings along with domain-specific problems, difficulties and challenges that remain unresolved are thoroughly discussed, including the absence of: (i) precise definitions of anomalous power consumption, (ii) annotated datasets, (iii) unified metrics to assess the performance of existing solutions, (iv) platforms for reproducibility and (v) privacy-preservation. Following, insights about current research trends are discussed to widen the applications and effectiveness of the anomaly detection technology before deriving future directions attracting significant attention. This article serves as a comprehensive reference to understand the current technological progress in anomaly detection of energy consumption based on artificial intelligence.Comment: 11 Figures, 3 Table