460 research outputs found
Managing uncertainty in integrated environmental modelling:the UncertWeb framework
Web-based distributed modelling architectures are gaining increasing recognition as potentially useful tools to build holistic environmental models, combining individual components in complex workflows. However, existing web-based modelling frameworks currently offer no support for managing uncertainty. On the other hand, the rich array of modelling frameworks and simulation tools which support uncertainty propagation in complex and chained models typically lack the benefits of web based solutions such as ready publication, discoverability and easy access. In this article we describe the developments within the UncertWeb project which are designed to provide uncertainty support in the context of the proposed ‘Model Web’. We give an overview of uncertainty in modelling, review uncertainty management in existing modelling frameworks and consider the semantic and interoperability issues raised by integrated modelling. We describe the scope and architecture required to support uncertainty management as developed in UncertWeb. This includes tools which support elicitation, aggregation/disaggregation, visualisation and uncertainty/sensitivity analysis. We conclude by highlighting areas that require further research and development in UncertWeb, such as model calibration and inference within complex environmental models
Advancing Data Collection, Management, and Analysis for Quantifying Residential Water Use via Low Cost, Open Source, Smart Metering Infrastructure
Urbanization, climate change, aging infrastructure, and the cost of delivering water to residential customers make it vital that we achieve a higher efficiency in the management of urban water resources. Understanding how water is used at the household level is vital for this objective.Water meters measure water use for billing purposes, commonly at a monthly, or coarser temporal resolutions. This is insufficient to understand where water is used (i.e., the distribution of water use across different fixtures like toilets, showers, outdoor irrigation), when water is used (i.e., identifying peaks of consumption, instantaneous or at hourly, daily, weekly intervals), the efficiency of water using fixtures, or water use behaviors across different households. Most smart meters available today are not capable of collecting data at the temporal resolutions needed to fully characterize residential water use, and managing this data represents a challenge given the rapidly increasing volume of data generated. The research in this dissertation presents low cost, open source cyberinfrastructure (datalogging and data management systems) to collect and manage high temporal resolution, residential water use data. Performance testing of the cyberinfrastructure demonstrated the scalability of the system to multiple hundreds of simultaneous data collection devices. Using this cyberinfrastructure, we conducted a case study application in the cities of Logan and Providence, Utah where we found significant variability in the temporal distribution, timing, and volumes of indoor water use. This variability can impact the design of water conservation programs, estimations and forecast of water demand, and sizing of future water infrastructure. Outdoor water use was the largest component of residential water use, yet homeowners were not significantly overwatering their landscapes. Opportunities to improve the efficiency of water using fixtures and to conserve water by promoting behavior changes exist among participants
Hardware and software platforms to deploy and evaluate non-intrusive load monitoring systems
The work in this PhD thesis addresses the practical implications of deploying and testing Non-Intrusive Load Monitoring (NILM) and eco-feedback solutions in real-world scenarios. The contributions to this topic are centered around the design and development of NILM frameworks that have been deployed in the wild, supporting long-term research in ecofeedback and also serving the purpose of producing real-world datasets and furthering the state of the art regarding the performance metrics used to evaluate NILM algorithms. This thesis consists of three main parts: i) the development of tools and datasets for NILM and eco-feedback research, ii) the design, implementation and deployment of NILM and eco-feedback technologies in real world scenarios, and iii) an experimental comparison of performance metrics for event detection and event classification algorithms. In the first part we describe the Energy Monitoring and Disaggregation Data Format (EMD-DF) and the SustData and SustDataED public datasets. In second part we discuss the development and deployment of two hardware and software platforms in real households, to support eco-feedback research. We then report on more than five years of experience in deploying and maintaining such platforms. Our findings suggest that the main practical issues can be divided in two categories, technological (e.g., system installation) and social (e.g., maintaining a steady sample throughout the whole study). In the final part of this thesis we analyze experimentally the behavior of a number of performance metrics for event detection and event classification, identifying clusters and relationships between the different measures. Our results evidence some considerable differences in the behavior of the performance metrics when applied to the different problems.O trabalho desenvolvido nesta tese de doutoramento aborda as implicações praticas da instalação e avaliação de soluções de monitorização não intrusiva de cargas elétricas (NILM) e eco-feedback em cenários reais. As contribuições para este tópico estão centradas em torno da concepção e desenvolvimento de plataformas NILM que foram instaladas em ambientes não controlados, suportando a pesquisa de longo termo em eco-feedback e servindo também o propósito de produzir conjuntos de dados científicos, bem como promover o avanço do estado da arte acerca das métricas de desempenho utilizadas para avaliar algoritmos NILM. Esta tese é constituída por três partes principais: i) o desenvolvimento de ferramentas e conjuntos de dados científicos para investigação em NILM e eco-feedback, ii) a concepção, desenho e instalação de tecnologias NILM e eco-feedback em cenários reais, e iii) uma comparação experimental de métricas de desempenho para algoritmos de detecção e de classificação de eventos. Na primeira parte descrevemos o Energy Monitoring and Disaggregation Data Format (EMD-DF) e os conjuntos de dados científicos SustData e SustDataED. Na segunda parte discutimos o desenvolvimento e instalação de duas plataformas de hardware e software em residências atuais com a finalidade de suportar a investigação em eco-feedback. Aqui, reportamos sobre mais de cinco anos de experiência na instalação e manutenção destes sistemas. Os nossos resultados sugerem que as principais implicações práticas podem ser divididas em duas categorias, físicas (e.g., instalação do sistema) e sociais (e.g., manter uma amostra constante ao longo de todo o estudo). Na terceira parte analisamos experimentalmente o comportamento de uma série de métricas de desempenho quando estas são utilizadas para avaliar algoritmos de detecção e de classificação de eventos. Calculamos as correlações lineares e não lineares entre os vários pares de métricas, e com base nesses valores procuramos agrupar as métricas que evidenciam um comportamento semelhante. Os nossos resultados sugerem a existência de diferenças evidentes no comportamento das métricas quando aplicadas a ambos dos problemas.Fundação para a Ciência e a Tecnologi
Non-intrusive load monitoring solutions for low- and very low-rate granularity
Strathclyde theses - ask staff. Thesis no. : T15573Large-scale smart energy metering deployment worldwide and the integration of smart meters within the smart grid are enabling two-way communication between the consumer and energy network, thus ensuring an improved response to demand. Energy disaggregation or non-intrusive load monitoring (NILM), namely disaggregation of the total metered electricity consumption down to individual appliances using purely algorithmic tools, is gaining popularity as an added-value that makes the most of meter data.In this thesis, the first contribution tackles low-rate NILM problem by proposing an approach based on graph signal processing (GSP) that does not require any training.Note that Low-rate NILM refers to NILM of active power measurements only, at rates from 1 second to 1 minute. Adaptive thresholding, signal clustering and pattern matching are implemented via GSP concepts and applied to the NILM problem. Then for further demonstration of GSP potential, GSP concepts are applied at both, physical signal level via graph-based filtering and data level, via effective semi-supervised GSP-based feature matching. The proposed GSP-based NILM-improving methods are generic and can be used to improve the results of various event-based NILM approaches. NILM solutions for very low data rates (15-60 min) cannot leverage on low to highrates NILM approaches. Therefore, the third contribution of this thesis comprises three very low-rate load disaggregation solutions, based on supervised (i) K-nearest neighbours relying on features such as statistical measures of the energy signal, time usage profile of appliances and reactive power consumption (if available); unsupervised(ii) optimisation performing minimisation of error between aggregate and the sum of estimated individual loads, where energy consumed by always-on load is heuristically estimated prior to further disaggregation and appliance models are built only by manufacturer information; and (iii) GSP as a variant of aforementioned GSP-based solution proposed for low-rate load disaggregation, with an additional graph of time-of-day information.Large-scale smart energy metering deployment worldwide and the integration of smart meters within the smart grid are enabling two-way communication between the consumer and energy network, thus ensuring an improved response to demand. Energy disaggregation or non-intrusive load monitoring (NILM), namely disaggregation of the total metered electricity consumption down to individual appliances using purely algorithmic tools, is gaining popularity as an added-value that makes the most of meter data.In this thesis, the first contribution tackles low-rate NILM problem by proposing an approach based on graph signal processing (GSP) that does not require any training.Note that Low-rate NILM refers to NILM of active power measurements only, at rates from 1 second to 1 minute. Adaptive thresholding, signal clustering and pattern matching are implemented via GSP concepts and applied to the NILM problem. Then for further demonstration of GSP potential, GSP concepts are applied at both, physical signal level via graph-based filtering and data level, via effective semi-supervised GSP-based feature matching. The proposed GSP-based NILM-improving methods are generic and can be used to improve the results of various event-based NILM approaches. NILM solutions for very low data rates (15-60 min) cannot leverage on low to highrates NILM approaches. Therefore, the third contribution of this thesis comprises three very low-rate load disaggregation solutions, based on supervised (i) K-nearest neighbours relying on features such as statistical measures of the energy signal, time usage profile of appliances and reactive power consumption (if available); unsupervised(ii) optimisation performing minimisation of error between aggregate and the sum of estimated individual loads, where energy consumed by always-on load is heuristically estimated prior to further disaggregation and appliance models are built only by manufacturer information; and (iii) GSP as a variant of aforementioned GSP-based solution proposed for low-rate load disaggregation, with an additional graph of time-of-day information
Advancing the Cyberinfrastructure for Smart Water Metering and Water Demand Modeling
With rapid growth of urban populations and limited water resources, achieving an appropriate balance between water supply capacity and residential water demand poses a significant challenge to water supplying agencies. With the recent emergence of smart metering technology, where water use can be monitored and recorded at high resolution (e.g., observations of water use every 5 seconds), most existing research has been aimed at providing water managers with detailed information about the water use behavior of their consumers and the performance of water using fixtures. However, replacing existing meters with smart meters is expensive, and effectively using data produced by smart meters can be a roadblock for water utilities that lack sophisticated information technology expertise. The research in this dissertation presents low cost, open source cyberinfrastructure aimed at addressing these challenges. Components developed include an open source algorithm for identifying and classifying water end use events from smart meter data, a low cost datalogging and computational device that enables existing water meters to collect high resolution data and compute end use information, and a detailed water demand model that uses end use event information to simulate residential water use at a municipality level. Using this cyberinfrastructure, we conducted a case study application in the cities of Logan and Providence, Utah. We tested the applicability of the disaggregation algorithm in quantifying water end uses for different meter sizes and types. We tested the datalogging computational device at a residential household and demonstrated collection, disaggregation, and transfer of high resolution flow data and classified events into a secure server. Finally, we demonstrated a water demand model that simulates the detailed water end uses of Logan’s residents using a combination of a set of representative water end use events and monthly billing data. Using the data we collected and the outputs from the model, we demonstrated opportunities for conserving water through improving the efficiency of water using fixtures and promoting behavior changes
Contribuitions and developments on nonintrusive load monitoring
Energy efficiency is a key subject in our present world agenda, not only because of greenhouse gas emissions, which contribute to global warming, but also because of possible supply interruptions. In Brazil, energy wastage in the residential market is estimated to be around 15%. Previous studies have indicated that the most savings were achieved with specific appliance, electricity consumption feedback, which caused behavioral changes and encouraged consumers to pursue energy conservation. Nonintrusive Load Monitoring (NILM) is a relatively new term. It aims to disaggregate global consumption at an appliance level, using only a single point of measurement. Various methods have been suggested to infer when appliances are turned on and off, using the analysis of current and voltage aggregated waveforms. Within this context, we aim to provide a methodology for NILM to determine which sets of electrical features and feature extraction rates, obtained from aggregated household data, are essential to preserve equivalent levels of accuracy; thus reducing the amount of data that needs to be transferred to, and stored on, cloud servers. As an addendum to this thesis, a Brazilian appliance dataset, sampled from real appliances, was developed for future NILM developments and research. Beyond that, a low-cost NILM smart meter was developed to encourage consumers to change their habits to more sustainable methods.Eficiência energética é um assunto essencial na agenda mundial. No Brasil, o desperdício de energia no setor residencial é estimado em 15%. Estudos indicaram que maiores ganhos em eficiência são conseguidos quando o usuário recebe as informações de consumo detalhadas por cada aparelho, provocando mudanças comportamentais e incentivando os consumidores na conservação de energia. Monitoramento não intrusivo de cargas (NILM da sigla em inglês) é um termo relativamente novo. A sua finalidade é inferir o consumo de um ambiente até observar os consumos individualizados de cada equipamento utilizando-se de apenas um único ponto de medição. Métodos sofisticados têm sido propostos para inferir quando os aparelhos são ligados e desligados em um ambiente. Dentro deste contexto, este trabalho apresenta uma metodologia para a definição de um conjunto mínimo de características elétricas e sua taxa de extração que reduz a quantidade de dados a serem transmitidos e armazenados em servidores de processamento de dados, preservando níveis equivalentes de acurácia. São utilizadas diferentes técnicas de aprendizado de máquina visando à caracterização e solução do problema. Como adendo ao trabalho, apresenta-se um banco de dados de eletrodomésticos brasileiros, com amostras de equipamentos nacionais para desenvolvimentos futuros em NILM, além de um medidor inteligente de baixo custo para desagregação de cargas, visando tornar o consumo de energia mais sustentável
An Open Source Cyberinfrastructure for Collecting, Processing, Storing and Accessing High Temporal Resolution Residential Water Use Data
Collecting and managing high temporal resolution residential water use data is challenging due to cost and technical requirements associated with the volume and velocity of data collected. We developed an open-source, modular, generalized architecture called Cyberinfrastructure for Intelligent Water Supply (CIWS) to automate the process from data collection to analysis and presentation of high temporal residential water use data. A prototype implementation was built using existing open-source technologies, including smart meters, databases, and services. Two case studies were selected to test functionalities of CIWS, including push and pull data models within single family and multi-unit residential contexts, respectively. CIWS was tested for scalability and performance within our design constraints and proved to be effective within both case studies. All CIWS elements and the case study data described are freely available for re-use
Desagregação de consumos energéticos usando Machine Learning
Nowadays, we are surrounded by electric appliances. Either at home by the washing
machine, kettle, or oven, or work by the computer, cellphone, or printer. Such
devices help us daily, but their popularization increased the energy consumption
to concerning values. In an attempt to reduce energy consumption, governments
started enforcing policies regarding energy education to teach homeowners how to
reduce energy wastage on the demand side. One of those policies was the deployment
of smart meters, which allow the consumer to know how much energy is
being consumed at any given time through a display on the household energy meter.
Even though this measure was well received, the studies show that the best results
in energy conservation are obtained through real-time appliance level feedback. To
get such feedback, one can either measure every outlet in a household, which is
unviable for a broad deployment solution, or disaggregate the energy recorded by
the smart meter. NILM or Non-Intrusive Load Monitoring is the name we give to
the second option where we use the aggregated readings of a household to find the
energy consumed by each appliance. There were many proposals to solve NILM
ranging from HMMs to GSP, where deep learning models showed remarkable results,
obtaining state-of-the-art results. With the intent to create a complete NILM
solution, Withus partnered with the University of Aveiro and proposed this dissertation.
The initial objective was to develop a machine learning model to solve
NILM. Still, during the background analysis, we found the need to create a new
dataset which led to the expansion of the initial proposal to include the dataset
preprocessing and conversion. Regarding NILM, we proposed three new deep learning
models: a convolutional neural network with residual blocks, a recurrent neural
network, and a multilayer perceptron that uses discrete wavelet transforms as features.
These models went through multiple iterations, being evaluated first in the
simpler ON/OFF classification task and later modified and evaluated for the disaggregation
task. We compared our models to the state-of-the-art ones proposed in
NILMTK, where they presented better results than the real-time alternative, dAE,
reducing the NRMSE on average by 49%. We also got close to the best option that
classified with a 30 min delay, Seq2Point, increasing the error on average by 17%.
Besides that, we also analyze the best models from the previous comparison on the
benefit of transfer learning between datasets, where the results show a marginal
performance improvement when using transfer learning. This document presents
the solution outline definition, the multiple options considered for dataset processing
and the best solution, the models’ evolution and results, and the comparison
with the state-of-the-art models regarding generalization to different houses and
under transfer learning.Hoje em dia estamos rodeados de dispositivos elétricos. Quer seja em casa, pela
máquina de lavar, o microondas ou o forno ou no emprego pelo computador, o
telemóvel ou a impressora. Estes dispositivos ajudam-nos diariamente, mas com
a sua popularização o consumo energético atingiu valores preocupantes. Numa
tentativa de reduzir o consumo energético, os governos começaram a introduzir
políticas de educação energética para ensinar os consumidores a reduzir o desperdício
energético. Uma das medidas foi a implementação generalizada de smart
meters, que permitem ao consumidor saber quanta energia está a ser consumida
a qualquer altura através de um ecrã no contador da casa. Mesmo sendo bem
recebida, esta medida não é suficiente uma vez que os estudos indicam que os
melhores resultados são obtidos através de feedback ao nível do dispositivo em
tempo real. Para obtermos este feedback existem duas formas, podemos medir
cada tomada numa dada casa, o que é inviável para uma implementação em larga
escala, ou desagregar a energia registrada pelo smart meter que já está presente
na casa. NILM ou Non-Intrusive Load Monitoring é o nome dado à segunda opção
onde a energia agregada da casa é usada para descobrirmos a energia consumida
por cada dispositivo elétrico. Para resolver este problema foram propostas várias
alternativas, desde HMMs a GSP, onde os modelos de deep learning obtiveram
resultados notáveis sendo agora o estado da arte. Com o objetivo de produzir um
sistema NILM completo, a Withus juntou-se à Universidade de Aveiro e juntos propuseram
esta dissertação. O objetivo inicial era o desenvolvimento de um modelo
de machine learning para desagregar consumos elétricos. Contudo, durante análise
do estado da arte, deparamo-nos com a necessidade de criar um novo dataset, o
que levou à extensão da proposta inicial para incluir também o pré-processamento
e conversão do dataset. Para desagregação de consumos elétricos propusemos três
modelos: uma rede neuronal convolucional com blocos residuais, uma rede neuronal
recorrente e um multilayer perceptron que usa discrete wavelet transforms
como features. Estes modelos passaram por diversas iterações, sendo avaliados
primeiro na tarefa de classificação ON/OFF e depois modificados e avaliados para
desagregação. Os modelos foram ainda comparados com os do estado da arte
presentes no NILMTK, onde apresentaram melhores resultados que a alternativa
real-time, dAE, diminuindo o NRMSE em média 49% ficando próximos da melhor
alternativa que classifica com atraso, Seq2Point, apresentando um erro pior, em
média, de 17%. Para além disso, também analisamos os melhores modelos da
experiência anterior no benefício de usar transfer learning entre datasets, onde os
resultados mostram uma melhoria marginal quando usamos transfer learning. Este
documento apresenta a definição do esboço da solução, as múltiplas opções consideradas
para processamento de dataset e qual a melhor, a evolução dos modelos,
os seus resultados e a comparação com os modelos do estado da arte na capacidade
de generalização entre diferentes casas e de transfer learning entre datasets.Mestrado em Engenharia Informátic
Energy Data Analytics for Smart Meter Data
The principal advantage of smart electricity meters is their ability to transfer digitized electricity consumption data to remote processing systems. The data collected by these devices make the realization of many novel use cases possible, providing benefits to electricity providers and customers alike. This book includes 14 research articles that explore and exploit the information content of smart meter data, and provides insights into the realization of new digital solutions and services that support the transition towards a sustainable energy system. This volume has been edited by Andreas Reinhardt, head of the Energy Informatics research group at Technische Universität Clausthal, Germany, and Lucas Pereira, research fellow at Técnico Lisboa, Portugal
- …