Non-intrusive load monitoring solutions for low- and very low-rate granularity

Strathclyde theses - ask staff. Thesis no. : T15573Large-scale smart energy metering deployment worldwide and the integration of smart meters within the smart grid are enabling two-way communication between the consumer and energy network, thus ensuring an improved response to demand. Energy disaggregation or non-intrusive load monitoring (NILM), namely disaggregation of the total metered electricity consumption down to individual appliances using purely algorithmic tools, is gaining popularity as an added-value that makes the most of meter data.In this thesis, the first contribution tackles low-rate NILM problem by proposing an approach based on graph signal processing (GSP) that does not require any training.Note that Low-rate NILM refers to NILM of active power measurements only, at rates from 1 second to 1 minute. Adaptive thresholding, signal clustering and pattern matching are implemented via GSP concepts and applied to the NILM problem. Then for further demonstration of GSP potential, GSP concepts are applied at both, physical signal level via graph-based filtering and data level, via effective semi-supervised GSP-based feature matching. The proposed GSP-based NILM-improving methods are generic and can be used to improve the results of various event-based NILM approaches. NILM solutions for very low data rates (15-60 min) cannot leverage on low to highrates NILM approaches. Therefore, the third contribution of this thesis comprises three very low-rate load disaggregation solutions, based on supervised (i) K-nearest neighbours relying on features such as statistical measures of the energy signal, time usage profile of appliances and reactive power consumption (if available); unsupervised(ii) optimisation performing minimisation of error between aggregate and the sum of estimated individual loads, where energy consumed by always-on load is heuristically estimated prior to further disaggregation and appliance models are built only by manufacturer information; and (iii) GSP as a variant of aforementioned GSP-based solution proposed for low-rate load disaggregation, with an additional graph of time-of-day information.Large-scale smart energy metering deployment worldwide and the integration of smart meters within the smart grid are enabling two-way communication between the consumer and energy network, thus ensuring an improved response to demand. Energy disaggregation or non-intrusive load monitoring (NILM), namely disaggregation of the total metered electricity consumption down to individual appliances using purely algorithmic tools, is gaining popularity as an added-value that makes the most of meter data.In this thesis, the first contribution tackles low-rate NILM problem by proposing an approach based on graph signal processing (GSP) that does not require any training.Note that Low-rate NILM refers to NILM of active power measurements only, at rates from 1 second to 1 minute. Adaptive thresholding, signal clustering and pattern matching are implemented via GSP concepts and applied to the NILM problem. Then for further demonstration of GSP potential, GSP concepts are applied at both, physical signal level via graph-based filtering and data level, via effective semi-supervised GSP-based feature matching. The proposed GSP-based NILM-improving methods are generic and can be used to improve the results of various event-based NILM approaches. NILM solutions for very low data rates (15-60 min) cannot leverage on low to highrates NILM approaches. Therefore, the third contribution of this thesis comprises three very low-rate load disaggregation solutions, based on supervised (i) K-nearest neighbours relying on features such as statistical measures of the energy signal, time usage profile of appliances and reactive power consumption (if available); unsupervised(ii) optimisation performing minimisation of error between aggregate and the sum of estimated individual loads, where energy consumed by always-on load is heuristically estimated prior to further disaggregation and appliance models are built only by manufacturer information; and (iii) GSP as a variant of aforementioned GSP-based solution proposed for low-rate load disaggregation, with an additional graph of time-of-day information

Blind non-intrusive appliance load monitoring using graph-based signal processing

With ongoing massive smart energy metering deployments, disaggregation of household's total energy consumption down to individual appliances using purely software tools, aka. non-intrusive appliance load monitoring (NALM), has generated increased interest. However, despite the fact that NALM was proposed over 30 years ago, there are still many open challenges. Indeed, the majority of approaches require training and are sensitive to appliance changes requiring regular re-training. In this paper, we tackle this challenge by proposing a "blind" NALM approach that does not require any training. The main idea is to build upon an emerging field of graph-based signal processing to perform adaptive thresholding, signal clustering and feature matching. Using two datasets of active power measurements with 1min and 8sec resolution, we demonstrate the effectiveness of the proposed method using a state-of-the-art NALM approaches as benchmarks

On a training-less solution for non-intrusive appliance load monitoring using graph signal processing

With ongoing large-scale smart energy metering deployments worldwide, disaggregation of a household’s total energy consumption down to individual appliances using analytical tools, aka. non-intrusive appliance load monitoring (NALM), has generated increased research interest lately. NALM can deepen energy feedback, support appliance retrofit advice and support home automation. However, despite the fact that NALM was proposed over 30 years ago, there are still many open challenges with respect to its practicality and effectiveness at low sampling rates. Indeed, the majority of NALM approaches, supervised or unsupervised, require training to build appliance models, and are sensitive to appliance changes in the house, thus requiring regular re-training. In this paper, we tackle this challenge by proposing a NALM approach that does not require any training. The main idea is to build upon the emerging field of graph signal processing to perform adaptive thresholding, signal clustering and pattern matching. We determine the performance limits of our approach and demonstrate its usefulness in practice. Using two open access datasets - the US REDD dataset with active power measurements downsampled to 1min resolution and the UK REFIT dataset with 8sec resolution, we demonstrate the effectiveness of the proposed method for typical smart meter sampling rate, with state-of-the-art supervised and unsupervised NALM approaches as benchmarks

Blind non-intrusive appliance load monitoring using graph-based signal processing

With ongoing massive smart energy metering deployments, disaggregation of household's total energy consumption down to individual appliances using purely software tools, aka. non-intrusive appliance load monitoring (NALM), has generated increased interest. However, despite the fact that NALM was proposed over 30 years ago, there are still many open challenges. Indeed, the majority of approaches require training and are sensitive to appliance changes requiring regular re-training. In this paper, we tackle this challenge by proposing a 'blind' NALM approach that does not require any training. The main idea is to build upon an emerging field of graph-based signal processing to perform adaptive threshold-ing, signal clustering and feature matching. Using two datasets of active power measurements with 1min and 8sec resolution, we demonstrate the effectiveness of the proposed method using a state-of-the-art NALM approaches as benchmarks

Electricity usage profile disaggregation of hourly smart meter data

This paper is motivated by the growing demand of disaggregating electricity consumption measured by smart meters, down to appliance level. The very low 15-min to 60- min granularity of energy measurements available for analysis, as is standard by the majority of nationwide smart metering programmes, is posing serious challenges. The non-intrusive load monitoring (NILM) solutions for these very low data rates cannot leverage on low (1-60sec) to high rates (in the order of kHz to MHz) NILM approaches, and so far have not received much attention in the literature. In this paper, we propose a novel electricity profile hourly disaggregation of energy consumed (kWh) based on K-nearest neighbours (K-NN), that relies on features such as statistical measures of the energy signal, time usage profile of appliances and reactive power consumption (if available). We propose relative standard deviation as a metric to assess the quality of each feature per appliance. For validation, three publicly accessible real-world datasets are used, namely the REDD, REFIT and AMPds (Version 2), for up to 3 months

Non-intrusive Load Monitoring based on Self-supervised Learning

Deep learning models for non-intrusive load monitoring (NILM) tend to require a large amount of labeled data for training. However, it is difficult to generalize the trained models to unseen sites due to different load characteristics and operating patterns of appliances between data sets. For addressing such problems, self-supervised learning (SSL) is proposed in this paper, where labeled appliance-level data from the target data set or house is not required. Initially, only the aggregate power readings from target data set are required to pre-train a general network via a self-supervised pretext task to map aggregate power sequences to derived representatives. Then, supervised downstream tasks are carried out for each appliance category to fine-tune the pre-trained network, where the features learned in the pretext task are transferred. Utilizing labeled source data sets enables the downstream tasks to learn how each load is disaggregated, by mapping the aggregate to labels. Finally, the fine-tuned network is applied to load disaggregation for the target sites. For validation, multiple experimental cases are designed based on three publicly accessible REDD, UK-DALE, and REFIT data sets. Besides, state-of-the-art neural networks are employed to perform NILM task in the experiments. Based on the NILM results in various cases, SSL generally outperforms zero-shot learning in improving load disaggregation performance without any sub-metering data from the target data sets.Comment: 12 pages,10 figure

Improving event-based non-intrusive load monitoring using graph signal processing

Large-scale smart energy metering deployment worldwide and integration of smart meters within the smart grid will enable two-way communication between the consumer and energy network, thus ensuring improved response to demand. Energy disaggregation or Non-intrusive load monitoring (NILM), namely disaggregation of the total metered electricity consumption down to individual appliances using purely algorithmic tools, is gaining popularity as an added-value that makes the most of meter data. However, NILM remains a challenging problem since NILM is susceptible to sensor noise, unknown load noise, transient spikes and fluctuations. In this paper, we tackle this problem using novel graph signal processing (GSP) concepts, applied at both, physical signal level via graph-based filtering and data level, via effective semi-supervised GSP-based feature matching. The proposed GSP-based method is generic and can be used to improve results of various event-based NILM approaches. We demonstrate significant improvement in performance using three state-of-the-art NILM methods, both supervised and unsupervised, and real-world active power consumption readings from the REDD and REFIT datasets, sampled at 1 and 8 seconds, respectively

Development and validation of a prognostic nomogram for rectal cancer patients who underwent surgical resection

Objective: The purpose of this study was to develop and validate a nomogram model for the prediction of survival outcome in rectal cancer patients who underwent surgical resection.Methods: A total of 9,919 consecutive patients were retrospectively identified using the Surveillance, Epidemiology, and End Results (SEER) database. Significant prognostic factors were determined by the univariate and multivariate Cox analysis. The nomogram model for the prediction of cancer-specific survival (CSS) in rectal cancer patients were developed based on these prognostic variables, and its predictive power was assessed by the concordance index (C-index). Calibration curves were plotted to evaluate the associations between predicted probabilities and actual observations. The internal and external cohort were used to further validate the predictive performance of the prognostic nomogram.Results: All patients from the SEER database were randomly split into a training cohort (n = 6,944) and an internal validation cohort (n = 2,975). The baseline characteristics of two cohorts was comparable. Independent prognostic factors were identified as age, pT stage, lymph node metastasis, serum CEA level, tumor size, differentiation type, perineural invasion, circumferential resection margin involvement and inadequate lymph node yield. In the training cohort, the C-index of the nomogram was 0.719 (95% CI: 0.696–0.742), which was significantly higher than that of the TNM staging system (C-index: 0.606, 95% CI: 0.583–0.629). The nomogram had a C-index of 0.726 (95% CI: 0.691–0.761) for the internal validation cohort, indicating a good predictive power. In addition, an independent cohort composed of 202 rectal cancer patients from our institution were enrolled as the external validation. Compared with the TNM staging system (C-index: 0.573, 95% CI: 0.492–0.654), the prognostic nomogram still showed a better predictive performance, with the C-index of 0.704 (95% CI: 0.626–0.782). Calibration plots showed a good consistency between predicted probability and the actual observation in the training and two validation cohorts.Conclusion: The nomogram showed an excellent predictive ability for survival outcome of rectal cancer patients, and it might provide an accurate prognostic stratification and help clinicians determine individualized treatment strategies

A generic optimisation-based approach for improving non-intrusive load monitoring

The large-scale deployment of smart metering worldwide has ignited renewed interest in electrical load disaggregation, or non-intrusive load monitoring (NILM). Most NILM algorithms disaggregate one appliance at a time, remove the estimated appliance contribution from the total load, and then move on to disaggregate the next appliance. On one hand, this is efficient since multi-class classification is avoided and analytical models for each appliance can be developed independently of other appliances with the benefit of being transferred to unseen houses that have different sets of appliances. On the other hand, however, these methods can significantly under- or over- estimate the total consumption since they do not minimise the difference between the measured aggregate load and the sum of estimated individual loads. Motivated by minimising the latter difference without losing the benefits of existing NILM algorithms, we propose novel post-processing approaches for improving the accuracy of existing NILM. This is posed as an optimisation problem to refine the final NILM result using regularisation, based on the level of confidence in the original NILM output. First, we propose a heuristic method to solve this (combinatorial) boolean quadratic problem through relaxing zero-one constraint sets to compact zero-one intervals. Convex-based solutions, including norm-1, norm-2 and semi-definite programming-based relaxation, are proposed trading off accuracy and complexity. We demonstrate good performance of the proposed post-processing methods, applicable to any event-based NILM, compared with 4 state-of-the-art benchmarks, using public REFIT and REDD electrical load datasets