Enhancing household-level load forecasts using daily load profile clustering

Forecasting the electricity demand for individual households is important for both consumers and utilities due to the increasing decentralized nature of the electricity system. Particularly, utilities often have very little information about their consumers except for aggregate building level loads, without knowledge of interior details about the household appliance sets or occupants. In this paper, we explore the possibility of enhancing the day-ahead load forecasts for hundreds of individual households by clustering their daily load profile history to obtain each consumer's specific typical consumption patterns. The clustering method is based on load profile shape using the Earth Mover's Distance metric to calculate similarity between load profiles. The forecasting methods then predict the next day shape from the empirical probability of previous cluster transitions in the consumer's load history and estimate the magnitude either by using historical load relationships with temperature and forecast temperatures or previous day consumption levels. The generated forecasts are compared to a benchmark Multiple Linear Regression (MLR) day-ahead forecast and persistence forecasts for all individuals. While at the aggregate level the MLR method represents a significant improvement over persistence forecasts, on an individual level we find that the best forecasting model is specific to the individual. In particular, we find that the MLR model produces lower errors when consumers have a consistent daily temperature response and the cluster model with previous day magnitude produces lower errors for consumers whose consumption changes abruptly in magnitude for several days at a time. Our work adds to the state of knowledge surrounding individual household load forecasting and demonstrates the potential for cluster-based methodologies to enhance short term load forecasts

Review of Low Voltage Load Forecasting: Methods, Applications, and Recommendations

The increased digitalisation and monitoring of the energy system opens up numerous opportunities to decarbonise the energy system. Applications on low voltage, local networks, such as community energy markets and smart storage will facilitate decarbonisation, but they will require advanced control and management. Reliable forecasting will be a necessary component of many of these systems to anticipate key features and uncertainties. Despite this urgent need, there has not yet been an extensive investigation into the current state-of-the-art of low voltage level forecasts, other than at the smart meter level. This paper aims to provide a comprehensive overview of the landscape, current approaches, core applications, challenges and recommendations. Another aim of this paper is to facilitate the continued improvement and advancement in this area. To this end, the paper also surveys some of the most relevant and promising trends. It establishes an open, community-driven list of the known low voltage level open datasets to encourage further research and development.Comment: 37 pages, 6 figures, 2 tables, review pape

Residential Demand Response using Electricity Smart Meter Data

The electricity industry is currently undergoing changes in a transitioning period characterised by Energy 3D: Digitalisation, Decentralisation, and Decarbonisation. Smart meters are the vital infrastructure necessary to digitalise the energy system as well as enable advancements in decentralisation and decarbonisation. As of today, more than 500 million smart meters have been installed worldwide, with that number expected to rise to several billion installations over the decade. Smart meters enable electricity load to be measured with half-hourly granularity, providing an opportunity for demand-side management innovations that are likely to be advantageous for both utility companies and customers. Among these innovations, time-of- use (TOU) tariffs are widely considered to be the most promising solution for optimising energy consumption in the residential sector, however actual use is still limited. The objective of this thesis is to investigate opportunities and problems related to TOU tariffs utilising smart meter data at the national level. The authors have identified four major research gaps which need to be filled in order to expand commercial applications of TOU tariffs. These gaps are the described and addressed in the following chapters: the "TOU load adaptation forecasting problem", the "TOU winner detection problem", the "TOU public dataset problem", and the "excess generation forecasting problem". This thesis demonstrates three modelling approaches and one new TOU dataset (CAMSL). A significant contribution to the field is through the discover of new summary statistical features (statistical moments) and assesses the capacity of these to encapsulate other more widely used explanatory variables of demand response. The thesis is concluded by discussing future works and policy implications, such as the necessity of the more tailored modelling works and public live-stream of smart meter data, which could accelerate the roll-out of the demand side management at the residential sector.EPC

Non-Gaussian residual based short term load forecast adjustment for distribution feeders

The evolving role for electricity network operators means that load forecasting at the distribution level has become increasingly important, presenting the need for anticipation of the behavior of highly dynamic and diversely distributed loads. The commonly held assumption of Gaussian residuals in forecasting does not always hold for distribution network loads, increasing the uncertainty in balancing a system at this network level. To reduce the operational impact of forecast errors, this paper utilizes different multivariate joint probability distributions to capture the intra-day dependency structure of forecast residuals. Transforming these to the conditional form enables forecast corrections to be made at variable horizons even in the absence of the forecast model. Improvements in accuracy are demonstrated on benchmark load forecast models at distribution level low voltage substations. A practical distribution system application on scheduling embedded energy storage shows substantial reductions in grid imports and hence costs to distribution level customers from utilizing the proposed intraday correction approach

A Novel Closed-Loop Clustering Method for Hierarchical Load Forecasting

Hierarchical load forecasting (HLF) is an approach to generate forecasts for hierarchical loadtime series. The performance of HLF can be improved by optimizing the forecasting model and the hierarchical structure. Previous studies mostly focus on the forecasting model while the hierarchical structure is usually formed by clustering of natural attributes like geography, customer type, or the similarities between load profiles. A major limitation of these natural hierarchical structures is the mismatched objectives between clustering and forecasting. Clustering aims to minimize the dissimilarity among customers of a group while forecasting aims to minimize their forecasting errors.The two independent optimizations could limit the overall forecasting performance. Hence, this paper attempts to integrate the hierarchical structure and the forecasting model by a novel closed-loopclustering (CLC) algorithm. It links the objectives of forecasting and clustering by a feedback mechanism to return the goodness-of-fit as the criterion for the clustering. In this way, the hierarchical structure is enhanced by re-assigning the cluster membership and the parameters of the forecasting models are updated iteratively. The method is comparatively assessed with existing HLF methods. Using the same forecasting model, the proposed hierarchical structure outperforms the bottom-up structure by 52.20%, ensemble-based structure by 26.89%, load-profile structures by 19.90%, respectively. <br/

An Informed Long-term Forecasting Method for Electrical Distribution Network Operators

Northern Powergrid (NPG) is an electrical distribution network operator in the UK servicing Yorkshire and the Northeast of England. Currently they produce long-term eight year forecasts for each substation on the network with an emphasis on an annual maximum demand (MD) figure. The current method used by NPG is thought to oversimplify the problem and does not give enough insight into changes in substation demand. In order to inform their current forecast, the novel CL-ANFIS method uses a combination of machine learning techniques for both forecasting and general insight to the drivers of demand. Also introduced here are novel techniques for determination of MD at NPG and methods for handling load transfer periods. In order to address a problem of this size, a twofold approach is taken. One is to address the drivers of demand such as weather, economic or demographic data sets through the use of statistics and machine learning techniques. The other is to address the long-term forecasting problem with a transparent technique that can aid in explaining the drivers of demand on any given substation. Techniques used include cluster analysis on demographic data sets in addition to ANFIS as a forecasting method. The results of the novel CL-ANFIS method are compared against the current NPG forecast and show how more insight into substation demand profiles can drive the decision-making process. This is done through a combination of using a tailored customer database for NPG and leveraging the information provided by the membership functions of ANFIS

Design and Evaluation of a Machine Learning Based Model for Optimization of Residential Battery Energy Storage System Scheduling for Cost and Emissions Reductions

Usage of battery energy storage system (BESS) to facilitate demand response (DR) programs such as time of use (TOU) pricing can reduce utility bills for residential customers. However, using such a system for minimizing electric bills under these rate structures has the potential to cause an increase in emissions from the grid system. The increased emissions were majorly due to bulk energy storage of electricity produced by off-peak generators with higher emission rates and excess energy consumption due to battery inefficiency. BESS operating to optimize competing objectives to minimize utility cost and minimize CO2 emissions requires complex models that require an accurate forecast of future energy demand. These models get less effective as errors in demand forecasts increase. Demand forecasts for residential consumers are challenging due to high demand variability. Moreover, these models require computationally expensive mixed-integer linear programming (MILP) models in the day-to-day operation of BESS. In this work, a machine learning model (ML) was developed that attempts to predict an optimal battery schedule for an upcoming day based on easy to obtain information such as day of the week, month, previous day’s demand, average temperature, and relative humidity. The ML model’s utility bill and CO2 emission results were then compared to a no BESS scenario as well as a multi-objective optimization model based on perfect (OPT model) and forecasted (FORECAST model) demand data. The models were tested on two customers each from California and Arizona. The paired t-test comparison showed that the ML model results were not statistically different from the FORECAST model. The ML model was able to capture 65% of potential cost savings that could be generated from the OPT model. The model was also efficient in balancing the reduction of utility costs as well as CO2 emissions. Moreover, it requires less time and effort as is required for building and maintaining the FORECAST model

A Distributed and Real-time Machine Learning Framework for Smart Meter Big Data

The advanced metering infrastructure allows smart meters to collect high-resolution consumption data, thereby enabling consumers and utilities to understand their energy usage at different levels, which has led to numerous smart grid applications. Smart meter data, however, poses different challenges to developing machine learning frameworks than classic theoretical frameworks due to their big data features and privacy limitations. Therefore, in this work, we aim to address the challenges of building machine learning frameworks for smart meter big data. Specifically, our work includes three parts: 1) We first analyze and compare different learning algorithms for multi-level smart meter big data. A daily activity pattern recognition model has been developed based on non-intrusive load monitoring for appliance-level smart meter data. Then, a consensus-based load profiling and forecasting system has been proposed for individual building level and higher aggregated level smart meter data analysis; 2) Following discussion of multi-level smart meter data analysis from an offline perspective, a universal online functional analysis model has been proposed for multi-level real-time smart meter big data analysis. The proposed model consists of a multi-scale load dynamic profiling unit based on functional clustering and a multi-scale online load forecasting unit based on functional deep neural networks. The two units enable online tracking of the dynamic cluster trajectories and online forecasting of daily multi-scale demand; 3) To enable smart meter data analysis in the distributed environment, FederatedNILM was proposed, which is then combined with differential privacy to provide privacy guarantees for the appliance-level distributed machine learning framework. Based on federated deep learning enhanced with two schemes, namely the utility optimization scheme and the privacy-preserving scheme, the proposed distributed and privacy-preserving machine learning framework enables electric utilities and service providers to offer smart meter services on a large scale