Data Mining to Uncover Heterogeneous Water Use Behaviors From Smart Meter Data

Knowledge on the determinants and patterns of water demand for different consumers supports the design of customized demand management strategies. Smart meters coupled with big data analytics tools create a unique opportunity to support such strategies. Yet, at present, the information content of smart meter data is not fully mined and usually needs to be complemented with water fixture inventory and survey data to achieve detailed customer segmentation based on end use water usage. In this paper, we developed a data‐driven approach that extracts information on heterogeneous water end use routines, main end use components, and temporal characteristics, only via data mining existing smart meter readings at the scale of individual households. We tested our approach on data from 327 households in Australia, each monitored with smart meters logging water use readings every 5 s. As part of the approach, we first disaggregated the household‐level water use time series into different end uses via Autoflow. We then adapted a customer segmentation based on eigenbehavior analysis to discriminate among heterogeneous water end use routines and identify clusters of consumers presenting similar routines. Results revealed three main water end use profile clusters, each characterized by a primary end use: shower, clothes washing, and irrigation. Time‐of‐use and intensity‐of‐use differences exist within each class, as well as different characteristics of regularity and periodicity over time. Our customer segmentation analysis approach provides utilities with a concise snapshot of recurrent water use routines from smart meter data and can be used to support customized demand management strategies.TU Berlin, Open-Access-Mittel - 201

Chameleon: a Blind Double Trapdoor Hash Function for Securing AMI Data Aggregation

Data aggregation is an integral part of Advanced Metering Infrastructure (AMI) deployment that is implemented by the concentrator. Data aggregation reduces the number of transmissions, thereby reducing communication costs and increasing the bandwidth utilization of AMI. However, the concentrator poses a great risk of being tampered with, leading to erroneous bills and possible consumer disputes. In this paper, we propose an end-to-end integrity protocol using elliptic curve based chameleon hashing to provide data integrity and authenticity. The concentrator generates and sends a chameleon hash value of the aggregated readings to the Meter Data Management System (MDMS) for verification, while the smart meter with the trapdoor key computes and sends a commitment value to the MDMS so that the resulting chameleon hash value calculated by the MDMS is equivalent to the previous hash value sent by the concentrator. By comparing the two hash values, the MDMS can validate the integrity and authenticity of the data sent by the concentrator. Compared with the discrete logarithm implementation, the ECC implementation reduces the computational cost of MDMS, concentrator and smart meter by approximately 36.8%, 80%, and 99% respectively. We also demonstrate the security soundness of our protocol through informal security analysis

Measures to Reduce Concerns Related to Smart Meter Data_x000D_ Are detailed Consumer data needed for Smart Grid Operations?

Intelligent information systems are necessary to improve power grid operations. The current trend reveals the deployment of smart meter at consumer sites as the energy provider’s starting point to grid modernisation. Smart meters are considered as a core element in most of the smart grid projects. The smart metering system is designed to collect and transmit detailed consumption data. This data could profile the consumer’s lifestyle and this is among the reasons for consumer’s resistance to smart meters. It is also not clear how the power grid will benefit from individual consumer’s detailed power usage data. This article attempts to investigate this issue by analysing if detailed end-user data is required for the operation of other control systems in the grid. Further an analysis is conducted to identify which stakeholders will benefit from the detailed smart meter data. Based on the analysis remedial measures are proposed

Cluster Analysis and Model Comparison Using Smart Meter Data.

Load forecasting plays a crucial role in the world of smart grids. It governs many aspects of the smart grid and smart meter, such as demand response, asset management, investment, and future direction. This paper proposes time-series forecasting for short-term load prediction to unveil the load forecast benefits through different statistical and mathematical models, such as artificial neural networks, auto-regression, and ARIMA. It targets the problem of excessive computational load when dealing with time-series data. It also presents a business case that is used to analyze different clusters to find underlying factors of load consumption and predict the behavior of customers based on different parameters. On evaluating the accuracy of the prediction models, it is observed that ARIMA models with the (P, D, Q) values as (1, 1, 1) were most accurate compared to other values

A Distributed and Real-time Machine Learning Framework for Smart Meter Big Data

The advanced metering infrastructure allows smart meters to collect high-resolution consumption data, thereby enabling consumers and utilities to understand their energy usage at different levels, which has led to numerous smart grid applications. Smart meter data, however, poses different challenges to developing machine learning frameworks than classic theoretical frameworks due to their big data features and privacy limitations. Therefore, in this work, we aim to address the challenges of building machine learning frameworks for smart meter big data. Specifically, our work includes three parts: 1) We first analyze and compare different learning algorithms for multi-level smart meter big data. A daily activity pattern recognition model has been developed based on non-intrusive load monitoring for appliance-level smart meter data. Then, a consensus-based load profiling and forecasting system has been proposed for individual building level and higher aggregated level smart meter data analysis; 2) Following discussion of multi-level smart meter data analysis from an offline perspective, a universal online functional analysis model has been proposed for multi-level real-time smart meter big data analysis. The proposed model consists of a multi-scale load dynamic profiling unit based on functional clustering and a multi-scale online load forecasting unit based on functional deep neural networks. The two units enable online tracking of the dynamic cluster trajectories and online forecasting of daily multi-scale demand; 3) To enable smart meter data analysis in the distributed environment, FederatedNILM was proposed, which is then combined with differential privacy to provide privacy guarantees for the appliance-level distributed machine learning framework. Based on federated deep learning enhanced with two schemes, namely the utility optimization scheme and the privacy-preserving scheme, the proposed distributed and privacy-preserving machine learning framework enables electric utilities and service providers to offer smart meter services on a large scale

Cluster Analysis of Smart Metering Data - An Implementation in Practice

The introduction of smart meter technology is a great challenge for the German energy industry. It requires not only large investments in the communication and metering infrastructure, but also a redesign of traditional business processes. The newly incurring costs cannot be fully passed on to the end customers. One option to counterbalance these expenses is to exploit the newly generated smart metering data for the creation of new services and improved processes. For instance, performing a cluster analysis of smart metering data focused on the customers’ time-based consumption behavior allows for a detailed customer segmentation. In the article we present a cluster analysis performed on real-world consumption data from a smart meter project conducted by a German regional utilities company. We show how to integrate a cluster analysis approach into a business intelligence environment and evaluate this artifact as defined by design science. We discuss the results of the cluster analysis and highlight options to apply them to segment-specific tariff design

Towards secure end-to-end data aggregation in AMI through delayed-integrity-verification

The integrity and authenticity of the energy usage data in Advanced Metering Infrastructure (AMI) is crucial to ensure the correct energy load to facilitate generation, distribution and customer billing. Any malicious tampering to the data must be detected immediately. This paper introduces secure end-to-end data aggregation for AMI, a security protocol that allows the concentrators to securely aggregate the data collected from the smart meters, while enabling the utility back-end that receives the aggregated data to verify the integrity and data originality. Compromise of concentrators can be detected. The aggregated data is protected using Chameleon Signatures and then forwarded to the utility back-end for verification, accounting, and analysis. Using the Trapdoor Chameleon Hash Function, the smart meters can periodically send an evidence to the utility back-end, by computing an alternative message and a random value (m', r) such that m' consists of all previous energy usage measurements of the smart meter in a specified period of time. By verifying that the Chameleon Hash Value of (m', r) and that the energy usage matches those aggregated by the concentrators, the utility back-end is convinced of the integrity and authenticity of the data from the smart meters. Any data anomaly between smart meters and concentrators can be detected, thus indicating potential compromise of concentrators