Demand Response Strategy Based on Reinforcement Learning and Fuzzy Reasoning for Home Energy Management

As energy demand continues to increase, demand response (DR) programs in the electricity distribution grid are gaining momentum and their adoption is set to grow gradually over the years ahead. Demand response schemes seek to incentivise consumers to use green energy and reduce their electricity usage during peak periods which helps support grid balancing of supply-demand and generate revenue by selling surplus of energy back to the grid. This paper proposes an effective energy management system for residential demand response using Reinforcement Learning (RL) and Fuzzy Reasoning (FR). RL is considered as a model-free control strategy which learns from the interaction with its environment by performing actions and evaluating the results. The proposed algorithm considers human preference by directly integrating user feedback into its control logic using fuzzy reasoning as reward functions. Q-learning, a RL strategy based on a reward mechanism, is used to make optimal decisions to schedule the operation of smart home appliances by shifting controllable appliances from peak periods, when electricity prices are high, to off-peak hours, when electricity prices are lower without affecting the customer’s preferences. The proposed approach works with a single agent to control 14 household appliances and uses a reduced number of state-action pairs and fuzzy logic for rewards functions to evaluate an action taken for a certain state. The simulation results show that the proposed appliances scheduling approach can smooth the power consumption profile and minimise the electricity cost while considering user’s preferences, user’s feedbacks on each action taken and his/her preference settings. A user-interface is developed in MATLAB/Simulink for the Home Energy Management System (HEMS) to demonstrate the proposed DR scheme. The simulation tool includes features such as smart appliances, electricity pricing signals, smart meters, solar photovoltaic generation, battery energy storage, electric vehicle and grid supply.Peer reviewe

Customer Engagement Plans for Peak Load Reduction in Residential Smart Grids

In this paper, we propose and study the effectiveness of customer engagement plans that clearly specify the amount of intervention in customer's load settings by the grid operator for peak load reduction. We suggest two different types of plans, including Constant Deviation Plans (CDPs) and Proportional Deviation Plans (PDPs). We define an adjustable reference temperature for both CDPs and PDPs to limit the output temperature of each thermostat load and to control the number of devices eligible to participate in Demand Response Program (DRP). We model thermostat loads as power throttling devices and design algorithms to evaluate the impact of power throttling states and plan parameters on peak load reduction. Based on the simulation results, we recommend PDPs to the customers of a residential community with variable thermostat set point preferences, while CDPs are suitable for customers with similar thermostat set point preferences. If thermostat loads have multiple power throttling states, customer engagement plans with less temperature deviations from thermostat set points are recommended. Contrary to classical ON/OFF control, higher temperature deviations are required to achieve similar amount of peak load reduction. Several other interesting tradeoffs and useful guidelines for designing mutually beneficial incentives for both the grid operator and customers can also be identified

NILM techniques for intelligent home energy management and ambient assisted living: a review

The ongoing deployment of smart meters and different commercial devices has made electricity disaggregation feasible in buildings and households, based on a single measure of the current and, sometimes, of the voltage. Energy disaggregation is intended to separate the total power consumption into specific appliance loads, which can be achieved by applying Non-Intrusive Load Monitoring (NILM) techniques with a minimum invasion of privacy. NILM techniques are becoming more and more widespread in recent years, as a consequence of the interest companies and consumers have in efficient energy consumption and management. This work presents a detailed review of NILM methods, focusing particularly on recent proposals and their applications, particularly in the areas of Home Energy Management Systems (HEMS) and Ambient Assisted Living (AAL), where the ability to determine the on/off status of certain devices can provide key information for making further decisions. As well as complementing previous reviews on the NILM field and providing a discussion of the applications of NILM in HEMS and AAL, this paper provides guidelines for future research in these topics.Agência financiadora: Programa Operacional Portugal 2020 and Programa Operacional Regional do Algarve 01/SAICT/2018/39578 Fundação para a Ciência e Tecnologia through IDMEC, under LAETA: SFRH/BSAB/142998/2018 SFRH/BSAB/142997/2018 UID/EMS/50022/2019 Junta de Comunidades de Castilla-La-Mancha, Spain: SBPLY/17/180501/000392 Spanish Ministry of Economy, Industry and Competitiveness (SOC-PLC project): TEC2015-64835-C3-2-R MINECO/FEDERinfo:eu-repo/semantics/publishedVersio

Buildings-to-Grid Integration Framework

This paper puts forth a mathematical framework for Buildings-to-Grid (BtG) integration in smart cities. The framework explicitly couples power grid and building's control actions and operational decisions, and can be utilized by buildings and power grids operators to simultaneously optimize their performance. Simplified dynamics of building clusters and building-integrated power networks with algebraic equations are presented---both operating at different time-scales. A model predictive control (MPC)-based algorithm that formulates the BtG integration and accounts for the time-scale discrepancy is developed. The formulation captures dynamic and algebraic power flow constraints of power networks and is shown to be numerically advantageous. The paper analytically establishes that the BtG integration yields a reduced total system cost in comparison with decoupled designs where grid and building operators determine their controls separately. The developed framework is tested on standard power networks that include thousands of buildings modeled using industrial data. Case studies demonstrate building energy savings and significant frequency regulation, while these findings carry over in network simulations with nonlinear power flows and mismatch in building model parameters. Finally, simulations indicate that the performance does not significantly worsen when there is uncertainty in the forecasted weather and base load conditions.Comment: In Press, IEEE Transactions on Smart Gri

Demand and Storage Management in a Prosumer Nanogrid Based on Energy Forecasting

Energy efficiency and consumers' role in the energy system are among the strategic research topics in power systems these days. Smart grids (SG) and, specifically, microgrids, are key tools for these purposes. This paper presents a three-stage strategy for energy management in a prosumer nanogrid. Firstly, energy monitoring is performed and time-space compression is applied as a tool for forecasting energy resources and power quality (PQ) indices; secondly, demand is managed, taking advantage of smart appliances (SA) to reduce the electricity bill; finally, energy storage systems (ESS) are also managed to better match the forecasted generation of each prosumer. Results show how these strategies can be coordinated to contribute to energy management in the prosumer nanogrid. A simulation test is included, which proves how effectively the prosumers' power converters track the power setpoints obtained from the proposed strategy.Spanish Agencia Estatal de Investigacion ; Fondo Europeo de Desarrollo Regional