Demand Response Strategy Based on Reinforcement Learning and Fuzzy Reasoning for Home Energy Management

As energy demand continues to increase, demand response (DR) programs in the electricity distribution grid are gaining momentum and their adoption is set to grow gradually over the years ahead. Demand response schemes seek to incentivise consumers to use green energy and reduce their electricity usage during peak periods which helps support grid balancing of supply-demand and generate revenue by selling surplus of energy back to the grid. This paper proposes an effective energy management system for residential demand response using Reinforcement Learning (RL) and Fuzzy Reasoning (FR). RL is considered as a model-free control strategy which learns from the interaction with its environment by performing actions and evaluating the results. The proposed algorithm considers human preference by directly integrating user feedback into its control logic using fuzzy reasoning as reward functions. Q-learning, a RL strategy based on a reward mechanism, is used to make optimal decisions to schedule the operation of smart home appliances by shifting controllable appliances from peak periods, when electricity prices are high, to off-peak hours, when electricity prices are lower without affecting the customer’s preferences. The proposed approach works with a single agent to control 14 household appliances and uses a reduced number of state-action pairs and fuzzy logic for rewards functions to evaluate an action taken for a certain state. The simulation results show that the proposed appliances scheduling approach can smooth the power consumption profile and minimise the electricity cost while considering user’s preferences, user’s feedbacks on each action taken and his/her preference settings. A user-interface is developed in MATLAB/Simulink for the Home Energy Management System (HEMS) to demonstrate the proposed DR scheme. The simulation tool includes features such as smart appliances, electricity pricing signals, smart meters, solar photovoltaic generation, battery energy storage, electric vehicle and grid supply.Peer reviewe

Emission-aware Energy Storage Scheduling for a Greener Grid

Reducing our reliance on carbon-intensive energy sources is vital for reducing the carbon footprint of the electric grid. Although the grid is seeing increasing deployments of clean, renewable sources of energy, a significant portion of the grid demand is still met using traditional carbon-intensive energy sources. In this paper, we study the problem of using energy storage deployed in the grid to reduce the grid's carbon emissions. While energy storage has previously been used for grid optimizations such as peak shaving and smoothing intermittent sources, our insight is to use distributed storage to enable utilities to reduce their reliance on their less efficient and most carbon-intensive power plants and thereby reduce their overall emission footprint. We formulate the problem of emission-aware scheduling of distributed energy storage as an optimization problem, and use a robust optimization approach that is well-suited for handling the uncertainty in load predictions, especially in the presence of intermittent renewables such as solar and wind. We evaluate our approach using a state of the art neural network load forecasting technique and real load traces from a distribution grid with 1,341 homes. Our results show a reduction of >0.5 million kg in annual carbon emissions -- equivalent to a drop of 23.3% in our electric grid emissions.Comment: 11 pages, 7 figure, This paper will appear in the Proceedings of the ACM International Conference on Future Energy Systems (e-Energy 20) June 2020, Australi

Achieving an optimal trade-off between revenue and energy peak within a smart grid environment

We consider an energy provider whose goal is to simultaneously set revenue-maximizing prices and meet a peak load constraint. In our bilevel setting, the provider acts as a leader (upper level) that takes into account a smart grid (lower level) that minimizes the sum of users' disutilities. The latter bases its decisions on the hourly prices set by the leader, as well as the schedule preferences set by the users for each task. Considering both the monopolistic and competitive situations, we illustrate numerically the validity of the approach, which achieves an 'optimal' trade-off between three objectives: revenue, user cost, and peak demand

Distributed Stochastic Market Clearing with High-Penetration Wind Power

Integrating renewable energy into the modern power grid requires risk-cognizant dispatch of resources to account for the stochastic availability of renewables. Toward this goal, day-ahead stochastic market clearing with high-penetration wind energy is pursued in this paper based on the DC optimal power flow (OPF). The objective is to minimize the social cost which consists of conventional generation costs, end-user disutility, as well as a risk measure of the system re-dispatching cost. Capitalizing on the conditional value-at-risk (CVaR), the novel model is able to mitigate the potentially high risk of the recourse actions to compensate wind forecast errors. The resulting convex optimization task is tackled via a distribution-free sample average based approximation to bypass the prohibitively complex high-dimensional integration. Furthermore, to cope with possibly large-scale dispatchable loads, a fast distributed solver is developed with guaranteed convergence using the alternating direction method of multipliers (ADMM). Numerical results tested on a modified benchmark system are reported to corroborate the merits of the novel framework and proposed approaches.Comment: To appear in IEEE Transactions on Power Systems; 12 pages and 9 figure

Decentralized Demand Side Management with Rooftop PV in Residential Distribution Network

In the past extensive researches have been conducted on demand side management (DSM) program which aims at reducing peak loads and saving electricity cost. In this paper, we propose a framework to study decentralized household demand side management in a residential distribution network which consists of multiple smart homes with schedulable electrical appliances and some rooftop photovoltaic generation units. Each smart home makes individual appliance scheduling to optimize the electric energy cost according to the day-ahead forecast of electricity prices and its willingness for convenience sacrifice. Using the developed simulation model, we examine the performance of decentralized household DSM and study their impacts on the distribution network operation and renewable integration, in terms of utilization efficiency of rooftop PV generation, overall voltage deviation, real power loss, and possible reverse power flows.Comment: 5 pages, 7 figures, ISGT 2018 conferenc

Optimal Charging of Electric Vehicles in Smart Grid: Characterization and Valley-Filling Algorithms

Electric vehicles (EVs) offer an attractive long-term solution to reduce the dependence on fossil fuel and greenhouse gas emission. However, a fleet of EVs with different EV battery charging rate constraints, that is distributed across a smart power grid network requires a coordinated charging schedule to minimize the power generation and EV charging costs. In this paper, we study a joint optimal power flow (OPF) and EV charging problem that augments the OPF problem with charging EVs over time. While the OPF problem is generally nonconvex and nonsmooth, it is shown recently that the OPF problem can be solved optimally for most practical power networks using its convex dual problem. Building on this zero duality gap result, we study a nested optimization approach to decompose the joint OPF and EV charging problem. We characterize the optimal offline EV charging schedule to be a valley-filling profile, which allows us to develop an optimal offline algorithm with computational complexity that is significantly lower than centralized interior point solvers. Furthermore, we propose a decentralized online algorithm that dynamically tracks the valley-filling profile. Our algorithms are evaluated on the IEEE 14 bus system, and the simulations show that the online algorithm performs almost near optimality ($<1$ relative difference from the offline optimal solution) under different settings.Comment: This paper is temporarily withdrawn in preparation for journal submissio

Demand and Storage Management in a Prosumer Nanogrid Based on Energy Forecasting

Energy efficiency and consumers' role in the energy system are among the strategic research topics in power systems these days. Smart grids (SG) and, specifically, microgrids, are key tools for these purposes. This paper presents a three-stage strategy for energy management in a prosumer nanogrid. Firstly, energy monitoring is performed and time-space compression is applied as a tool for forecasting energy resources and power quality (PQ) indices; secondly, demand is managed, taking advantage of smart appliances (SA) to reduce the electricity bill; finally, energy storage systems (ESS) are also managed to better match the forecasted generation of each prosumer. Results show how these strategies can be coordinated to contribute to energy management in the prosumer nanogrid. A simulation test is included, which proves how effectively the prosumers' power converters track the power setpoints obtained from the proposed strategy.Spanish Agencia Estatal de Investigacion ; Fondo Europeo de Desarrollo Regional