A hierarchical scheme for balancing user comfort and electricity consumption of tank water heaters

Demand response (DR) programs offered by utility companies to residential consumers can negatively affect user comfort because of altering a regular operation of domestic devices. Advanced DR solutions need to account for flexibility of device usage in the form of energy savings that the customers can agree with. Specialized control schemes should inform utility companies about the level of such possible savings. This paper presents a hierarchical scheme that allows to control a domestic hot water storage system according to user comfort requests. The scheme uses a multi-objective optimization approach to interconnect comfort and electricity consumption of the tank water heater. The simulations show that the proposed control can not only significantly (by 95.6%) improve the user thermal comfort during hot water activities, but can also reduce energy consumption by 16.4% as compared to the weekly regular operation of the heater

Deep Reinforcement Learning Based Approach for Multi-Agent Control of Residential Electric Water Heaters for Distribution Load Management

The push towards decarbonization and electrification of the society is leading to increased electricity demand. Many countries, including Canada, are utilizing non-greenhouse gas (GHG) emitting sources and renewable energy sources (RES) to meet this increasing demand. Many of the RES, however, are intermittent and uncertain, and are non-load following sources of electricity. Technologies supporting demand flexibility are being increasingly used to respond to intermittent changes in RES supply and meet the power grid requirements by modifying the energy consumption patterns of residential loads. The work presented in this thesis discusses the application of electric water heaters (EWHs) as flexible and controllable loads. EWHs, accounting for a significant portion (44%) of water heaters in the Canadian residential sector, and being the second largest consumer of electricity in the household sector (20%), are becoming a viable source for providing load flexibility. This thesis presents a multi-agent reinforcement learning (MARL) approach to address the energy management problem of EWHs. Two agents, the residential aggregator agent (RAA)- for EWH control and the utility agent (UA)- to represent the role of a utility, are designed to interact with each other and the (reinforcement learning) environment to maximize their respective rewards. A novel control algorithm using a binning process is employed by the RAA to control operations of certain groups of EWHs. The multi-agent deep deterministic policy gradient (MADDPG) algorithm is implemented for this problem and used in training the RAA and UA to follow the optimal policy. The proposed EWH energy management approach is tested for consumers in Ontario, New Brunswick and Quebec which have varying consumer tariff rates. The results demonstrate the ability of the proposed RAA and UA to control the behaviour of EWHs via price incentive signals, thus providing benefits for the consumers and the utility