Demand Response Strategy Based on Reinforcement Learning and Fuzzy Reasoning for Home Energy Management

As energy demand continues to increase, demand response (DR) programs in the electricity distribution grid are gaining momentum and their adoption is set to grow gradually over the years ahead. Demand response schemes seek to incentivise consumers to use green energy and reduce their electricity usage during peak periods which helps support grid balancing of supply-demand and generate revenue by selling surplus of energy back to the grid. This paper proposes an effective energy management system for residential demand response using Reinforcement Learning (RL) and Fuzzy Reasoning (FR). RL is considered as a model-free control strategy which learns from the interaction with its environment by performing actions and evaluating the results. The proposed algorithm considers human preference by directly integrating user feedback into its control logic using fuzzy reasoning as reward functions. Q-learning, a RL strategy based on a reward mechanism, is used to make optimal decisions to schedule the operation of smart home appliances by shifting controllable appliances from peak periods, when electricity prices are high, to off-peak hours, when electricity prices are lower without affecting the customer’s preferences. The proposed approach works with a single agent to control 14 household appliances and uses a reduced number of state-action pairs and fuzzy logic for rewards functions to evaluate an action taken for a certain state. The simulation results show that the proposed appliances scheduling approach can smooth the power consumption profile and minimise the electricity cost while considering user’s preferences, user’s feedbacks on each action taken and his/her preference settings. A user-interface is developed in MATLAB/Simulink for the Home Energy Management System (HEMS) to demonstrate the proposed DR scheme. The simulation tool includes features such as smart appliances, electricity pricing signals, smart meters, solar photovoltaic generation, battery energy storage, electric vehicle and grid supply.Peer reviewe

Residential Demand Side Management model, optimization and future perspective: A review

The residential load sector plays a vital role in terms of its impact on overall power balance, stability, and efficient power management. However, the load dynamics of the energy demand of residential users are always nonlinear, uncontrollable, and inelastic concerning power grid regulation and management. The integration of distributed generations (DGs) and advancement of information and communication technology (ICT) even though handles the related issues and challenges up to some extent, till the flexibility, energy management and scheduling with better planning are necessary for the residential sector to achieve better grid stability and efficiency. To address these issues, it is indispensable to analyze the demand-side management (DSM) for the complex residential sector considering various operational constraints, objectives, identifying various factors that affect better planning, scheduling, and management, to project the key features of various approaches and possible future research directions. This review has been done based on the related literature to focus on modeling, optimization methods, major objectives, system operation constraints, dominating factors impacting overall system operation, and possible solutions enhancing residential DSM operation. Gaps in future research and possible prospects have been discussed briefly to give a proper insight into the current implementation of DSM. This extensive review of residential DSM will help all the researchers in this area to innovate better energy management strategies and reduce the effect of system uncertainties, variations, and constraints

Smart home energy management: An analysis of a novel dynamic pricing and demand response aware control algorithm for households with distributed renewable energy generation and storage

Home energy management systems (HEMS) technology can provide a smart and efficient way of optimising energy usage in residential buildings. One of the main goals of the Smart Grid is to achieve Demand Response (DR) by increasing end users’ participation in decision making and increasing the level of awareness that will lead them to manage their energy consumption in an efficient way. This research presents an intelligent HEMS algorithm that manages and controls a range of household appliances with different demand response (DR) limits in an automated way without requiring consumer intervention. In addition, a novel Multiple Users and Load Priority (MULP) scheme is proposed to organise and schedule the list of load priorities in advance for multiple users sharing a house and its appliances. This algorithm focuses on control strategies for controllable loads including air-conditioners, dishwashers, clothes dryers, water heaters, pool pumps and electrical vehicles. Moreover, to investigate the impact on efficiency and reliability of the proposed HEMS algorithm, small-scale renewable energy generation facilities and energy storage systems (ESSs), including batteries and electric vehicles have been incorporated. To achieve this goal, different mathematical optimisation approaches such as linear programming, heuristic methods and genetic algorithms have been applied for optimising the schedule of residential loads using different demand side management and demand response programs as well as optimising the size of a grid connected renewable energy system. Thorough incorporation of a single objective optimisation problem under different system constraints, the proposed algorithm not only reduces the residential energy usage and utility bills, but also determines an optimal scheduling for appliances to minimise any impacts on the level of consumer comfort. To verify the efficiency and robustness of the proposed algorithm a number of simulations were performed under different scenarios. The simulations for load scheduling were carried out over 24 hour periods based on real-time and day ahead electricity prices. The results obtained showed that the proposed MULP scheme resulted in a noticeable decrease in the electricity bill when compared to the other scenarios with no automated scheduling and when a renewable energy system and ESS are not incorporated. Additionally, further simulation results showed that widespread deployment of small scale fixed energy storage and electric vehicle battery storage alongside an intelligent HEMS could enable additional reductions in peak energy usage, and household energy cost. Furthermore, the results also showed that incorporating an optimally designed grid-connected renewable energy system into the proposed HEMS algorithm could significantly reduce household electricity bills, maintain comfort levels, and reduce the environmental footprint. The results of this research are considered to be of great significance as the proposed HEMS approach may help reduce the cost of integrating renewable energy resources into the national grid, which will be reflected in more users adopting these technologies. This in turn will lead to a reduction in the dependence on traditional energy resources that can have negative impacts on the environment. In particular, if a significant proportion of households in a region were to implement the proposed HEMS with the incorporation of small scale storage, then the overall peak demand could be significantly reduced providing great benefits to the grid operator as well as the households

Demand response program for smart grid through real time pricing and home energy management system

Aim of demand response (DR) programs are to change the usage pattern of electricity in such a way that, beneficial to the consumers as well as to the distributors by applying some methods or technology. This way additional cost to erect new energy sources can be postponed in power grid. Best method to implement demand response (DR) program is by influencing consumer through the implementation of real time pricing scheme. To harness the benefit of DR, automated home energy management system is essential. This paper presents a comprehensive demand response system with real time pricing. The real time price is determined after considering price elasticity of various classes of consumers and their load profiles. A real time clustering algorithm suitable for big data of smart grid is devised for the segmentation of consumers. This paper is novel in its design for real time pricing and modelling and automatic scheduling of appliances for home energy management. Simulation results showed that this new real time pricing method is suitable for DR programs to reduce the peak load of the system as well as reducing the energy expenditure of houses, while ensuring profit for the retailer

Intelligent Decision Support System for Energy Management in Demand Response Programs and Residential and Industrial Sectors of the Smart Grid

This PhD thesis addresses the complexity of the energy efficiency control problem in residential and industrial customers of Smart electrical Grid, and examines the main factors that affect energy demand, and proposes an intelligent decision support system for applications of demand response. A multi criteria decision making algorithm is combined with a combinatorial optimization technique to assist energy managers to decide whether to participate in demand response programs or obtain energy from distributed energy resources

Managing Flexible Loads in Residential Areas

Load flexibility in households is a promising option for efficient and reliable operation of future power systems. Due to the distributed nature of residential demand, coordination mechanisms have to cope with a large number of flexible units. This thesis provides a model for demand response analysis and proposes different mechanisms for coordinating flexible loads. In particular, the potential to match intermittent output of renewable generators with electricity demand is investigated

NILM techniques for intelligent home energy management and ambient assisted living: a review

The ongoing deployment of smart meters and different commercial devices has made electricity disaggregation feasible in buildings and households, based on a single measure of the current and, sometimes, of the voltage. Energy disaggregation is intended to separate the total power consumption into specific appliance loads, which can be achieved by applying Non-Intrusive Load Monitoring (NILM) techniques with a minimum invasion of privacy. NILM techniques are becoming more and more widespread in recent years, as a consequence of the interest companies and consumers have in efficient energy consumption and management. This work presents a detailed review of NILM methods, focusing particularly on recent proposals and their applications, particularly in the areas of Home Energy Management Systems (HEMS) and Ambient Assisted Living (AAL), where the ability to determine the on/off status of certain devices can provide key information for making further decisions. As well as complementing previous reviews on the NILM field and providing a discussion of the applications of NILM in HEMS and AAL, this paper provides guidelines for future research in these topics.Agência financiadora: Programa Operacional Portugal 2020 and Programa Operacional Regional do Algarve 01/SAICT/2018/39578 Fundação para a Ciência e Tecnologia through IDMEC, under LAETA: SFRH/BSAB/142998/2018 SFRH/BSAB/142997/2018 UID/EMS/50022/2019 Junta de Comunidades de Castilla-La-Mancha, Spain: SBPLY/17/180501/000392 Spanish Ministry of Economy, Industry and Competitiveness (SOC-PLC project): TEC2015-64835-C3-2-R MINECO/FEDERinfo:eu-repo/semantics/publishedVersio

Heuristic Optimization of Consumer Electricity Costs Using a Generic Cost Model

Many new demand response strategies are emerging for energy management in smart grids. Real-Time Energy Pricing (RTP) is one important aspect of consumer Demand Side Management (DSM), which encourages consumers to participate in load scheduling. This can help reduce peak demand and improve power system efficiency. The use of Intelligent Decision Support Systems (IDSSs) for load scheduling has become necessary in order to enable consumers to respond to the changing economic value of energy across different hours of the day. The type of scheduling problem encountered by a consumer IDSS is typically NP-hard, which warrants the search for good heuristics with efficient computational performance and ease of implementation. This paper presents an extensive evaluation of a heuristic scheduling algorithm for use in a consumer IDSS. A generic cost model for hourly pricing is utilized, which can be configured for traditional on/off peak pricing, RTP, Time of Use Pricing (TOUP), Two-Tier Pricing (2TP) and combinations thereof. The heuristic greedily schedules controllable appliances to minimize smart appliance energy costs and has a polynomial worst-case computation time. Extensive computational experiments demonstrate the effectiveness of the algorithm and the obtained results indicate the gaps between the optimal achievable costs are negligible