Demand Response Strategy Based on Reinforcement Learning and Fuzzy Reasoning for Home Energy Management

As energy demand continues to increase, demand response (DR) programs in the electricity distribution grid are gaining momentum and their adoption is set to grow gradually over the years ahead. Demand response schemes seek to incentivise consumers to use green energy and reduce their electricity usage during peak periods which helps support grid balancing of supply-demand and generate revenue by selling surplus of energy back to the grid. This paper proposes an effective energy management system for residential demand response using Reinforcement Learning (RL) and Fuzzy Reasoning (FR). RL is considered as a model-free control strategy which learns from the interaction with its environment by performing actions and evaluating the results. The proposed algorithm considers human preference by directly integrating user feedback into its control logic using fuzzy reasoning as reward functions. Q-learning, a RL strategy based on a reward mechanism, is used to make optimal decisions to schedule the operation of smart home appliances by shifting controllable appliances from peak periods, when electricity prices are high, to off-peak hours, when electricity prices are lower without affecting the customer’s preferences. The proposed approach works with a single agent to control 14 household appliances and uses a reduced number of state-action pairs and fuzzy logic for rewards functions to evaluate an action taken for a certain state. The simulation results show that the proposed appliances scheduling approach can smooth the power consumption profile and minimise the electricity cost while considering user’s preferences, user’s feedbacks on each action taken and his/her preference settings. A user-interface is developed in MATLAB/Simulink for the Home Energy Management System (HEMS) to demonstrate the proposed DR scheme. The simulation tool includes features such as smart appliances, electricity pricing signals, smart meters, solar photovoltaic generation, battery energy storage, electric vehicle and grid supply.Peer reviewe

An Agent Based Simulation of Smart Metering Technology Adoption

Based on the classic behavioural theory “the Theory of Planned Behaviour”, we develop an agent-based model to simulate the diffusion of smart metering technology in the electricity market. We simulate the emergent adoption of smart metering technology under different management strategies and economic regulations. Our research results show that in terms of boosting the take-off of smart meters in the electricity market, choosing the initial users on a random and geographically dispersed basis and encouraging meter competition between energy suppliers can be two very effective strategies. We also observe an “S-curve” diffusion of smart metering technology and a “lock-in” effect in the model. The research results provide us with insights as to effective policies and strategies for the roll-out of smart metering technology in the electricity market

Towards transactive energy systems: An analysis on current trends

This paper presents a comprehensive analysis on the latest advances in transactive energy systems. The main contribution of this work is centered on the definition of transactive energy concepts and how such systems can be implemented in the smart grid paradigm. The analyzed works have been categorized into three lines of research: (i) transactive network management; (ii) transactive control; and (iii) peer-to-peer markets. It has been found that most of the current approaches for transactive energy are available as a model, lacking the real implementation to have a complete validation. For that purpose, both scientific and practical aspects of transactive energy should be studied in parallel, implementing adequate simulation platforms and tools to scrutiny the results.This work has received funding from the European Union's Horizon 2020 research and innovation programme under project DOMINOES (grant agreement No. 771066) and from FEDER Funds through COMPETE program and from National Funds through FCT under the project UID/EEA/00760/2019.info:eu-repo/semantics/publishedVersio

Architectures for smart end-user services in the power grid

Abstract-The increase of distributed renewable electricity generators, such as solar cells and wind turbines, requires a new energy management system. These distributed generators introduce bidirectional energy flows in the low-voltage power grid, requiring novel coordination mechanisms to balance local supply and demand. Closed solutions exist for energy management on the level of individual homes. However, no service architectures have been defined that allow the growing number of end-users to interact with the other power consumers and generators and to get involved in more rational energy consumption patterns using intuitive applications. We therefore present a common service architecture that allows houses with renewable energy generation and smart energy devices to plug into a distributed energy management system, integrated with the public power grid. Next to the technical details, we focus on the usability aspects of the end-user applications in order to contribute to high service adoption and optimal user involvement. The presented architecture facilitates end-users to reduce net energy consumption, enables power grid providers to better balance supply and demand, and allows new actors to join with new services. We present a novel simulator that allows to evaluate both the power grid and data communication aspects, and illustrate a 22% reduction of the peak load by deploying a central coordinator inside the home gateway of an end-user

Artificial Intelligence and Machine Learning Approaches to Energy Demand-Side Response: A Systematic Review

Recent years have seen an increasing interest in Demand Response (DR) as a means to provide flexibility, and hence improve the reliability of energy systems in a cost-effective way. Yet, the high complexity of the tasks associated with DR, combined with their use of large-scale data and the frequent need for near real-time de-cisions, means that Artificial Intelligence (AI) and Machine Learning (ML) — a branch of AI — have recently emerged as key technologies for enabling demand-side response. AI methods can be used to tackle various challenges, ranging from selecting the optimal set of consumers to respond, learning their attributes and pref-erences, dynamic pricing, scheduling and control of devices, learning how to incentivise participants in the DR schemes and how to reward them in a fair and economically efficient way. This work provides an overview of AI methods utilised for DR applications, based on a systematic review of over 160 papers, 40 companies and commercial initiatives, and 21 large-scale projects. The papers are classified with regards to both the AI/ML algorithm(s) used and the application area in energy DR. Next, commercial initiatives are presented (including both start-ups and established companies) and large-scale innovation projects, where AI methods have been used for energy DR. The paper concludes with a discussion of advantages and potential limitations of reviewed AI techniques for different DR tasks, and outlines directions for future research in this fast-growing area

Smart Grid Technologies in Europe: An Overview

The old electricity network infrastructure has proven to be inadequate, with respect to modern challenges such as alternative energy sources, electricity demand and energy saving policies. Moreover, Information and Communication Technologies (ICT) seem to have reached an adequate level of reliability and flexibility in order to support a new concept of electricity network—the smart grid. In this work, we will analyse the state-of-the-art of smart grids, in their technical, management, security, and optimization aspects. We will also provide a brief overview of the regulatory aspects involved in the development of a smart grid, mainly from the viewpoint of the European Unio

A review of microgrid development in the United States – A decade of progress on policies, demonstrations, controls, and software tools

Microgrids have become increasingly popular in the United States. Supported by favorable federal and local policies, microgrid projects can provide greater energy stability and resilience within a project site or community. This paper reviews major federal, state, and utility-level policies driving microgrid development in the United States. Representative U.S. demonstration projects are selected and their technical characteristics and non-technical features are introduced. The paper discusses trends in the technology development of microgrid systems as well as microgrid control methods and interactions within the electricity market. Software tools for microgrid design, planning, and performance analysis are illustrated with each tool's core capability. Finally, the paper summarizes the successes and lessons learned during the recent expansion of the U.S. microgrid industry that may serve as a reference for other countries developing their own microgrid industries

Reinforcement learning in local energy markets

Local energy markets (LEMs) are well suited to address the challenges of the European energy transition movement. They incite investments in renewable energy sources (RES), can improve the integration of RES into the energy system, and empower local communities. However, as electricity is a low involvement good, residential households have neither the expertise nor do they want to put in the time and effort to trade themselves on their own on short-term LEMs. Thus, machine learning algorithms are proposed to take over the bidding for households under realistic market information. We simulate a LEM on a 15 min merit-order market mechanism and deploy reinforcement learning as strategic learning for the agents. In a multi-agent simulation of 100 households including PV, micro-cogeneration, and demand shifting appliances, we show how participants in a LEM can achieve a self-sufficiency of up to 30% with trading and 41,4% with trading and demand response (DR) through an installation of only 5kWp PV panels in 45% of the households under affordable energy prices. A sensitivity analysis shows how the results differ according to the share of renewable generation and degree of demand flexibility