Optimal Decentralized Energy Management of Electrical and Thermal Distributed Energy Resources and Loads in Microgrids Using Reinforcement Learning

In this paper, a decentralized energy management system is presented for intelligent microgrids with the presence of distributed resources using reinforcement learning. Due to the unpredictable nature of renewable energy resources, the variability of load consumption, and the nonlinear model of batteries, the design of a microgrid energy management system is associated with many challenges. In addition, centralized control structures in large-scale systems increase computational volume and complexity in control algorithms. In this paper, a fully decentralized multi-agent structure for a microgrid energy management system is proposed and the Markov decision process is used to model the stochastic behavior of agents in the microgrid. Electrical and thermal distributed resources, batteries, and consumers are considered intelligent and independent agents. They have the learning ability to explore and exploit the environment in a fully decentralized manner and achieve their optimal policies. The proposed method for hourly microgrid management is model-independent and based on learning. The method maximizes the profits of all manufacturers, minimizes consumer costs, and reduces the dependence of the microgrid on the maingrid. Finally, using real data from renewable energy sources and consumers, the accuracy of the proposed method in the Iranian electricity market is simulated and verified

Comments on ‘Improved synthesis conditions for mixed H2/H∞ gain-scheduling control subject to uncertain scheduling parameters'

In this note, some comments are provided on the article ‘Improved synthesis conditions for mixed H2/H∞ gain-scheduling control subject to uncertain scheduling parameters’. The problems of H2 and H∞ gain-scheduled control exploiting inexact scheduling parameters are tackled in that article. Unfortunately, due to occurrence of some severe errors the obtained synthesis conditions are wholly fallacious

Decentralized Reinforcement Learning Approach for Microgrid Energy Management in Stochastic Environment

Microgrids are considered to be smart power grids that can integrate Distributed Energy Resources (DERs) in the main grid cleanly and reliably. Due to the random and unpredictable nature of Renewable Energy Sources (RESs) and electricity demand, designing a control system for microgrid energy management is a complex task. In addition, the policies of microgrid agents are changing over time to improve their expected profits. Therefore, the problem is stochastic and the policies of the agents are not stationary and deterministic. This paper proposes a fully decentralized multiagent Energy Management System (EMS) for microgrids using the reinforcement learning and stochastic game. The microgrid agents, comprising customers, and DERs are considered as intelligent and autonomous decision makers. The proposed method solves a distributed optimization problem for each self-interested decision maker. Interactions between the decision makers and the environment during the learning phase lead the system to converge to the optimal equilibrium point in which the benefits of all the agents are maximized. Simulation studies using a real dataset demonstrate the effectiveness of the proposed method for the hourly energy management of microgrids

Decentralized energy management system for smart microgrids using reinforcement learning

Abstract This paper presents a novel fully decentralized and intelligent energy management system (EMS) for a smart microgrid based on reinforcement learning (RL) strategy. The purpose of the proposed EMS is to maximize the benefit of all microgrid entities comprising customers and distributed energy resources (DERs). Due to unpredictable features of renewable energy sources and variability of consumers’ demands, designing the microgrid EMS is a complicated task. To overcome this issue, the multi‐agent hour‐ahead energy management problem is modelled as a finite Markov decision process. The microgrid entities are considered as intelligent agents. The optimal policy of agents is obtained through a newly developed framework of the model‐free Q‐learning algorithm to maximize the benefit of all renewable and non‐renewable energy resources and battery energy storage system. The degradation model of the battery is considered to reduce the number of battery replacements. To ensure customers’ comfort, customers’ expenses are decreased without demand curtailment via introducing two types of load shifting techniques. The microgrid operation is analysed under four scenarios comprising no‐learning, generator‐learning, customer‐learning, and whole‐learning. the performance of the proposed algorithm is compared to the Monte Carlo method and simulation results on the real power‐grid dataset show the superiority of the algorithm

An LMI Approach to Nonlinear State-Feedback Stability of Uncertain Time-Delay Systems in the Presence of Lipschitzian Nonlinearities

This article proposes a new nonlinear state-feedback stability controller utilizing linear matrix inequality (LMI) for time-delay nonlinear systems in the presence of Lipschitz nonlinearities and subject to parametric uncertainties. Following the Lyapunov–Krasovskii stabilization scheme, the asymptotic stability criterion resulted in the LMI form and the nonlinear state-feedback control technique was determined. Due to their significant contributions to the system stability, time delays and system uncertainties were taken into account while the suggested scheme was designed so that the system’s stabilization was satisfied in spite of time delays and system uncertainties. The benefit of the proposed method is that not only is the control scheme independent of the system order, but it is also fairly simple. Hence, there is no complexity in using the proposed technique. Finally, to justify the proficiency and performance of the suggested technique, a numerical system and a rotational inverted pendulum were studied. Numerical simulations and experimental achievements prove the efficiency of the suggested control technique

An LMI approach to nonlinear state-feedback stability of uncertain time-delay systems in the presence of lipschitzian nonlinearities

This article proposes a new nonlinear state-feedback stability controller utilizing linear matrix inequality (LMI) for time-delay nonlinear systems in the presence of Lipschitz nonlinearities and subject to parametric uncertainties. Following the Lyapunov–Krasovskii stabilization scheme, the asymptotic stability criterion resulted in the LMI form and the nonlinear state-feedback control technique was determined. Due to their significant contributions to the system stability, time delays and system uncertainties were taken into account while the suggested scheme was designed so that the system’s stabilization was satisfied in spite of time delays and system uncertainties. The benefit of the proposed method is that not only is the control scheme independent of the system order, but it is also fairly simple. Hence, there is no complexity in using the proposed technique. Finally, to justify the proficiency and performance of the suggested technique, a numerical system and a rotational inverted pendulum were studied. Numerical simulations and experimental achievements prove the efficiency of the suggested control technique.</p

Regulation with Guaranteed Convergence Rate for Continuous-Time Systems with Completely Unknown Dynamics in the Presence of Disturbance

This paper presents the design of a novel H ∞ -based control framework for state regulation of continuous-time linear systems with completely unknown dynamics. The proposed method solves the regulation problem with the desired convergence rate and simultaneously seeks to attenuate the adverse effect of disturbance on the system. The H ∞ regulation problem assumes a cost function that considers regulation with a guaranteed rate of convergence as well as disturbance attenuation. The problem is then turned into a two-player zero-sum game optimization problem that can be solved by solving the associated algebraic Riccati equation (ARE), which provides a model-based solution. To solve this problem in a model-free way, a novel integral reinforcement learning (IRL) algorithm is designed to learn the solution online without requiring any prior knowledge of the system dynamics. It is shown that the model-free method (i.e., IRL-based method) provides the same solution as the model-based method (i.e., ARE). The effectiveness of the proposed method is ascertained through simulation examples; it is shown that the proposed method effectively addresses the problem for both stable and unstable systems

An LMI approach to nonlinear state-feedback stability of uncertain time-delay systems in the presence of lipschitzian nonlinearities

This article proposes a new nonlinear state-feedback stability controller utilizing linear matrix inequality (LMI) for time-delay nonlinear systems in the presence of Lipschitz nonlinearities and subject to parametric uncertainties. Following the Lyapunov–Krasovskii stabilization scheme, the asymptotic stability criterion resulted in the LMI form and the nonlinear state-feedback control technique was determined. Due to their significant contributions to the system stability, time delays and system uncertainties were taken into account while the suggested scheme was designed so that the system’s stabilization was satisfied in spite of time delays and system uncertainties. The benefit of the proposed method is that not only is the control scheme independent of the system order, but it is also fairly simple. Hence, there is no complexity in using the proposed technique. Finally, to justify the proficiency and performance of the suggested technique, a numerical system and a rotational inverted pendulum were studied. Numerical simulations and experimental achievements prove the efficiency of the suggested control technique.Mechatronic Systems Desig