Load Frequency Control: A Deep Multi-Agent Reinforcement Learning Approach

The paradigm shift in energy generation towards microgrid-based architectures is changing the landscape of the energy control structure heavily in distribution systems. More specifically, distributed generation is deployed in the network demanding decentralised control mechanisms to ensure reliable power system operations. In this work, a Multi-Agent Reinforcement Learning approach is proposed to deliver an agentbased solution to implement load frequency control without the need of a centralised authority. Multi-Agent Deep Deterministic Policy Gradient is used to approximate the frequency control at the primary and the secondary levels. Each generation unit is represented as an agent that is modelled by a Recurrent Neural Network. Agents learn the optimal way of acting and interacting with the environment to maximise their long term performance and to balance generation and load, thus restoring frequency. In this paper we prove using three test systems, with two, four and eight generators, that our Multi-Agent Reinforcement Learning approach can efficiently be used to perform frequency control in a decentralised way

Modeling and Controlling a Hybrid Multi-Agent based Microgrid in Presence of Different Physical and Cyber Components

This dissertation starts with modeling of two different and important parts of the distribution power systems, i.e. distribution line and photovoltaic (PV) systems. Firstly, it studies different approximation methods and develops a new approach for simplification of Carson\u27s equations to model distribution lines for unbalanced power flow and short circuit analysis. The results of applying the proposed method on a three-phase unbalanced distribution system are compared with different existing methods as well as actual impedance values obtained from numerical integration method. Then steady state modeling and optimal placing of multiple PV system are investigated in order to reduce the total loss in the system. The results show the effectiveness of the proposed method in minimizing the total loss in a distribution power system.;The dissertation starts the discussion about microgrid modeling and control by implementing a novel frequency control approach in a microgrid. This study has been carried out step by step by modeling different part of the power system and proposing different algorithms. Firstly, the application of Renewable Energy Sources (RES) accompanied with Energy Storage Systems (ESS) in a hybrid system is studied in the presence of Distributed Generation (DG) resources in Load Frequency Control (LFC) problem of microgrid power system with significant penetration of wind speed disturbances. The next step is to investigate the effect of PHEVs in modelling and controlling the microgid. Therefore, system with different penetrations of PHEVs and different stochastic behaviors of PHEVs is modeled. Different kinds of control approaches, including PI control as conventional method and proposed optimal LQR and dynamic programming methods, have been utilized and the results have been compared with each other. Then, Multi Agent System (MAS) is utilized as a control solution which contributes the cyber aspects of microgrid system. The modeled microgrid along with dynamic models of different components is implemented in a centralized multi-agent based structure. The robustness of the proposed controller has been tested against different frequency changes including cyber attack implications with different timing and severity. New attack detection through learning method is also proposed and tested. The results show improvement in frequency response of the microgrid system using the proposed control method and defense strategy against cyber attacks.;Finally, a new multi-agent based control method along with an advanced secondary voltage and frequency control using Particle Swarm Optimization (PSO) and Adaptive Dynamic Programming (ADP) is proposed and tested in the modeled microgrid considering nonlinear heterogeneous dynamic models of DGs. The results are shown and compared with conventional control approaches and different multi-agent structures. It is observed that the results are improved by using the new multi-agent structure and secondary control method.;In summary, contributions of this dissertation center in three main topics. Firstly, new accurate methods for modeling the distribution line impedance and PV system is developed. Then advanced control and defense strategy method for frequency regulation against cyber intrusions and load changes in a microgrid is proposed. Finally, a new hierarchical multi-agent based control algorithm is designed for secondary voltage and frequency control of the microgrid. (Abstract shortened by ProQuest.)

FMI Compliant Approach to Investigate the Impact of Communication to Islanded Microgrid Secondary Control

In multi-master islanded microgrids, the inverter controllers need to share the signals and to coordinate, in either centralized or distributed way, in order to operate properly and to assure a good functionality of the grid. The central controller is used in centralized strategy. In distributed control, Multi-agent system (MAS) is considered to be a suitable solution for coordination of such system. However the latency and disturbance of the network may disturb the communication from central controller to local controllers or among agents or and negatively influence the grid operation. As a consequence, communication aspects need to be properly addressed during the control design and assessment. In this paper, we propose a holistic approach with co-simulation using Functional Mockup Interface (FMI) standard to validate the microgrid control system taking into account the communication network. A use-case of islanded microgrid frequency secondary control with MAS under consensus algorithm is implemented to demonstrate the impact of communication and to illustrate the proposed holistic approach.Comment: Proceedings of the IEEE PES ISGT Asia 2017 conferenc

A review of microgrid development in the United States – A decade of progress on policies, demonstrations, controls, and software tools

Microgrids have become increasingly popular in the United States. Supported by favorable federal and local policies, microgrid projects can provide greater energy stability and resilience within a project site or community. This paper reviews major federal, state, and utility-level policies driving microgrid development in the United States. Representative U.S. demonstration projects are selected and their technical characteristics and non-technical features are introduced. The paper discusses trends in the technology development of microgrid systems as well as microgrid control methods and interactions within the electricity market. Software tools for microgrid design, planning, and performance analysis are illustrated with each tool's core capability. Finally, the paper summarizes the successes and lessons learned during the recent expansion of the U.S. microgrid industry that may serve as a reference for other countries developing their own microgrid industries

Secondary Frequency and Voltage Control of Islanded Microgrids via Distributed Averaging

In this work we present new distributed controllers for secondary frequency and voltage control in islanded microgrids. Inspired by techniques from cooperative control, the proposed controllers use localized information and nearest-neighbor communication to collectively perform secondary control actions. The frequency controller rapidly regulates the microgrid frequency to its nominal value while maintaining active power sharing among the distributed generators. Tuning of the voltage controller provides a simple and intuitive trade-off between the conflicting goals of voltage regulation and reactive power sharing. Our designs require no knowledge of the microgrid topology, impedances or loads. The distributed architecture allows for flexibility and redundancy, and eliminates the need for a central microgrid controller. We provide a voltage stability analysis and present extensive experimental results validating our designs, verifying robust performance under communication failure and during plug-and-play operation.Comment: Accepted for publication in IEEE Transactions on Industrial Electronic