610 research outputs found

    Deep Reinforcement Learning for DER Cyber-Attack Mitigation

    Full text link
    The increasing penetration of DER with smart-inverter functionality is set to transform the electrical distribution network from a passive system, with fixed injection/consumption, to an active network with hundreds of distributed controllers dynamically modulating their operating setpoints as a function of system conditions. This transition is being achieved through standardization of functionality through grid codes and/or international standards. DER, however, are unique in that they are typically neither owned nor operated by distribution utilities and, therefore, represent a new emerging attack vector for cyber-physical attacks. Within this work we consider deep reinforcement learning as a tool to learn the optimal parameters for the control logic of a set of uncompromised DER units to actively mitigate the effects of a cyber-attack on a subset of network DER

    Coordinated Optimal Voltage Control in Distribution Networks with Data-Driven Methods

    Get PDF
    Voltage control is facing significant challenges with the increasing integration of photovoltaic (PV) systems and electric vehicles (EVs) in active distribution networks. This is leading to major transformations of control schemes that require more sophisticated coordination between different voltage regulation devices in different timescales. Except for conventional Volt/Var control (VVC) devices such on-load tap change (OLTC) and capacitor banks (CBs), inverter-based PVs are encouraged to participate in voltage regulation considering their flexible reactive power regulation capability. With the vehicle to grid (V2G) technology and inverter-based interface at charging stations, the charging power of an EV can be also controlled to support voltages. These emerging technologies facilitate the development of two-stage coordinated optimal voltage control schemes. However, these new control schemes pursue a fast response speed with local control strategies in shorter snapshots, which fails to track the optimal solutions for the distribution system operation. The voltage control methods mainly aim to mitigate voltage violations and reduce network power loss, but they seldom focus on satisfying the various requirements of PV and EV customers. This may discourage customer-owned resources from participating in ancillary services such as voltage regulation. Moreover, model-based voltage control methods highly rely on the accurate knowledge of power system models and parameters, which is sometimes difficult to obtain in real-life distribution networks. The goal of this thesis is to propose a data-driven two-stage voltage control framework to fill the research gaps mentioned above, showing what frameworks, models and solution methods can be used in the optimal voltage control of modern active distribution systems to tackle the security and economic challenges posed by high integration of PVs and EVs

    Deep Reinforcement Learning for Control of Microgrids: A Review

    Get PDF
    A microgrid is widely accepted as a prominent solution to enhance resilience and performance in distributed power systems. Microgrids are flexible for adding distributed energy resources in the ecosystem of the electrical networks. Control techniques are used to synchronize distributed energy resources (DERs) due to their turbulent nature. DERs including alternating current, direct current and hybrid load with storage systems have been used in microgrids quite frequently due to which controlling the flow of energy in microgrids have been complex task with traditional control approaches. Distributed as well central approach to apply control algorithms is well-known methods to regulate frequency and voltage in microgrids. Recently techniques based of artificial intelligence are being applied for the problems that arise in operation and control of latest generation microgrids and smart grids. Such techniques are categorized in machine learning and deep learning in broader terms. The objective of this research is to survey the latest strategies of control in microgrids using the deep reinforcement learning approach (DRL). Other techniques of artificial intelligence had already been reviewed extensively but the use of DRL has increased in the past couple of years. To bridge the gap for the researchers, this survey paper is being presented with a focus on only Microgrids control DRL techniques for voltage control and frequency regulation with distributed, cooperative and multi agent approaches are presented in this research

    Fusion of Model-free Reinforcement Learning with Microgrid Control: Review and Vision

    Get PDF
    Challenges and opportunities coexist in microgrids as a result of emerging large-scale distributed energy resources (DERs) and advanced control techniques. In this paper, a comprehensive review of microgrid control is presented with its fusion of model-free reinforcement learning (MFRL). A high-level research map of microgrid control is developed from six distinct perspectives, followed by bottom-level modularized control blocks illustrating the configurations of grid-following (GFL) and grid-forming (GFM) inverters. Then, mainstream MFRL algorithms are introduced with an explanation of how MFRL can be integrated into the existing control framework. Next, the application guideline of MFRL is summarized with a discussion of three fusing approaches, i.e., model identification and parameter tuning, supplementary signal generation, and controller substitution, with the existing control framework. Finally, the fundamental challenges associated with adopting MFRL in microgrid control and corresponding insights for addressing these concerns are fully discussed.Comment: 14 pages, 4 figures, published on IEEE Transaction on Smart Grid 2022 Nov 15. See: https://ieeexplore-ieee-org.utk.idm.oclc.org/stamp/stamp.jsp?arnumber=995140

    Attention enabled multi-agent DRL for decentralized volt-VAR control of active distribution system using PV inverters and SVCs

    Get PDF

    Deep Reinforcement Learning for Distribution Network Operation and Electricity Market

    Full text link
    The conventional distribution network and electricity market operation have become challenging under complicated network operating conditions, due to emerging distributed electricity generations, coupled energy networks, and new market behaviours. These challenges include increasing dynamics and stochastics, and vast problem dimensions such as control points, measurements, and multiple objectives, etc. Previously the optimization models were often formulated as conventional programming problems and then solved mathematically, which could now become highly time-consuming or sometimes infeasible. On the other hand, with the recent advancement of artificial intelligence technologies, deep reinforcement learning (DRL) algorithms have demonstrated their excellent performances in various control and optimization fields. This indicates a potential alternative to address these challenges. In this thesis, DRL-based solutions for distribution network operation and electricity market have been investigated and proposed. Firstly, a DRL-based methodology is proposed for Volt/Var Control (VVC) optimization in a large distribution network, to effectively control bus voltages and reduce network power losses. Further, this thesis proposes a multi-agent (MA)DRL-based methodology under a complex regional coordinated VVC framework, and it can address spatial and temporal uncertainties. The DRL algorithm is also improved to adapt to the applications. Then, an integrated energy and heating systems (IEHS) optimization problem is solved by a MADRL-based methodology, where conventionally this could only be solved by simplifications or iterations. Beyond the applications in distribution network operation, a new electricity market service pricing method based on a DRL algorithm is also proposed. This DRL-based method has demonstrated good performance in this virtual storage rental service pricing problem, whereas this bi-level problem could hardly be solved directly due to a non-convex and non-continuous lower-level problem. These proposed methods have demonstrated advantageous performances under comprehensive case studies, and numerical simulation results have validated the effectiveness and high efficiency under different sophisticated operation conditions, solution robustness against temporal and spatial uncertainties, and optimality under large problem dimensions
    corecore