Federated Reinforcement Learning for Electric Vehicles Charging Control on Distribution Networks

With the growing popularity of electric vehicles (EVs), maintaining power grid stability has become a significant challenge. To address this issue, EV charging control strategies have been developed to manage the switch between vehicle-to-grid (V2G) and grid-to-vehicle (G2V) modes for EVs. In this context, multi-agent deep reinforcement learning (MADRL) has proven its effectiveness in EV charging control. However, existing MADRL-based approaches fail to consider the natural power flow of EV charging/discharging in the distribution network and ignore driver privacy. To deal with these problems, this paper proposes a novel approach that combines multi-EV charging/discharging with a radial distribution network (RDN) operating under optimal power flow (OPF) to distribute power flow in real time. A mathematical model is developed to describe the RDN load. The EV charging control problem is formulated as a Markov Decision Process (MDP) to find an optimal charging control strategy that balances V2G profits, RDN load, and driver anxiety. To effectively learn the optimal EV charging control strategy, a federated deep reinforcement learning algorithm named FedSAC is further proposed. Comprehensive simulation results demonstrate the effectiveness and superiority of our proposed algorithm in terms of the diversity of the charging control strategy, the power fluctuations on RDN, the convergence efficiency, and the generalization ability

A Systematic Literature Review of Peer-to-Peer, Community Self-Consumption, and Transactive Energy Market Models

Capper, T., Gorbatcheva, A., Mustafa, M. A., Bahloul, M., Schwidtal, J. M., Chitchyan, R., Andoni, M., Robu, V., Montakhabi, M., Scott, I., Francis, C., Mbavarira, T., Espana, J. M., & Kiesling, L. (2021). A Systematic Literature Review of Peer-to-Peer, Community Self-Consumption, and Transactive Energy Market Models. Social Science Research Network (SSRN), Elsevier. https://doi.org/10.2139/ssrn.3959620Peer-to-peer and transactive energy markets, and community or collective self-consumption offer new models for trading energy locally. Over the past 10 years there has been significant growth in the amount of academic literature and trial projects examining how these energy trading models might function. This systematic literature review of 139 peer-reviewed journal articles examines the market designs used in these energy trading models. The Business Ecosystem Architecture Modelling framework is used to extract information about the market models used in the literature and identify differences and similarities between the models. This paper identifies six archetypal market designs and three archetypal auction mechanisms used in markets presented in the reviewed literature. It classifies the types of commodities being traded, the benefits of the markets and other features such as the types of grid models. Finally, this paper identifies five evidence gaps which need future research before these markets can be widely adopted.publishersversionpublishe

Discovering Communities for Microgrids with Spatial-Temporal Net Energy

Smart grid has integrated an increasing number of distributed energy resources to improve the efficiency and flexibility of power generation and consumption as well as the resilience of the power grid. The energy consumers on the power grid, e.g., households, equipped with distributed energy resources can be considered as “microgrids” that both generate and consume electricity. In this paper, we study the energy community discovery problems which identify energy communities for the microgrids to facilitate energy management, e.g., load balancing, energy sharing and trading on the grid. Specifically, we present efficient algorithms to discover such communities of microgrids considering both their geo-locations and net energy (NE) over any period. Finally, we experimentally validate the performance of the algorithms using both synthetic and real datasets

A generic holonic control architecture for heterogeneous multi-scale and multi-objective smart microgrids

Designing the control infrastructure of future “smart” power grids is a challenging task. Future grids will integrate a wide variety of heterogeneous producers and consumers that are unpredictable and operate at various scales. Information and Communication Technology (ICT) solutions will have to control these in order to attain global objectives at the macrolevel, while also considering private interests at the microlevel. This article proposes a generic holonic architecture to help the development of ICT control systems that meet these requirements. We show how this architecture can integrate heterogeneous control designs, including state-of-the-art smart grid solutions. To illustrate the applicability and utility of this generic architecture, we exemplify its use via a concrete proof-of-concept implementation for a holonic controller, which integrates two types of control solutions and manages a multiscale, multiobjective grid simulator in several scenarios. We believe that the proposed contribution is essential for helping to understand, to reason about, and to develop the “smart” side of future power grids

Taming Instabilities in Power Grid Networks by Decentralized Control

Renewables will soon dominate energy production in our electric power system. And yet, how to integrate renewable energy into the grid and the market is still a subject of major debate. Decentral Smart Grid Control (DSGC) was recently proposed as a robust and decentralized approach to balance supply and demand and to guarantee a grid operation that is both economically and dynamically feasible. Here, we analyze the impact of network topology by assessing the stability of essential network motifs using both linear stability analysis and basin volume for delay systems. Our results indicate that if frequency measurements are averaged over sufficiently large time intervals, DSGC enhances the stability of extended power grid systems. We further investigate whether DSGC supports centralized and/or decentralized power production and find it to be applicable to both. However, our results on cycle-like systems suggest that DSGC favors systems with decentralized production. Here, lower line capacities and lower averaging times are required compared to those with centralized production.Comment: 21 pages, 6 figures This is a pre-print of a manuscript submitted to The European Physical Journal. The final publication is available at Springer via http://dx.doi.org/10.1140/epjst/e2015-50136-