Load Frequency Control: A Deep Multi-Agent Reinforcement Learning Approach

The paradigm shift in energy generation towards microgrid-based architectures is changing the landscape of the energy control structure heavily in distribution systems. More specifically, distributed generation is deployed in the network demanding decentralised control mechanisms to ensure reliable power system operations. In this work, a Multi-Agent Reinforcement Learning approach is proposed to deliver an agentbased solution to implement load frequency control without the need of a centralised authority. Multi-Agent Deep Deterministic Policy Gradient is used to approximate the frequency control at the primary and the secondary levels. Each generation unit is represented as an agent that is modelled by a Recurrent Neural Network. Agents learn the optimal way of acting and interacting with the environment to maximise their long term performance and to balance generation and load, thus restoring frequency. In this paper we prove using three test systems, with two, four and eight generators, that our Multi-Agent Reinforcement Learning approach can efficiently be used to perform frequency control in a decentralised way

Cost Efficient Distributed Load Frequency Control in Power Systems

The introduction of new technologies and increased penetration of renewable resources is altering the power distribution landscape which now includes a larger numbers of micro-generators. The centralized strategies currently employed for performing frequency control in a cost efficient way need to be revisited and decentralized to conform with the increase of distributed generation in the grid. In this paper, the use of Multi-Agent and Multi-Objective Reinforcement Learning techniques to train models to perform cost efficient frequency control through decentralized decision making is proposed. More specifically, we cast the frequency control problem as a Markov Decision Process and propose the use of reward composition and action composition multi-objective techniques and compare the results between the two. Reward composition is achieved by increasing the dimensionality of the reward function, while action composition is achieved through linear combination of actions produced by multiple single objective models. The proposed framework is validated through comparing the observed dynamics with the acceptable limits enforced in the industry and the cost optimal setups

Effects of Solar and Wind Generation Integration on Feeder Hosting Capacity

With the increased penetration of distributed generation (DG) utilities are beginning to see impacts on their system, especially on the ability of a feeder to accommodate DG. In this paper we introduce a stochastic simulation framework to assess the effects on hosting capacity from solar and wind generation for various loading scenarios. The general approach includes the use of a k-means clustering algorithm for segmenting and grouping the raw wind, solar, and load data to define patterns and assign probabilities to each pattern. Monte Carlo simulations are adopted for calculating probabilistic outcomes for a variety of wind, solar, and load scenarios, with the use of a distribution planning software. The outcomes of the simulations, i.e., statistics of minimum and maximum feeder hosting capacity, are used to derive their probability distribution functions (pdfs). The pdfs of the minimum and maximum hosting capacity provide insights into the effects on loading from various wind and solar DG scenarios. The proposed framework is illustrated for a representative utility feeder

Stochastic Hosting Capacity in LV Distribution Networks

Hosting capacity is defined as the level of penetration that a particular technology can connect to a distribution network without causing power quality problems. In this work, we study the impact of solar photovoltaics (PV) on voltage rise. In most cases, the locations and sizes of the PV are not known in advance, so hosting capacity must be considered as a random variable. Most hosting capacity methods study the problem considering a large number of scenarios, many of which provide little additional information. We overcome this problem by studying only cases where voltage constraints are active, with results illustrating a reduction in the number of scenarios required by an order of magnitude. A linear power flow model is utilised for this task, showing excellent performance. The hosting capacity is finally studied as a function of the number of generators connected, demonstrating that assumptions about the penetration level will have a large impact on the conclusions drawn for a given network

Cascade Hydroelectric Power System Model and its Application to an Optimal Dispatch Design

In this paper we propose an optimal dispatch scheme for a cascade hydroelectric power system that maximises the system efficiency, and minimises the spillage effects. Our approach proposes a methodology that has low computational burden and may be implemented for the short-term operation of a cascade hydroelectric power system. To this end, the non- linear relationships that describe the system physical constraints, e.g., power output, are transformed into affine relationships; thus reducing the computational complexity. The transformations are based on the construction of convex envelopes around bilinear functions; piecewise affine functions; and exploitation of optimisation properties. We demonstrate the efficacy of the proposed methodology with the Seven Forks system located in Kenya, and evaluate the performance of our method in terms of water volume and potential energy saved

Automatic Generation Control and its Implementation in Real Time

In power systems, the control mechanism responsible for maintaining the system frequency to the nominal value and the real power interchange between balancing authority areas to the scheduled values is referred to as automatic generation control (AGC). The purpose of this paper is to present a systematic way to determine, in real time, the power allocated to each generator participating in AGC by taking into account the cost and quality of the AGC service provided. To this end, we formulate the economic dispatch process and gain insights into the economic characteristics of the generating units. We value the quality of AGC service by taking into consideration the ramping constraints of the generating units. The proposed methodology is illustrated in the WECC system and is compared with other allocation methods

An Assessment of the Impact of Uncertainty on Automatic Generation Control Systems

This paper proposes a framework to quantify the impact of uncertainty that arises from load variations, renewable-based generation, and noise in communication channels on the automatic generation control (AGC) system. To this end, we rely on a model of the power system that includes the synchronous generator dynamics, the network, and the AGC system dynamics, as well as the effect of various sources of uncertainty. Then, we develop a method to analytically propagate the uncertainty from the aforementioned sources to the system frequency and area control error (ACE), and obtain expressions that approximate their probability distribution functions. We make use of this framework to obtain probabilistic expressions for the frequency performance criteria developed by the North American Electric Reliability Corporation (NERC); such expressions may be used to determine the limiting values of uncertainty that the system may withstand. The proposed ideas are illustrated through the Western Electricity Coordination Council (WECC) 9-bus 3-machine system and a 140-bus 48-machine system

Balancing Authority Area Coordination with Limited Exchange of Information

In this paper, we propose a coordination scheme between balancing authority (BA) areas in an interconnected power system that decreases the regulation amount needed as well as the associated costs. Our approach aims at mimicking the behavior of the automatic generation control (AGC) system in a scenario where the whole interconnected system is assumed to be operated by a single BA area. To this end, we modify the area control error (ACE), which is fed into the AGC system of each BA area, and determine the AGC allocation based on a distributed algorithm that identifies the least expensive generators, with the mismatch of the total regulation needed being the only information exchanged between BA areas. We demonstrate the proposed ideas with the 3-machine 9-bus Western Electricity Coordinating Council (WECC) system, and compare the performance of our method with other three existing coordination approaches

Optimal Dispatch of Pumped Storage Hydro Cascade under Uncertainty

In this paper, we propose an optimal dispatch scheme for a pumped storage hydro cascade that maximizes the energy per cubic meter of water in the system taking into account uncertainty in the net load variations. To this end, we introduce a model to describe the behaviour of a pumped storage hydro cascade and formulate its optimal dispatch. We then incorporate forecast scenarios in the optimal dispatch, and define a robust variant of the developed system. The resulting optimization problem is intractable due to the infinite number of constraints. Using tools from robust optimization, we reformulate the resulting problem in a tractable form that is amenable to existing numerical tools and show that the computed dispatch is immunised against uncertainty. The efficacy of the proposed approach is demonstrated by means of a realistic case study based on the Seven Forks system located in Kenya

Clustering of Usage Profiles for Electric Vehicle Behaviour Analysis

Accurately predicting the behaviour of electric vehicles is going to be imperative for network operators. In order for vehicles to participate in either smart charging schemes or providing grid services, their availability and charge requirements must be forecasted. Their relative novelty means that data concerning electric vehicles is scarce and biased, however we have been collecting data on conventional vehicles for many years. This paper uses cluster analysis of travel survey data from the UK to identify typical conventional vehicle usage profiles. To this end, we determine the feature vector, introduce an appropriate distance metric, and choose a number of clusters. Five clusters are identified, and their suitability for electrification is discussed. A smaller data set of electric vehicles is then used to compare the current electric fleet behaviour with the conventional one