Reinforcement Learning and Tree Search Methods for the Unit Commitment Problem

The unit commitment (UC) problem, which determines operating schedules of generation units to meet demand, is a fundamental task in power systems operation. Existing UC methods using mixed-integer programming are not well-suited to highly stochastic systems. Approaches which more rigorously account for uncertainty could yield large reductions in operating costs by reducing spinning reserve requirements; operating power stations at higher efficiencies; and integrating greater volumes of variable renewables. A promising approach to solving the UC problem is reinforcement learning (RL), a methodology for optimal decision-making which has been used to conquer long-standing grand challenges in artificial intelligence. This thesis explores the application of RL to the UC problem and addresses challenges including robustness under uncertainty; generalisability across multiple problem instances; and scaling to larger power systems than previously studied. To tackle these issues, we develop guided tree search, a novel methodology combining model-free RL and model-based planning. The UC problem is formalised as a Markov decision process and we develop an open-source environment based on real data from Great Britain's power system to train RL agents. In problems of up to 100 generators, guided tree search is shown to be competitive with deterministic UC methods, reducing operating costs by up to 1.4\%. An advantage of RL is that the framework can be easily extended to incorporate considerations important to power systems operators such as robustness to generator failure, wind curtailment or carbon prices. When generator outages are considered, guided tree search saves over 2\% in operating costs as compared with methods using conventional $N-x$ reserve criteria

Contingency-Constrained Unit Commitment With Intervening Time for System Adjustments

The N-1-1 contingency criterion considers the con- secutive loss of two components in a power system, with intervening time for system adjustments. In this paper, we consider the problem of optimizing generation unit commitment (UC) while ensuring N-1-1 security. Due to the coupling of time periods associated with consecutive component losses, the resulting problem is a very large-scale mixed-integer linear optimization model. For efficient solution, we introduce a novel branch-and-cut algorithm using a temporally decomposed bilevel separation oracle. The model and algorithm are assessed using multiple IEEE test systems, and a comprehensive analysis is performed to compare system performances across different contingency criteria. Computational results demonstrate the value of considering intervening time for system adjustments in terms of total cost and system robustness.Comment: 8 pages, 5 figure

An Evolutionary Computational Approach for the Problem of Unit Commitment and Economic Dispatch in Microgrids under Several Operation Modes

In the last decades, new types of generation technologies have emerged and have been gradually integrated into the existing power systems, moving their classical architectures to distributed systems. Despite the positive features associated to this paradigm, new problems arise such as coordination and uncertainty. In this framework, microgrids constitute an effective solution to deal with the coordination and operation of these distributed energy resources. This paper proposes a Genetic Algorithm (GA) to address the combined problem of Unit Commitment (UC) and Economic Dispatch (ED). With this end, a model of a microgrid is introduced together with all the control variables and physical constraints. To optimally operate the microgrid, three operation modes are introduced. The first two attend to optimize economical and environmental factors, while the last operation mode considers the errors induced by the uncertainties in the demand forecasting. Therefore, it achieves a robust design that guarantees the power supply for different confidence levels. Finally, the algorithm was applied to an example scenario to illustrate its performance. The achieved simulation results demonstrate the validity of the proposed approach.Ministerio de Ciencia, Innovación y Universidades TEC2016-80242-PMinisterio de Economía y Competitividad PCIN-2015-043Universidad de Sevilla Programa propio de I+D+

Reinforcement learning and A* search for the unit commitment problem

Previous research has combined model-free reinforcement learning with model-based tree search methods to solve the unit commitment problem with stochastic demand and renewables generation. This approach was limited to shallow search depths and suffered from significant variability in run time across problem instances with varying complexity. To mitigate these issues, we extend this methodology to more advanced search algorithms based on A* search. First, we develop a problem-specific heuristic based on priority list unit commitment methods and apply this in Guided A* search, reducing run time by up to 94% with negligible impact on operating costs. In addition, we address the run time variability issue by employing a novel anytime algorithm, Guided IDA*, replacing the fixed search depth parameter with a time budget constraint. We show that Guided IDA* mitigates the run time variability of previous guided tree search algorithms and enables further operating cost reductions of up to 1%

Benefits of demand-side response in providing frequency response service in the future GB power system

The demand for ancillary service is expected to increase significantly in the future Great Britain (GB) electricity system due to high penetration of wind. In particular, the need for frequency response, required to deal with sudden frequency drops following a loss of generator, will increase because of the limited inertia capability of wind plants. This paper quantifies the requirements for primary frequency response and analyses the benefits of frequency response provision from demand-side response (DSR). The results show dramatic changes in frequency response requirements driven by high penetration of wind. Case studies carried out by using an advanced stochastic generation scheduling model suggest that the provision of frequency response from DSR could greatly reduce the system operation cost, wind curtailment, and carbon emissions in the future GB system characterized by high penetration of wind. Furthermore, the results demonstrate that the benefit of DSR shows significant diurnal and seasonal variation, whereas an even more rapid (instant) delivery of frequency response from DSR could provide significant additional value. Our studies also indicate that the competing technologies to DSR, namely battery storage, and more flexible generation could potentially reduce its value by up to 35%, still leaving significant room to deploy DSR as frequency response provider