Reinforcement Learning and Tree Search Methods for the Unit Commitment Problem

The unit commitment (UC) problem, which determines operating schedules of generation units to meet demand, is a fundamental task in power systems operation. Existing UC methods using mixed-integer programming are not well-suited to highly stochastic systems. Approaches which more rigorously account for uncertainty could yield large reductions in operating costs by reducing spinning reserve requirements; operating power stations at higher efficiencies; and integrating greater volumes of variable renewables. A promising approach to solving the UC problem is reinforcement learning (RL), a methodology for optimal decision-making which has been used to conquer long-standing grand challenges in artificial intelligence. This thesis explores the application of RL to the UC problem and addresses challenges including robustness under uncertainty; generalisability across multiple problem instances; and scaling to larger power systems than previously studied. To tackle these issues, we develop guided tree search, a novel methodology combining model-free RL and model-based planning. The UC problem is formalised as a Markov decision process and we develop an open-source environment based on real data from Great Britain's power system to train RL agents. In problems of up to 100 generators, guided tree search is shown to be competitive with deterministic UC methods, reducing operating costs by up to 1.4\%. An advantage of RL is that the framework can be easily extended to incorporate considerations important to power systems operators such as robustness to generator failure, wind curtailment or carbon prices. When generator outages are considered, guided tree search saves over 2\% in operating costs as compared with methods using conventional $N-x$ reserve criteria

Multilevel decision-making: A survey

© 2016 Elsevier Inc. All rights reserved. Multilevel decision-making techniques aim to deal with decentralized management problems that feature interactive decision entities distributed throughout a multiple level hierarchy. Significant efforts have been devoted to understanding the fundamental concepts and developing diverse solution algorithms associated with multilevel decision-making by researchers in areas of both mathematics/computer science and business areas. Researchers have emphasized the importance of developing a range of multilevel decision-making techniques to handle a wide variety of management and optimization problems in real-world applications, and have successfully gained experience in this area. It is thus vital that a high quality, instructive review of current trends should be conducted, not only of the theoretical research results but also the practical developments in multilevel decision-making in business. This paper systematically reviews up-to-date multilevel decision-making techniques and clusters related technique developments into four main categories: bi-level decision-making (including multi-objective and multi-follower situations), tri-level decision-making, fuzzy multilevel decision-making, and the applications of these techniques in different domains. By providing state-of-the-art knowledge, this survey will directly support researchers and practical professionals in their understanding of developments in theoretical research results and applications in relation to multilevel decision-making techniques

Unit commitment : tight formulation and pricing

In recent years, with increasing renewable supply variability, thermal power plants have started up and shut down more frequently. These discrete commitment decisions, optimized in the unit commitment (UC) problem, have an impact on system operations as well as generation expansion planning (GEP). The non-convex costs associated with the commitment decisions may also lead to generators' incentive to deviate from the optimal dispatch under locational marginal prices. In this dissertation, we first propose a convex relaxation of UC based on a primal formulation of the Lagrangian dual problem. This convex relaxation is used (i) to solve the convex hull pricing problem in polynomial time, providing prices with better incentives in non-convex electricity markets, and (ii) to construct a computationally efficient GEP model that represents operational flexibility limits. Next, we present a tight formulation for the commitment of combined-cycle units with representation of their transition ramping. Finally, we propose a pricing method that reduces out-of-market payments in multi-interval real-time markets.Electrical and Computer Engineerin

Traffic prediction and bilevel network design

Cette thèse porte sur la modélisation du trafic dans les réseaux routiers et comment celle-ci est intégrée dans des modèles d'optimisation. Ces deux sujets ont évolué de manière plutôt disjointe: le trafic est prédit par des modèles mathématiques de plus en plus complexes, mais ce progrès n'a pas été incorporé dans les modèles de design de réseau dans lesquels les usagers de la route jouent un rôle crucial. Le but de cet ouvrage est d'intégrer des modèles d'utilités aléatoires calibrés avec de vraies données dans certains modèles biniveaux d'optimisation et ce, par une décomposition de Benders efficace. Cette décomposition particulière s'avère être généralisable par rapport à une grande classe de problèmes communs dans la litérature et permet d'en résoudre des exemples de grande taille. Le premier article présente une méthodologie générale pour utiliser des données GPS d'une flotte de véhicules afin d'estimer les paramètres d'un modèle de demande dit recursive logit. Les traces GPS sont d'abord associées aux liens d'un réseau à l'aide d'un algorithme tenant compte de plusieurs facteurs. Les chemins formés par ces suites de liens et leurs caractéristiques sont utilisés afin d'estimer les paramètres d'un modèle de choix. Ces paramètres représentent la perception qu'ont les usagers de chacune de ces caractéristiques par rapport au choix de leur chemin. Les données utilisées dans cet article proviennent des véhicules appartenant à plusieurs compagnies de transport opérant principalement dans la région de Montréal. Le deuxième article aborde l'intégration d'un modèle de choix de chemin avec utilités aléatoires dans une nouvelle formulation biniveau pour le problème de capture de flot de trafic. Le modèle proposé permet de représenter différents comportements des usagers par rapport à leur choix de chemin en définissant les utilités d'arcs appropriées. Ces utilités sont stochastiques ce qui contribue d'autant plus à capturer un comportement réaliste des usagers. Le modèle biniveau est rendu linéaire à travers l'ajout d'un terme lagrangien basé sur la dualité forte et ceci mène à une décomposition de Benders particulièrement efficace. Les expériences numériques sont principalement menés sur un réseau représentant la ville de Winnipeg ce qui démontre la possibilité de résoudre des problèmes de taille relativement grande. Le troisième article démontre que l'approche du second article peut s'appliquer à une forme particulière de modèles biniveaux qui comprennent plusieurs problèmes différents. La décomposition est d'abord présentée dans un cadre général, puis dans un contexte où le second niveau du modèle biniveau est un problème de plus courts chemins. Afin d'établir que ce contexte inclut plusieurs applications, deux applications distinctes sont adaptées à la forme requise: le transport de matières dangeureuses et la capture de flot de trafic déterministe. Une troisième application, la conception et l'établissement de prix de réseau simultanés, est aussi présentée de manière similaire à l'Annexe B de cette thèse.The subject of this thesis is the modeling of traffic in road networks and its integration in optimization models. In the literature, these two topics have to a large extent evolved independently: traffic is predicted more accurately by increasingly complex mathematical models, but this progress has not been incorporated in network design models where road users play a crucial role. The goal of this work is to integrate random utility models calibrated with real data into bilevel optimization models through an efficient Benders decomposition. This particular decomposition generalizes to a wide class of problems commonly found in the literature and can be used to solved large-scale instances. The first article presents a general methodology to use GPS data gathered from a fleet of vehicles to estimate the parameters of a recursive logit demand model. The GPS traces are first matched to the arcs of a network through an algorithm taking into account various factors. The paths resulting from these sequences of arcs, along with their characteristics, are used to estimate parameters of a choice model. The parameters represent users' perception of each of these characteristics in regards to their path choice behaviour. The data used in this article comes from trucks used by a number of transportation companies operating mainly in the Montreal region. The second article addresses the integration of a random utility maximization model in a new bilevel formulation for the general flow capture problem. The proposed model allows for a representation of different user behaviors in regards to their path choice by defining appropriate arc utilities. These arc utilities are stochastic which further contributes in capturing real user behavior. This bilevel model is linearized through the inclusion of a Lagrangian term based on strong duality which paves the way for a particularly efficient Benders decomposition. The numerical experiments are mostly conducted on a network representing the city of Winnipeg which demonstrates the ability to solve problems of a relatively large size. The third article illustrates how the approach used in the second article can be generalized to a particular form of bilevel models which encompasses many different problems. The decomposition is first presented in a general setting and subsequently in a context where the lower level of the bilevel model is a shortest path problem. In order to demonstrate that this form is general, two distinct applications are adapted to fit the required form: hazmat transportation network design and general flow capture. A third application, joint network design and pricing, is also similarly explored in Appendix B of this thesis

The Cooperative Maximum Capture Facility Location Problem

In the Maximum Capture Facility Location (MCFL) problem with a binary choice rule, a company intends to locate a series of facilities to maximize the captured demand, and customers patronize the facility that maximizes their utility. In this work, we generalize the MCFL problem assuming that the facilities of the decision maker act cooperatively to increase the customers' utility over the company. We propose a utility maximization rule between the captured utility of the decision maker and the opt-out utility of a competitor already installed in the market. Furthermore, we model the captured utility by means of an Ordered Median function (OMf) of the partial utilities of newly open facilities. We name this problem "the Cooperative Maximum Capture Facility Location problem" (CMCFL). The OMf serves as a means to compute the utility of each customer towards the company as an aggregation of ordered partial utilities, and constitutes a unifying framework for CMCFL models. We introduce a multiperiod non-linear bilevel formulation for the CMCFL with an embedded assignment problem characterizing the captured utilities. For this model, two exact resolution approaches are presented: a MILP reformulation with valid inequalities and an effective approach based on Benders' decomposition. Extensive computational experiments are provided to test our results with randomly generated data and an application to the location of charging stations for electric vehicles in the city of Trois-Rivi\`eres, Qu\`ebec, is addressed.Comment: 32 pages, 8 tables, 2 algorithms, 8 figure

REKABETÇİ TESİS YER SEÇİMİ PROBLEMLERİNE İLİŞKİN BİR TARAMA ÇALIŞMASI

Amaç: Bu çalışmanın amacı, son yıllarda Rekabetçi Tesis Yer Seçimi (RTYS) problemlerinin ve problem bileşenlerinin literatürde ele alınış biçimlerine ilişkin bir bilimsel yayın taraması sunmaktadır. Yöntem: İlk olarak literatürde problemin temel bileşenlerinin ele alınış biçimlerine yer verilmiştir. Daha sonra RTYS problemi için literatürdeki en temel sınıflandırma kriteri olan rekabet tiplerine göre problem türleri incelenmiştir. Son olarak genişletilmiş RTYS problem türlerini ve çok amaçlı RTYS problemlerini ele alan çalışmalara yer verilmiş ve tarama çalışmasının sonuçları sunulmuştur. Bulgular: Tarama çalışması sonucu RTYS alanında gelecek vadeden çalışma konuları; birden fazla firmanın pazar paylarının enbüyüklenmesi amaçlarının çok-amaçlı olarak ele alındığı RTYS problemleri, müşterilerin tesis seçimlerinin çok amaçlı eniyileme kullanılarak yapıldığı RTYS problemleri, ikiden fazla rakip firma içeren RTYS problemleri olarak belirlenmiştir. Özgünlük: RTYS, hem tedarik zinciri için en önemli stratejik kararlardan biri olması hem de gerçek hayat problemlerine uygulanabilirliğinin yüksek olması sebebiyle araştırmacıların üzerinde durdukları bir alan olmuştur. Özellikle son yıllarda RTYS problemleri ve varyasyonları üzerinde önemli gelişmeler kaydedilmiştir. RTYS literatürüne ilişkin son çalışma Ashtiani (2016) tarafından yapılmıştır ve 2015 yılına kadar yapılan çalışmaları içermektedir. Bu çalışmada 2010 – 2020 arasında yapılan bilimsel çalışmaları içeren özgün bir tarama çalışması sunulmaktadır

Secure and cost-effective operation of low carbon power systems under multiple uncertainties

Power system decarbonisation is driving the rapid deployment of renewable energy sources (RES) like wind and solar at the transmission and distribution level. Their differences from the synchronous thermal plants they are displacing make secure and efficient grid operation challenging. Frequency stability is of particular concern due to the current lack of provision of frequency ancillary services like inertia or response from RES generators. Furthermore, the weather dependency of RES generation coupled with the proliferation of distributed energy resources (DER) like small-scale solar or electric vehicles permeates future low-carbon systems with uncertainty under which legacy scheduling methods are inadequate. Overly cautious approaches to this uncertainty can lead to inefficient and expensive systems, whilst naive methods jeopardise system security. This thesis significantly advances the frequency-constrained scheduling literature by developing frameworks that explicitly account for multiple new uncertainties. This is in addition to RES forecast uncertainty which is the exclusive focus of most previous works. The frameworks take the form of convex constraints that are useful in many market and scheduling problems. The constraints equip system operators with tools to explicitly guarantee their preferred level of system security whilst unlocking substantial value from emerging and abundant DERs. A major contribution is to address the exclusion of DERs from the provision of ancillary services due to their intrinsic uncertainty from aggregation. This is done by incorporating the uncertainty into the system frequency dynamics, from which deterministic convex constraints are derived. In addition to managing uncertainty to facilitate emerging DERs to provide legacy frequency services, a novel frequency containment service is designed. The framework allows a small amount of load shedding to assist with frequency containment during high RES low inertia periods. The expected cost of this service is probabilistic as it is proportional to the probability of a contingency occurring. The framework optimally balances the potentially higher expected costs of an outage against the operational cost benefits of lower ancillary service requirements day-to-day. The developed frameworks are applied extensively to several case studies. These validate their security and demonstrate their significant economic and emission-saving benefits.Open Acces

Smart Sustainable Mobility: Analytics and Algorithms for Next-Generation Mobility Systems

To this date, mobility ecosystems around the world operate on an uncoordinated, inefficient and unsustainable basis. Yet, many technology-enabled solutions that have the potential to remedy these societal negatives are already at our disposal or just around the corner. Innovations in vehicle technology, IoT devices, mobile connectivity and AI-powered information systems are expected to bring about a mobility system that is connected, autonomous, shared and electric (CASE). In order to fully leverage the sustainability opportunities afforded by CASE, system-level coordination and management approaches are needed. This Thesis sets out an agenda for Information Systems research to shape the future of CASE mobility through data, analytics and algorithms (Chapter 1). Drawing on causal inference, (spatial) machine learning, mathematical programming and reinforcement learning, three concrete contributions toward this agenda are developed. Chapter 2 demonstrates the potential of pervasive and inexpensive sensor technology for policy analysis. Connected sensing devices have significantly reduced the cost and complexity of acquiring high-resolution, high-frequency data in the physical world. This affords researchers the opportunity to track temporal and spatial patterns of offline phenomena. Drawing on a case from the bikesharing sector, we demonstrate how geo-tagged IoT data streams can be used for tracing out highly localized causal effects of large-scale mobility policy interventions while offering actionable insights for policy makers and practitioners. Chapter 3 sets out a solution approach to a novel decision problem faced by operators of shared mobility fleets: allocating vehicle inventory optimally across a network when competition is present. The proposed three-stage model combines real-time data analytics, machine learning and mixed integer non-linear programming into an integrated framework. It provides operational decision support for fleet managers in contested shared mobility markets by generating optimal vehicle re-positioning schedules in real time. Chapter 4 proposes a method for leveraging data-driven digital twin (DT) frameworks for large multi-stage stochastic design problems. Such problem classes are notoriously difficult to solve with traditional stochastic optimization. Drawing on the case of Electric Vehicle Charging Hubs (EVCHs), we show how high-fidelity, data-driven DT simulation environments fused with reinforcement learning (DT-RL) can achieve (close-to) arbitrary scalability and high modeling flexibility. In benchmark experiments we demonstrate that DT-RL-derived designs result in superior cost and service-level performance under real-world operating conditions