Constrained optimization under uncertainty for decision-making problems: Application to Real-Time Strategy games

Decision-making problems can be modeled as combinatorial optimization problems with Constraint Programming formalisms such as Constrained Optimization Problems. However, few Constraint Programming formalisms can deal with both optimization and uncertainty at the same time, and none of them are convenient to model problems we tackle in this paper. Here, we propose a way to deal with combinatorial optimization problems under uncertainty within the classical Constrained Optimization Problems formalism by injecting the Rank Dependent Utility from decision theory. We also propose a proof of concept of our method to show it is implementable and can solve concrete decision-making problems using a regular constraint solver, and propose a bot that won the partially observable track of the 2018 {\mu}RTS AI competition. Our result shows it is possible to handle uncertainty with regular Constraint Programming solvers, without having to define a new formalism neither to develop dedicated solvers. This brings new perspective to tackle uncertainty in Constraint Programming.Comment: Published at the 2019 IEEE Congress on Evolutionary Computation (CEC'19

A decomposition strategy for decision problems with endogenous uncertainty using mixed-integer programming

Despite methodological advances for modeling decision problems under uncertainty, faithfully representing endogenous uncertainty still proves challenging, both in terms of modeling capabilities and computational requirements. A novel framework called Decision Programming provides an approach for solving such decision problems using off-the-shelf mathematical optimization solvers. This is made possible by using influence diagrams to represent a given decision problem, which is then formulated as a mixed-integer linear programming problem. In this paper, we focus on the type of endogenous uncertainty that received less attention in the introduction of Decision Programming: conditionally observed information. Multi-stage stochastic programming (MSSP) models use conditional non-anticipativity constraints (C-NACs) to represent such uncertainties, and we show how such constraints can be incorporated into Decision Programming models. This allows us to consider the two main types of endogenous uncertainty simultaneously, namely decision-dependent information structure and decision-dependent probability distribution. Additionally, we present a decomposition approach that provides significant computational savings and also enables considering continuous decision variables in certain parts of the problem, whereas the original formulation was restricted to discrete variables only. The extended framework is illustrated with two example problems. The first considers an illustrative multiperiod game and the second is a large-scale cost-benefit problem regarding climate change mitigation. Neither of these example problems could be solved with existing frameworks.Comment: 26 pages, 10 figure

Bayesian Stochastic Gradient Descent for Stochastic Optimization with Streaming Input Data

We consider stochastic optimization under distributional uncertainty, where the unknown distributional parameter is estimated from streaming data that arrive sequentially over time. Moreover, data may depend on the decision of the time when they are generated. For both decision-independent and decision-dependent uncertainties, we propose an approach to jointly estimate the distributional parameter via Bayesian posterior distribution and update the decision by applying stochastic gradient descent on the Bayesian average of the objective function. Our approach converges asymptotically over time and achieves the convergence rates of classical SGD in the decision-independent case. We demonstrate the empirical performance of our approach on both synthetic test problems and a classical newsvendor problem

Probabilistic engineering analysis and design under time-dependent uncertainty

Time-dependent uncertainties, such as time-variant stochastic loadings and random deterioration of material properties, are inherent in engineering applications. Not considering these uncertainties in the design process may result in catastrophic failures after the designed products are put into operation. Although significant progress has been made in probabilistic engineering design, quantifying and mitigating the effects of time-dependent uncertainty is still challenging. This dissertation aims to help build high reliability into products under time-dependent uncertainty by addressing two research issues. The first one is to efficiently and accurately predict the time-dependent reliability while the second one is to effectively design the time-dependent reliability into the product. For the first research issue, new time-dependent reliability analysis methodologies are developed, including the joint upcrossing rate method, the surrogate model method, the global efficient optimization, and the random field approach. For the second research issue, a time-dependent sequential optimization and reliability analysis method is proposed. The developed approaches are applied to the reliability analysis of designing a hydrokinetic turbine blade subjected to stochastic river flow loading. Extension of the proposed methods to the reliability analysis with mixture of random and interval variables is also a contribution of this dissertation. The engineering examples tested in in this work demonstrate that the proposed time-dependent reliability methods can improve the efficiency and accuracy more than 100% and that high reliability can be successfully built into products with the proposed method. The research results can benefit a wide range of areas, such as life cycle cost optimization and decision making --Abstract, page iv

Constant Depth Decision Rules for multistage optimization under uncertainty

In this paper, we introduce a new class of decision rules, referred to as Constant Depth Decision Rules (CDDRs), for multistage optimization under linear constraints with uncertainty-affected right-hand sides. We consider two uncertainty classes: discrete uncertainties which can take at each stage at most a fixed number d of different values, and polytopic uncertainties which, at each stage, are elements of a convex hull of at most d points. Given the depth mu of the decision rule, the decision at stage t is expressed as the sum of t functions of mu consecutive values of the underlying uncertain parameters. These functions are arbitrary in the case of discrete uncertainties and are poly-affine in the case of polytopic uncertainties. For these uncertainty classes, we show that when the uncertain right-hand sides of the constraints of the multistage problem are of the same additive structure as the decision rules, these constraints can be reformulated as a system of linear inequality constraints where the numbers of variables and constraints is O(1)(n+m)d^mu N^2 with n the maximal dimension of control variables, m the maximal number of inequality constraints at each stage, and N the number of stages. As an illustration, we discuss an application of the proposed approach to a Multistage Stochastic Program arising in the problem of hydro-thermal production planning with interstage dependent inflows. For problems with a small number of stages, we present the results of a numerical study in which optimal CDDRs show similar performance, in terms of optimization objective, to that of Stochastic Dual Dynamic Programming (SDDP) policies, often at much smaller computational cost

Адаптивне керування слабкокерованими марковськими та напівмарковськими моделями в дискретному часі

Досліджується байєсів підхід до проблеми марковських процесів рішень в умовах стохастичної невизначеності, коли невідомі перехідні ймовірності слабко збурені, і тільки збурення залежать від стратегії рішень. Процес рішень припускається стаціонарним, розглядається в дискретному часі з скінченним, зчисленним або вимірним фазовим простором і ґрунтується на принципі розділення задач оцінювання та оптимізації.Исследуется байесов подход к проблеме марковских процессов решений в условиях стохастической неопределенности, когда неизвестные переходные вероятности слабо возмущены, и только возмущения зависят от стратегии решений. Процесс решений предполагается стационарным в дискретном времени с конечным, счетным или измеримым фазовым пространством и базируется на принципе разделения задач оценивания и оптимизации.A Bayesian approach to Markov decision process problem under stochastic uncertainty, when unknown transition probabilities are weakly disturbed with disturbances dependent on a decision strategy only is investigated. Observed decision process is assumed to be stationary in discrete time with finite, countable or measurable phase state is based on separation principle of assessment and optimization problems

Efficient time-dependent system reliability analysis

Engineering systems are usually subjected to time-variant loads and operate under time-dependent uncertainty; system performances are therefore time-dependent. Accurate and efficient estimate of system reliability is crucial for decision makings on system design, lifetime cost estimate, maintenance strategy, etc. Although significant progresses have been made in time-independent reliability analysis for components and systems, time-dependent system reliability methodologies are still limited. This dissertation is motivated by the need of accurate and effective reliability prediction for engineering systems under time-dependent uncertainty. Based on the classic First and Second Order Reliability Method (FORM and SORM), a system reliability method is developed for multidisciplinary systems involving stationary stochastic processes. A dependent Kriging method is also developed for general components. This method accounts for dependent responses from surrogate models and is therefore more accurate than existing Kriging Monte Carlo simulation methods that neglect the dependence between responses. The extension of the dependent Kriging method to systems is also a contribution of this dissertation. To overcome the difficulty of obtaining extreme value distributions and get rid of global optimization with a double-loop procedure, a Kriging surrogate modeling method is also proposed. This method provides a new perspective of surrogate modeling for time-dependent systems and is applicable to general systems having random variables, time, and stochastic processes. The proposed methods are evaluated through a wide range of engineering systems, including a compound cylinders system, a liquid hydrogen fuel tank, function generator mechanisms, slider-crank mechanisms, and a Daniels system --Abstract, page iv

Option valuation of smart grid technology projects under endogenous and exogenous uncertainty

Electricity demand and renewables penetration are set to increase worldwide over the coming decades as part of the global decarbonisation effort. As a result, distribution networks are expected to face challenges related to increased peaks and undesirable voltage excursions. Hence, significant network reinforcements may be required over the next decades. However, a very significant challenge in realizing this transition is the increased uncertainty that surrounds future distributed generation and load connections in terms of size, location and timing. This uncertainty inadvertently will give rise to the prospect of inefficient investments and stranded assets given that current planning practices remain deterministic. It follows that new planning frameworks are needed that allow the quantification of option value and achieve reduction of stranding risk by encouraging cost-efficient strategic investments through smart technologies under both endogenous and exogenous sources of uncertainty. This thesis presents multi-epoch stochastic optimization models, for the distribution network planning problem, that consider a set of investment options with different techno-economical characteristics so as to reflect the multitude of choices available to planners in a realistic setting characterized by endogenous or exogenous uncertainty. These optimization models are rendered tractable through the use of novel decomposition schemes that effectively help manage the associated increased computational burden. The corresponding simulation results validate that smart technologies constitute valuable options for enabling cost effective integration of distributed generation units and underline the importance of early investment in such assets under decision-dependent uncertainty. In addition, the results emphasize that deterministic approaches systematically undervalue the flexibility that smart assets provide, thereby posing a barrier to the advent of the flexible smart grid paradigm.Open Acces