On Optimizing the Conditional Value-at-Risk of a Maximum Cost for Risk-Averse Safety Analysis

The popularity of Conditional Value-at-Risk (CVaR), a risk functional from finance, has been growing in the control systems community due to its intuitive interpretation and axiomatic foundation. We consider a non-standard optimal control problem in which the goal is to minimize the CVaR of a maximum random cost subject to a Borel-space Markov decision process. The objective takes the form $\text{CVaR}_{\alpha}(\max_{t=0,1,\dots,N} C_t)$, where $\alpha$ is a risk-aversion parameter representing a fraction of worst cases, $C_t$ is a stage or terminal cost, and $N \in \mathbb{N}$ is the length of a finite discrete-time horizon. The objective represents the maximum departure from a desired operating region averaged over a given fraction $\alpha$ of worst cases. This problem provides a safety criterion for a stochastic system that is informed by both the probability and severity of the potential consequences of the system's trajectory. In contrast, existing safety analysis frameworks apply stage-wise risk constraints (i.e., $\rho(C_t)$ must be small for all $t$, where $\rho$ is a risk functional) or assess the probability of constraint violation without quantifying its possible severity. To the best of our knowledge, the problem of interest has not been solved. To solve the problem, we propose and study a family of stochastic dynamic programs on an augmented state space. We prove that the optimal CVaR of a maximum cost enjoys an equivalent representation in terms of the solutions to this family of dynamic programs under appropriate assumptions. We show the existence of an optimal policy that depends on the dynamics of an augmented state under a measurable selection condition. Moreover, we demonstrate how our safety analysis framework is useful for assessing the severity of combined sewer overflows under precipitation uncertainty.Comment: A shorter version is under review for IEEE Transactions on Automatic Control, submitted December 202

학습 기반 자율시스템의 리스크를 고려하는 분포적 강인 최적화

학위논문 (석사) -- 서울대학교 대학원 : 공과대학 전기·정보공학부, 2020. 8. 양인순.In this thesis, a risk-aware motion control scheme is considered for autonomous systems to avoid randomly moving obstacles when the true probability distribution of uncertainty is unknown. We propose a novel model predictive control (MPC) method for motion planning and decision-making that systematically adjusts the safety and conservativeness in an environment with randomly moving obstacles. The key component is the Conditional Value-at-Risk (CVaR), employed to limit the safety risk in the MPC problem. Having the empirical distribution obtained using a limited amount of sample data, Sample Average Approximation (SAA) is applied to compute the safety risk. Furthermore, we propose a method, which limits the risk of unsafety even when the true distribution of the obstacles movements deviates, within an ambiguity set, from the empirical one. By choosing the ambiguity set as a statistical ball with its radius measured by the Wasserstein metric, we achieve a probabilistic guarantee of the out-of-sample risk, evaluated using new sample data generated independently of the training data. A set of reformulations are applied on both SAA-based MPC (SAA-MPC) and Wasserstein Distributionally Robust MPC (DR-MPC) to make them tractable. In addition, we combine the DR-MPC method with Gaussian Process (GP) to predict the future motion of the obstacles from past observations of the environment. The performance of the proposed methods is demonstrated and analyzed through simulation studies using a nonlinear vehicle model and a linearized quadrotor model.본 연구에서 자율 시스템이 알려지지 않은 확률 분포로 랜덤하게 움직이는 장애물을 피하기 위한 위험 인식을 고려하는 모션 제어 기법을 개발한다. 따라서 본 논문에서 안전성과 보수성을 체계적으로 조절하는 새로운 Model Predictive Control (MPC) 방법을 제안한다. 본 방벙의 핵심 요소는 MPC 문제의 안전성 리스크를 제한하는 Conditional Value-at-Risk (CVaR)라는 리스크 척도이다. 안전성 리스크를 계산하기 위해 제한된 양의 표본 데이터를 이용하여 얻어진 경험적 분포를 사용하는 Sample Average Approximation (SAA)을 적용한다. 또한, 경험적 분포로부터 실제 분포가 Ambiguity Set라는 집합 내에서 벗어나도 리스크를 제한하는 방법을 제안한다. Ambiguity Set를 Wasserstein 거리로 측정된 반지름을 가진 통계적 공으로 선택함으로써 훈련 데이터와 독립적으로 생성된 새로운 샘플 데이터를 사용하여 평가한 out-of-sample risk에 대한 확률적 보장을 달성한다. 본 논문에서 SAA기반 MPC (SAA-MPC)와 Wasserstein Distributionally Robust MPC (DR-MPC)를 여러 과정을 통하여 다루기 쉬운 프로그램으로 재편성한다. 또한, 환경의 과거 관측으로부터 장애물의 미래 움직임을 예측하기 위해 Distributionally Robust MPC 방법을 Gaussian Process (GP)와 결합한다. 본 연구에서 개발되는 기법들의 성능을 비선형 자동차 모델과 선형화된 쿼드로터 모델을 이용한 시뮬레이션 연구를 통하여 분석한다.1 BACKGROUND AND OBJECTIVES 1 1.1 Motivation and Objectives 1 1.2 Research Contributions 2 1.3 Thesis Organization 3 2 RISK-AWARE MOTION PLANNING AND CONTROL USING CVAR-CONSTRAINED OPTIMIZATION 5 2.1 Introduction 5 2.2 System and Obstacle Models 8 2.3 CVaR-constrained Motion Planning and Control 10 2.3.1 Reference Trajectory Planning 10 2.3.2 Safety Risk 11 2.3.3 Risk-Constrained Model Predictive Control 13 2.3.4 Linearly Constrained Mixed Integer Convex Program 18 2.4 Numerical Experiments 20 2.4.1 Effect of Confidence Level 21 2.4.2 Effect of Sample Size 23 2.5 Conclusions 24 3 WASSERSTEIN DISTRIBUTIONALLY ROBUST MPC 28 3.1 Introduction 28 3.2 System and Obstacle Models 31 3.3 Wasserstein Distributionally Robust MPC 33 3.3.1 Distance to the Safe Region 36 3.3.2 Reformulation of Distributionally Robust Risk Constraint 38 3.3.3 Reformulation of the Wasserstein DR-MPC Problem 43 3.4 Out-of-Sample Performance Guarantee 45 3.5 Numerical Experiments 47 3.5.1 Nonlinear Car-Like Vehicle Model 48 3.5.2 Linearized Quadrotor Model 53 3.6 Conclusions 57 4 LEARNING-BASED DISTRIBUTIONALLY ROBUST MPC 58 4.1 Introduction 58 4.2 Learning the Movement of Obstacles Using Gaussian Processes 60 4.2.1 Obstacle Model 60 4.2.2 Gaussian Process Regression 61 4.2.3 Prediction of the Obstacle's Motion 63 4.3 Gaussian Process based Wasserstein DR-MPC 65 4.4 Numerical Experiments 70 4.5 Conclusions 74 5 CONCLUSIONS AND FUTURE WORK 75 Abstract (In Korean) 87Maste

Safety-Aware Optimal Control of Stochastic Systems Using Conditional Value-at-Risk

In this paper, we consider a multi-objective control problem for stochastic systems that seeks to minimize a cost of interest while ensuring safety. We introduce a novel measure of safety risk using the conditional value-at-risk and a set distance to formulate a safety risk-constrained optimal control problem. Our reformulation method using an extremal representation of the safety risk measure provides a computationally tractable dynamic programming solution. A useful byproduct of the proposed solution is the notion of a risk-constrained safe set, which is a new stochastic safety verification tool. We also establish useful connections between the risk-constrained safe sets and the popular probabilistic safe sets. The tradeoff between the risk tolerance and the mean performance of our controller is examined through an inventory control problem.N