학습 기반 자율시스템의 리스크를 고려하는 분포적 강인 최적화

학위논문 (석사) -- 서울대학교 대학원 : 공과대학 전기·정보공학부, 2020. 8. 양인순.In this thesis, a risk-aware motion control scheme is considered for autonomous systems to avoid randomly moving obstacles when the true probability distribution of uncertainty is unknown. We propose a novel model predictive control (MPC) method for motion planning and decision-making that systematically adjusts the safety and conservativeness in an environment with randomly moving obstacles. The key component is the Conditional Value-at-Risk (CVaR), employed to limit the safety risk in the MPC problem. Having the empirical distribution obtained using a limited amount of sample data, Sample Average Approximation (SAA) is applied to compute the safety risk. Furthermore, we propose a method, which limits the risk of unsafety even when the true distribution of the obstacles movements deviates, within an ambiguity set, from the empirical one. By choosing the ambiguity set as a statistical ball with its radius measured by the Wasserstein metric, we achieve a probabilistic guarantee of the out-of-sample risk, evaluated using new sample data generated independently of the training data. A set of reformulations are applied on both SAA-based MPC (SAA-MPC) and Wasserstein Distributionally Robust MPC (DR-MPC) to make them tractable. In addition, we combine the DR-MPC method with Gaussian Process (GP) to predict the future motion of the obstacles from past observations of the environment. The performance of the proposed methods is demonstrated and analyzed through simulation studies using a nonlinear vehicle model and a linearized quadrotor model.본 연구에서 자율 시스템이 알려지지 않은 확률 분포로 랜덤하게 움직이는 장애물을 피하기 위한 위험 인식을 고려하는 모션 제어 기법을 개발한다. 따라서 본 논문에서 안전성과 보수성을 체계적으로 조절하는 새로운 Model Predictive Control (MPC) 방법을 제안한다. 본 방벙의 핵심 요소는 MPC 문제의 안전성 리스크를 제한하는 Conditional Value-at-Risk (CVaR)라는 리스크 척도이다. 안전성 리스크를 계산하기 위해 제한된 양의 표본 데이터를 이용하여 얻어진 경험적 분포를 사용하는 Sample Average Approximation (SAA)을 적용한다. 또한, 경험적 분포로부터 실제 분포가 Ambiguity Set라는 집합 내에서 벗어나도 리스크를 제한하는 방법을 제안한다. Ambiguity Set를 Wasserstein 거리로 측정된 반지름을 가진 통계적 공으로 선택함으로써 훈련 데이터와 독립적으로 생성된 새로운 샘플 데이터를 사용하여 평가한 out-of-sample risk에 대한 확률적 보장을 달성한다. 본 논문에서 SAA기반 MPC (SAA-MPC)와 Wasserstein Distributionally Robust MPC (DR-MPC)를 여러 과정을 통하여 다루기 쉬운 프로그램으로 재편성한다. 또한, 환경의 과거 관측으로부터 장애물의 미래 움직임을 예측하기 위해 Distributionally Robust MPC 방법을 Gaussian Process (GP)와 결합한다. 본 연구에서 개발되는 기법들의 성능을 비선형 자동차 모델과 선형화된 쿼드로터 모델을 이용한 시뮬레이션 연구를 통하여 분석한다.1 BACKGROUND AND OBJECTIVES 1 1.1 Motivation and Objectives 1 1.2 Research Contributions 2 1.3 Thesis Organization 3 2 RISK-AWARE MOTION PLANNING AND CONTROL USING CVAR-CONSTRAINED OPTIMIZATION 5 2.1 Introduction 5 2.2 System and Obstacle Models 8 2.3 CVaR-constrained Motion Planning and Control 10 2.3.1 Reference Trajectory Planning 10 2.3.2 Safety Risk 11 2.3.3 Risk-Constrained Model Predictive Control 13 2.3.4 Linearly Constrained Mixed Integer Convex Program 18 2.4 Numerical Experiments 20 2.4.1 Effect of Confidence Level 21 2.4.2 Effect of Sample Size 23 2.5 Conclusions 24 3 WASSERSTEIN DISTRIBUTIONALLY ROBUST MPC 28 3.1 Introduction 28 3.2 System and Obstacle Models 31 3.3 Wasserstein Distributionally Robust MPC 33 3.3.1 Distance to the Safe Region 36 3.3.2 Reformulation of Distributionally Robust Risk Constraint 38 3.3.3 Reformulation of the Wasserstein DR-MPC Problem 43 3.4 Out-of-Sample Performance Guarantee 45 3.5 Numerical Experiments 47 3.5.1 Nonlinear Car-Like Vehicle Model 48 3.5.2 Linearized Quadrotor Model 53 3.6 Conclusions 57 4 LEARNING-BASED DISTRIBUTIONALLY ROBUST MPC 58 4.1 Introduction 58 4.2 Learning the Movement of Obstacles Using Gaussian Processes 60 4.2.1 Obstacle Model 60 4.2.2 Gaussian Process Regression 61 4.2.3 Prediction of the Obstacle's Motion 63 4.3 Gaussian Process based Wasserstein DR-MPC 65 4.4 Numerical Experiments 70 4.5 Conclusions 74 5 CONCLUSIONS AND FUTURE WORK 75 Abstract (In Korean) 87Maste

Infinite horizon average cost dynamic programming subject to total variation distance ambiguity

We analyze the per unit-time infinite horizon average cost Markov control model, subject to a total variation distance ambiguity on the controlled process conditional distribution. This stochastic optimal control problem is formulated as a minimax optimization problem in which the minimization is over the admissible set of control strategies, while the maximization is over the set of conditional distributions which are in a ball, with respect to the total variation distance, centered at a nominal distribution. We derive two new equivalent dynamic programming equations, and a new policy iteration algorithm. The main feature of the new dynamic programming equations is that the optimal control strategies are insensitive to inaccuracies or ambiguities in the controlled process conditional distribution. The main feature of the new policy iteration algorithm is that the policy evaluation and policy improvement steps are performed using the maximizing conditional distribution, which is obtained via a water filling solution of aggregating states together to form new states. Throughout the paper, we illustrate the new dynamic programming equations and the corresponding policy iteration algorithm to various examples.Peer reviewe