Distributionally Robust Optimization for Sequential Decision Making

The distributionally robust Markov Decision Process (MDP) approach asks for a distributionally robust policy that achieves the maximal expected total reward under the most adversarial distribution of uncertain parameters. In this paper, we study distributionally robust MDPs where ambiguity sets for the uncertain parameters are of a format that can easily incorporate in its description the uncertainty's generalized moment as well as statistical distance information. In this way, we generalize existing works on distributionally robust MDP with generalized-moment-based and statistical-distance-based ambiguity sets to incorporate information from the former class such as moments and dispersions to the latter class that critically depends on empirical observations of the uncertain parameters. We show that, under this format of ambiguity sets, the resulting distributionally robust MDP remains tractable under mild technical conditions. To be more specific, a distributionally robust policy can be constructed by solving a sequence of one-stage convex optimization subproblems

The effect of expected income on individual migration decisions

economic models;migration;decision making;income distribution

Risk-Aware Wasserstein Distributionally Robust Control of Vessels in Natural Waterways

In the realm of maritime transportation, autonomous vessel navigation in natural inland waterways faces persistent challenges due to unpredictable natural factors. Existing scheduling algorithms fall short in handling these uncertainties, compromising both safety and efficiency. Moreover, these algorithms are primarily designed for non-autonomous vessels, leading to labor-intensive operations vulnerable to human error. To address these issues, this study proposes a risk-aware motion control approach for vessels that accounts for the dynamic and uncertain nature of tide islands in a distributionally robust manner. Specifically, a model predictive control method is employed to follow the reference trajectory in the time-space map while incorporating a risk constraint to prevent grounding accidents. To address uncertainties in tide islands, a novel modeling technique represents them as stochastic polytopes. Additionally, potential inaccuracies in waterway depth are addressed through a risk constraint that considers the worst-case uncertainty distribution within a Wasserstein ambiguity set around the empirical distribution. Using sensor data collected in the Guadalquivir River, we empirically demonstrate the performance of the proposed method through simulations on a vessel. As a result, the vessel successfully navigates the waterway while avoiding grounding accidents, even with a limited dataset of observations. This stands in contrast to existing non-robust controllers, highlighting the robustness and practical applicability of the proposed approach

학습 기반 자율시스템의 리스크를 고려하는 분포적 강인 최적화

학위논문 (석사) -- 서울대학교 대학원 : 공과대학 전기·정보공학부, 2020. 8. 양인순.In this thesis, a risk-aware motion control scheme is considered for autonomous systems to avoid randomly moving obstacles when the true probability distribution of uncertainty is unknown. We propose a novel model predictive control (MPC) method for motion planning and decision-making that systematically adjusts the safety and conservativeness in an environment with randomly moving obstacles. The key component is the Conditional Value-at-Risk (CVaR), employed to limit the safety risk in the MPC problem. Having the empirical distribution obtained using a limited amount of sample data, Sample Average Approximation (SAA) is applied to compute the safety risk. Furthermore, we propose a method, which limits the risk of unsafety even when the true distribution of the obstacles movements deviates, within an ambiguity set, from the empirical one. By choosing the ambiguity set as a statistical ball with its radius measured by the Wasserstein metric, we achieve a probabilistic guarantee of the out-of-sample risk, evaluated using new sample data generated independently of the training data. A set of reformulations are applied on both SAA-based MPC (SAA-MPC) and Wasserstein Distributionally Robust MPC (DR-MPC) to make them tractable. In addition, we combine the DR-MPC method with Gaussian Process (GP) to predict the future motion of the obstacles from past observations of the environment. The performance of the proposed methods is demonstrated and analyzed through simulation studies using a nonlinear vehicle model and a linearized quadrotor model.본 연구에서 자율 시스템이 알려지지 않은 확률 분포로 랜덤하게 움직이는 장애물을 피하기 위한 위험 인식을 고려하는 모션 제어 기법을 개발한다. 따라서 본 논문에서 안전성과 보수성을 체계적으로 조절하는 새로운 Model Predictive Control (MPC) 방법을 제안한다. 본 방벙의 핵심 요소는 MPC 문제의 안전성 리스크를 제한하는 Conditional Value-at-Risk (CVaR)라는 리스크 척도이다. 안전성 리스크를 계산하기 위해 제한된 양의 표본 데이터를 이용하여 얻어진 경험적 분포를 사용하는 Sample Average Approximation (SAA)을 적용한다. 또한, 경험적 분포로부터 실제 분포가 Ambiguity Set라는 집합 내에서 벗어나도 리스크를 제한하는 방법을 제안한다. Ambiguity Set를 Wasserstein 거리로 측정된 반지름을 가진 통계적 공으로 선택함으로써 훈련 데이터와 독립적으로 생성된 새로운 샘플 데이터를 사용하여 평가한 out-of-sample risk에 대한 확률적 보장을 달성한다. 본 논문에서 SAA기반 MPC (SAA-MPC)와 Wasserstein Distributionally Robust MPC (DR-MPC)를 여러 과정을 통하여 다루기 쉬운 프로그램으로 재편성한다. 또한, 환경의 과거 관측으로부터 장애물의 미래 움직임을 예측하기 위해 Distributionally Robust MPC 방법을 Gaussian Process (GP)와 결합한다. 본 연구에서 개발되는 기법들의 성능을 비선형 자동차 모델과 선형화된 쿼드로터 모델을 이용한 시뮬레이션 연구를 통하여 분석한다.1 BACKGROUND AND OBJECTIVES 1 1.1 Motivation and Objectives 1 1.2 Research Contributions 2 1.3 Thesis Organization 3 2 RISK-AWARE MOTION PLANNING AND CONTROL USING CVAR-CONSTRAINED OPTIMIZATION 5 2.1 Introduction 5 2.2 System and Obstacle Models 8 2.3 CVaR-constrained Motion Planning and Control 10 2.3.1 Reference Trajectory Planning 10 2.3.2 Safety Risk 11 2.3.3 Risk-Constrained Model Predictive Control 13 2.3.4 Linearly Constrained Mixed Integer Convex Program 18 2.4 Numerical Experiments 20 2.4.1 Effect of Confidence Level 21 2.4.2 Effect of Sample Size 23 2.5 Conclusions 24 3 WASSERSTEIN DISTRIBUTIONALLY ROBUST MPC 28 3.1 Introduction 28 3.2 System and Obstacle Models 31 3.3 Wasserstein Distributionally Robust MPC 33 3.3.1 Distance to the Safe Region 36 3.3.2 Reformulation of Distributionally Robust Risk Constraint 38 3.3.3 Reformulation of the Wasserstein DR-MPC Problem 43 3.4 Out-of-Sample Performance Guarantee 45 3.5 Numerical Experiments 47 3.5.1 Nonlinear Car-Like Vehicle Model 48 3.5.2 Linearized Quadrotor Model 53 3.6 Conclusions 57 4 LEARNING-BASED DISTRIBUTIONALLY ROBUST MPC 58 4.1 Introduction 58 4.2 Learning the Movement of Obstacles Using Gaussian Processes 60 4.2.1 Obstacle Model 60 4.2.2 Gaussian Process Regression 61 4.2.3 Prediction of the Obstacle's Motion 63 4.3 Gaussian Process based Wasserstein DR-MPC 65 4.4 Numerical Experiments 70 4.5 Conclusions 74 5 CONCLUSIONS AND FUTURE WORK 75 Abstract (In Korean) 87Maste