Ordered Risk Minimization: Learning More from Less Data

We consider the worst-case expectation of a permutation invariant ambiguity set of discrete distributions as a proxy-cost for data-driven expected risk minimization. For this framework, we coin the term ordered risk minimization to highlight how results from order statistics inspired the proxy-cost. Specifically, we show how such costs serve as point-wise high-confidence upper bounds of the expected risk. The confidence level can be determined tightly for any sample size. Conversely we also illustrate how to calibrate the size of the ambiguity set such that the high-confidence upper bound has some user specified confidence. This calibration procedure notably supports $\phi$-divergence based ambiguity sets. Numerical experiments then illustrate how the resulting scheme both generalizes better and is less sensitive to tuning parameters compared to the empirical risk minimization approach

On the exact feasibility of convex scenario programs with discarded constraints

We revisit the so-called sampling and discarding approach used to quantify the probability of constraint violation of a solution to convex scenario programs when some of the original samples are allowed to be discarded. Motivated by two scenario programs that possess analytic solutions and the fact that the existing bound for scenario programs with discarded constraints is not tight, we analyze a removal scheme that consists of a cascade of optimization problems, where at each step we remove a superset of the active constraints. By relying on results from compression learning theory, we show that such a removal scheme leads to less conservative bounds for the probability of constraint violation than the existing ones. We also show that the proposed bound is tight by characterizing a class of optimization problems that achieves the given upper bound. The performance improvement of the proposed methodology is illustrated by an example that involves a resource sharing linear program

Optimality conditions in control problems with random state constraints in probabilistic or almost-sure form

In this paper, we discuss optimality conditions for optimization problems involving random state constraints, which are modeled in probabilistic or almost sure form. While the latter can be understood as the limiting case of the former, the derivation of optimality conditions requires substantially different approaches. We apply them to a linear elliptic partial differential equation (PDE) with random inputs. In the probabilistic case, we rely on the spherical-radial decomposition of Gaussian random vectors in order to formulate fully explicit optimality conditions involving a spherical integral. In the almost sure case, we derive optimality conditions and compare them to a model based on robust constraints with respect to the (compact) support of the given distribution

Probable Domain Generalization via Quantile Risk Minimization

Domain generalization (DG) seeks predictors which perform well on unseen test distributions by leveraging data drawn from multiple related training distributions or domains. To achieve this, DG is commonly formulated as an average- or worst-case problem over the set of possible domains. However, predictors that perform well on average lack robustness while predictors that perform well in the worst case tend to be overly-conservative. To address this, we propose a new probabilistic framework for DG where the goal is to learn predictors that perform well with high probability. Our key idea is that distribution shifts seen during training should inform us of probable shifts at test time, which we realize by explicitly relating training and test domains as draws from the same underlying meta-distribution. To achieve probable DG, we propose a new optimization problem called Quantile Risk Minimization (QRM). By minimizing the $\alpha$-quantile of predictor's risk distribution over domains, QRM seeks predictors that perform well with probability $\alpha$. To solve QRM in practice, we propose the Empirical QRM (EQRM) algorithm and provide: (i) a generalization bound for EQRM; and (ii) the conditions under which EQRM recovers the causal predictor as $\alpha \to 1$. In our experiments, we introduce a more holistic quantile-focused evaluation protocol for DG and demonstrate that EQRM outperforms state-of-the-art baselines on datasets from WILDS and DomainBed.Comment: NeurIPS 2022 camera-ready (+ minor corrections

Power System Operations with Probabilistic Guarantees

This study is motivated by the fact that uncertainties from deepening penetration of renewable energy resources have posed critical challenges to the secure and reliable operations of future electrical grids. Among various tools for decision making in uncertain environments, this study focuses on chance-constrained optimization, which provides explicit probabilistic guarantees on the feasibility of optimal solutions. Although quite a few methods have been proposed to solve chance-constrained optimization problems, there is a lack of comprehensive review and comparative analysis of the proposed methods. In this work, we provide a detailed tutorial on existing algorithms and a survey of major theoretical results of chance-constrained optimization theory. Data-driven methods, which are not constrained by any specific distributions of the underlying uncertainties, are of particular interest. Built upon chance-constrained optimization, we propose a three-stage power system operation framework with probabilistic guarantees: (1) the optimal unit commitment in the operational planning stage; (2) the optimal reactive power dispatch to address the voltage security issue in the hours-ahead adjustment period; and (3) the secure and reliable power system operation under uncertainties in real time. In the day-ahead operational planning stage, we propose a chance-constrained SCUC (c-SCUC) framework, which ensures that the risk of violating constraints is within an acceptable threshold. Using the scenario approach, c-SCUC is reformulated to the scenario-based SCUC (s-SCUC) problem. By choosing an appropriate number of scenarios, we provide theoretical guarantees on the posterior risk level of the solution to s-SCUC. Inspired by the latest progress of the scenario approach on non-convex problems, we demonstrate the structural properties of general scenario problems and analyze the specific characteristics of s-SCUC. Those characteristics were exploited to benefit the scalability and computational performance of s-SCUC. In the adjustment period, this work first investigates the benefits of look-ahead coordination of both continuous-state and discrete-state reactive power support devices across multiple control areas. The conventional static optimal reactive power dispatch is extended to a “moving-horizon” type formulation for the consideration of spatial and temporal variations. The optimal reactive power dispatch problem is further enhanced with chance constraints by considering the uncertainties from both renewables and contingencies. This chance-constrained optimal reactive power dispatch (c-ORPD) formulation offers system operators an effective tool to schedule voltage support devices such that the system voltage security can be ensured with quantifiable level of risk. Security-constrained Economic Dispatch (SCED) lies at the center of real-time operation of power systems and modern electricity markets. It determines the most cost-efficient output levels of generators while keeping the real-time balance between supply and demand. In this study, we formulate and solve chance-constrained SCED (c-SCED), which ensures system security under uncertainties from renewables. The c-SCED problem also serves as a benchmark problem for a critical comparison of existing algorithms to solve chance-constrained optimization problems

On the Convexity of Level-sets of Probability Functions

In decision-making problems under uncertainty, probabilistic constraints are a valuable tool to express safety of decisions. They result from taking the probability measure of a given set of random inequalities depending on the decision vector. Even if the original set of inequalities is convex, this favourable property is not immediately transferred to the probabilistically constrained feasible set and may in particular depend on the chosen safety level. In this paper, we provide results guaranteeing the convexity of feasible sets to probabilistic constraints when the safety level is greater than a computable threshold. Our results extend all the existing ones and also cover the case where decision vectors belong to Banach spaces. The key idea in our approach is to reveal the level of underlying convexity in the nominal problem data (e.g., concavity of the probability function) by auxiliary transforming functions. We provide several examples illustrating our theoretical developments

COMBINED ROBUST OPTIMAL DESIGN, PATH AND MOTION PLANNING FOR UNMANNED AERIAL VEHICLE SYSTEMS SUBJECT TO UNCERTAINTY

Unmanned system performance depends heavily on both how the system is planned to be operated and the design of the unmanned system, both of which can be heavily impacted by uncertainty. This dissertation presents methods for simultaneously optimizing both of these aspects of an unmanned system when subject to uncertainty. This simultaneous optimization under uncertainty of unmanned system design and planning is demonstrated in the context of optimizing the design and flight path of an unmanned aerial vehicle (UAV) subject to an unknown set of wind conditions. This dissertation explores optimizing the path of the UAV down to the level of determining flight trajectories accounting for the UAVs dynamics (motion planning) while simultaneously optimizing design. Uncertainty is considered from the robust (no probability distribution known) standpoint, with the capability to account for a general set of uncertain parameters that affects the UAVs performance. New methods are investigated for solving motion planning problems for UAVs, which are applied to the problem of mitigating the risk posed by UAVs flying over inhabited areas. A new approach to solving robust optimization problems is developed, which uses a combination of random sampling and worst case analysis. The new robust optimization approach is shown to efficiently solve robust optimization problems, even when existing robust optimization methods would fail. A new approach for robust optimal motion planning that considers a “black-box” uncertainty model is developed based off the new robust optimization approach. The new robust motion planning approach is shown to perform better under uncertainty than methods which do not use a “black-box” uncertainty model. A new method is developed for solving design and path planning optimization problems for unmanned systems with discrete (graph-based) path representations, which is then extended to work on motion planning problems. This design and motion planning approach is used within the new robust optimization approach to solve a robust design and motion planning optimization problem for a UAV. Results are presented comparing these methods against a design study using a DOE, which show that the proposed methods can be less computationally expensive than existing methods for design and motion planning problems