Optimization of Complex Systems in the Presence of Uncertainty and Approximations

Engineering decisions are invariably made under substantial uncertainty about current and future system cost and response, including cost and response associated with low-probability but high-consequence events. Such events motivate approaches that typically have centered on constraining or minimizing probability of failure, in contrast to the risk-neutral approach of constraining or minimizing expected values. The research under this proposal has, instead, developed concepts of risk-averse decision making between these extremes with the aim of achieving an advanced methodology better able to deal with risks and reliability in engineering design. Measures of risk that go beyond statistical quantiles to so-called superquantiles (CVaR) and their mixtures have been the main focus. The results have explored their superior properties and enhanced computability along with surprising implications that standard least-squares regression in statistical approximations might better be supplanted by generalizations like quantile and even superquantile regression. Superquantile regression, which provides a cautious and powerful tool, is completely new. It is entirely a product of this grant research.The research was a collaborative effort with Johannes Royset of the Naval Postgraduate School, who had separate funding from AFOSR.Air Force Office of Science and ResearchDistribution A - Approved for public releas

Geodesic PCA in the Wasserstein space

We introduce the method of Geodesic Principal Component Analysis (GPCA) on the space of probability measures on the line, with finite second moment, endowed with the Wasserstein metric. We discuss the advantages of this approach, over a standard functional PCA of probability densities in the Hilbert space of square-integrable functions. We establish the consistency of the method by showing that the empirical GPCA converges to its population counterpart, as the sample size tends to infinity. A key property in the study of GPCA is the isometry between the Wasserstein space and a closed convex subset of the space of square-integrable functions, with respect to an appropriate measure. Therefore, we consider the general problem of PCA in a closed convex subset of a separable Hilbert space, which serves as basis for the analysis of GPCA and also has interest in its own right. We provide illustrative examples on simple statistical models, to show the benefits of this approach for data analysis. The method is also applied to a real dataset of population pyramids

Chance constrained problems: a bilevel convex optimization perspective

Chance constraints are a valuable tool for the design of safe decisions in uncertain environments; they are used to model satisfaction of a constraint with a target probability. However, because of possible non-convexity and non-smoothness, optimizing over a chance constrained set is challenging. In this paper, we establish an exact reformulation of chance constrained problems as a bilevel problems with convex lower-levels. We then derive a tractable penalty approach, where the penalized objective is a difference-of-convex function that we minimize with a suitable bundle algorithm. We release an easy-to-use open-source python toolbox implementing the approach, with a special emphasis on fast computational subroutines

Assessing Risk of Exceedance Events with Buffered Probability of Exceedance and Superquantiles

Many systems, or components of systems, often have associated with them a critical safety threshold. If events then occur that have magnitude larger than such a threshold, such as a weather event or physical stress on a component, critical failures become likely and/or overall system integrity can become critically compromised. Therefore, regulations and risk assessments are often formulated mathematically in terms of Probability of Exceedance (POE). This characteristic, however, can hide important information about the frequency, magnitude, and overall risk of exceedance events. This includes the magnitude of events that do exceed the threshold. Additionally, the frequency and magnitude of near-exceedance events, just below the threshold, are ignored entirely. We overview a new probabilistic characterization of exceedance risk call Buffered Probability of Exceedance (bPOE), also reviewing a closely related concept called the superquantile. We show that bPOE simultaneously assess both the frequency and magnitude of both exceedance events and near-exceedance events. After introducing bPOE and superquantiles, we show how it can be viewed as superior to POE as a measure of exceedance risk. We then present a simple parametric distribution fitting procedure that utilizes bPOE and the superquantile, two characteristics that we see are advantageous to consider when estimation of exceedance risk and tail density are the focus of the fitting procedure

Set-Convergence and Its Application: A Tutorial

Optimization problems, generalized equations, and the multitude of other variational problems invariably lead to the analysis of sets and set-valued mappings as well as their approximations. We review the central concept of set-convergence and explain its role in defining a notion of proximity between sets, especially for epigraphs of functions and graphs of set-valued mappings. The development leads to an approximation theory for optimization problems and generalized equations with profound consequences for the construction of algorithms. We also introduce the role of set-convergence in variational geometry and subdifferentiability with applications to optimality conditions. Examples illustrate the importance of set-convergence in stability analysis, error analysis, construction of algorithms, statistical estimation, and probability theory

Calculating CVaR and bPOE for Common Probability Distributions With Application to Portfolio Optimization and Density Estimation

Conditional Value-at-Risk (CVaR) and Value-at-Risk (VaR), also called the superquantile and quantile, are frequently used to characterize the tails of probability distribution's and are popular measures of risk. Buffered Probability of Exceedance (bPOE) is a recently introduced characterization of the tail which is the inverse of CVaR, much like the CDF is the inverse of the quantile. These quantities can prove very useful as the basis for a variety of risk-averse parametric engineering approaches. Their use, however, is often made difficult by the lack of well-known closed-form equations for calculating these quantities for commonly used probability distribution's. In this paper, we derive formulas for the superquantile and bPOE for a variety of common univariate probability distribution's. Besides providing a useful collection within a single reference, we use these formulas to incorporate the superquantile and bPOE into parametric procedures. In particular, we consider two: portfolio optimization and density estimation. First, when portfolio returns are assumed to follow particular distribution families, we show that finding the optimal portfolio via minimization of bPOE has advantages over superquantile minimization. We show that, given a fixed threshold, a single portfolio is the minimal bPOE portfolio for an entire class of distribution's simultaneously. Second, we apply our formulas to parametric density estimation and propose the method of superquantile's (MOS), a simple variation of the method of moment's (MM) where moment's are replaced by superquantile's at different confidence levels. With the freedom to select various combinations of confidence levels, MOS allows the user to focus the fitting procedure on different portions of the distribution, such as the tail when fitting heavy-tailed asymmetric data.Comment: Fixed typo in Proposition 5 (changed - to +) and added referenc

Stochastic Optimization for Spectral Risk Measures

Spectral risk objectives - also called $L$-risks - allow for learning systems to interpolate between optimizing average-case performance (as in empirical risk minimization) and worst-case performance on a task. We develop stochastic algorithms to optimize these quantities by characterizing their subdifferential and addressing challenges such as biasedness of subgradient estimates and non-smoothness of the objective. We show theoretically and experimentally that out-of-the-box approaches such as stochastic subgradient and dual averaging are hindered by bias and that our approach outperforms them