Optimal Uncertainty Quantification

We propose a rigorous framework for Uncertainty Quantification (UQ) in which the UQ objectives and the assumptions/information set are brought to the forefront. This framework, which we call \emph{Optimal Uncertainty Quantification} (OUQ), is based on the observation that, given a set of assumptions and information about the problem, there exist optimal bounds on uncertainties: these are obtained as values of well-defined optimization problems corresponding to extremizing probabilities of failure, or of deviations, subject to the constraints imposed by the scenarios compatible with the assumptions and information. In particular, this framework does not implicitly impose inappropriate assumptions, nor does it repudiate relevant information. Although OUQ optimization problems are extremely large, we show that under general conditions they have finite-dimensional reductions. As an application, we develop \emph{Optimal Concentration Inequalities} (OCI) of Hoeffding and McDiarmid type. Surprisingly, these results show that uncertainties in input parameters, which propagate to output uncertainties in the classical sensitivity analysis paradigm, may fail to do so if the transfer functions (or probability distributions) are imperfectly known. We show how, for hierarchical structures, this phenomenon may lead to the non-propagation of uncertainties or information across scales. In addition, a general algorithmic framework is developed for OUQ and is tested on the Caltech surrogate model for hypervelocity impact and on the seismic safety assessment of truss structures, suggesting the feasibility of the framework for important complex systems. The introduction of this paper provides both an overview of the paper and a self-contained mini-tutorial about basic concepts and issues of UQ.Comment: 90 pages. Accepted for publication in SIAM Review (Expository Research Papers). See SIAM Review for higher quality figure

Towards Machine Wald

The past century has seen a steady increase in the need of estimating and predicting complex systems and making (possibly critical) decisions with limited information. Although computers have made possible the numerical evaluation of sophisticated statistical models, these models are still designed \emph{by humans} because there is currently no known recipe or algorithm for dividing the design of a statistical model into a sequence of arithmetic operations. Indeed enabling computers to \emph{think} as \emph{humans} have the ability to do when faced with uncertainty is challenging in several major ways: (1) Finding optimal statistical models remains to be formulated as a well posed problem when information on the system of interest is incomplete and comes in the form of a complex combination of sample data, partial knowledge of constitutive relations and a limited description of the distribution of input random variables. (2) The space of admissible scenarios along with the space of relevant information, assumptions, and/or beliefs, tend to be infinite dimensional, whereas calculus on a computer is necessarily discrete and finite. With this purpose, this paper explores the foundations of a rigorous framework for the scientific computation of optimal statistical estimators/models and reviews their connections with Decision Theory, Machine Learning, Bayesian Inference, Stochastic Optimization, Robust Optimization, Optimal Uncertainty Quantification and Information Based Complexity.Comment: 37 page

Extremum-Seeking Guidance and Conic-Sector-Based Control of Aerospace Systems

This dissertation studies guidance and control of aerospace systems. Guidance algorithms are used to determine desired trajectories of systems, and in particular, this dissertation examines constrained extremum-seeking guidance. This type of guidance is part of a class of algorithms that drives a system to the maximum or minimum of a performance function, where the exact relation between the function's input and output is unknown. This dissertation abstracts the problem of extremum-seeking to constrained matrix manifolds. Working with a constrained matrix manifold necessitates mathematics other than the familiar tools of linear systems. The performance function is optimized on the manifold by estimating a gradient using a Kalman filter, which can be modified to accommodate a wide variety of constraints and can filter measurement noise. A gradient-based optimization technique is then used to determine the extremum of the performance function. The developed algorithms are applied to aircraft and spacecraft. Control algorithms determine which system inputs are required to drive the systems outputs to follow the trajectory given by guidance. Aerospace systems are typically nonlinear, which makes control more challenging. One approach to control nonlinear systems is linear parameter varying (LPV) control, where well-established linear control techniques are extended to nonlinear systems. Although LPV control techniques work quite well, they require an LPV model of a system. This model is often an approximation of the real nonlinear system to be controlled, and any stability and performance guarantees that are derived using the system approximation are usually void on the real system. A solution to this problem can be found using the Passivity Theorem and the Conic Sector Theorem, two input-output stability theories, to synthesize LPV controllers. These controllers guarantee closed-loop stability even in the presence of system approximation. Several control techniques are derived and implemented in simulation and experimentation, where it is shown that these new controllers are robust to plant uncertainty.PHDAerospace EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/143993/1/aexwalsh_1.pd

Projected Stochastic Gradients for Convex Constrained Problems in Hilbert Spaces

Convergence of a projected stochastic gradient algorithm is demonstrated for convex objective functionals with convex constraint sets in Hilbert spaces. In the convex case, the sequence of iterates ${u_n}$ converges weakly to a point in the set of minimizers with probability one. In the strongly convex case, the sequence converges strongly to the unique optimum with probability one. An application to a class of PDE constrained problems with a convex objective, convex constraint and random elliptic PDE constraints is shown. Theoretical results are demonstrated numerically.Comment: 28 page

Optimal Uncertainty Quantification

We propose a rigorous framework for Uncertainty Quantification (UQ) in which the UQ objectives and the assumptions/information set are brought to the forefront. This framework, which we call Optimal Uncertainty Quantification (OUQ), is based on the observation that, given a set of assumptions and information about the problem, there exist optimal bounds on uncertainties: these are obtained as extreme values of well-defined optimization problems corresponding to extremizing probabilities of failure, or of deviations, subject to the constraints imposed by the scenarios compatible with the assumptions and information. In particular, this framework does not implicitly impose inappropriate assumptions, nor does it repudiate relevant information. Although OUQ optimization problems are extremely large, we show that under general conditions, they have finite-dimensional reductions. As an application, we develop Optimal Concentration Inequalities (OCI) of Hoeffding and McDiarmid type. Surprisingly, contrary to the classical sensitivity analysis paradigm, these results show that uncertainties in input parameters do not necessarily propagate to output uncertainties. In addition, a general algorithmic framework is developed for OUQ and is tested on the Caltech surrogate model for hypervelocity impact, suggesting the feasibility of the framework for important complex systems

Adaptive Systems: History, Techniques, Problems, and Perspectives

We survey some of the rich history of control over the past century with a focus on the major milestones in adaptive systems. We review classic methods and examples in adaptive linear systems for both control and observation/identification. The focus is on linear plants to facilitate understanding, but we also provide the tools necessary for many classes of nonlinear systems. We discuss practical issues encountered in making these systems stable and robust with respect to additive and multiplicative uncertainties. We discuss various perspectives on adaptive systems and their role in various fields. Finally, we present some of the ongoing research and expose problems in the field of adaptive control

Efficient evaluation of expectations of functions of a L\'evy process and its extremum

We prove simple general formulas for expectations of functions of a L\'evy process and its running extremum. Under additional conditions, we derive analytical formulas using the Fourier/Laplace inversion and Wiener-Hopf factorization, and discuss efficient numerical methods for realization of these formulas. As applications, the cumulative probability distribution function of the process and its running maximum and the price of the option to exchange the power of a stock for its maximum are calculated. The most efficient numerical methods use the sinh-acceleration technique and simplified trapezoid rule. The program in Matlab running on a Mac with moderate characteristics achieves the precision E-7 and better in several milliseconds, and E-14 - in a fraction of a second