Optimization of Gaussian Random Fields

Many engineering systems are subject to spatially distributed uncertainty, i.e. uncertainty that can be modeled as a random field. Altering the mean or covariance of this uncertainty will in general change the statistical distribution of the system outputs. We present an approach for computing the sensitivity of the statistics of system outputs with respect to the parameters describing the mean and covariance of the distributed uncertainty. This sensitivity information is then incorporated into a gradient-based optimizer to optimize the structure of the distributed uncertainty to achieve desired output statistics. This framework is applied to perform variance optimization for a model problem and to optimize the manufacturing tolerances of a gas turbine compressor blade

Polynomial diffusions on compact quadric sets

Polynomial processes are defined by the property that conditional expectations of polynomial functions of the process are again polynomials of the same or lower degree. Many fundamental stochastic processes, including affine processes, are polynomial, and their tractable structure makes them important in applications. In this paper we study polynomial diffusions whose state space is a compact quadric set. Necessary and sufficient conditions for existence, uniqueness, and boundary attainment are given. The existence of a convenient parameterization of the generator is shown to be closely related to the classical problem of expressing nonnegative polynomials---specifically, biquadratic forms vanishing on the diagonal---as a sum of squares. We prove that in dimension $d\le 4$ every such biquadratic form is a sum of squares, while for $d\ge6$ there are counterexamples. The case $d=5$ remains open. An equivalent probabilistic description of the sum of squares property is provided, and we show how it can be used to obtain results on pathwise uniqueness and existence of smooth densities.Comment: Forthcoming in Stochastic Processes and their Application

Approximate dynamic programming for large scale systems

Sequential decision making under uncertainty is at the heart of a wide variety of practical problems. These problems can be cast as dynamic programs and the optimal value function can be computed by solving Bellman's equation. However, this approach is limited in its applicability. As the number of state variables increases, the state space size grows exponentially, a phenomenon known as the curse of dimensionality, rendering the standard dynamic programming approach impractical. An effective way of addressing curse of dimensionality is through parameterized value function approximation. Such an approximation is determined by relatively small number of parameters and serves as an estimate of the optimal value function. But in order for this approach to be effective, we need Approximate Dynamic Programming (ADP) algorithms that can deliver `good' approximation to the optimal value function and such an approximation can then be used to derive policies for effective decision-making. From a practical standpoint, in order to assess the effectiveness of such an approximation, there is also a need for methods that give a sense for the suboptimality of a policy. This thesis is an attempt to address both these issues. First, we introduce a new ADP algorithm based on linear programming, to compute value function approximations. LP approaches to approximate DP have typically relied on a natural `projection' of a well studied linear program for exact dynamic programming. Such programs restrict attention to approximations that are lower bounds to the optimal cost-to-go function. Our program -- the `smoothed approximate linear program' -- is distinct from such approaches and relaxes the restriction to lower bounding approximations in an appropriate fashion while remaining computationally tractable. The resulting program enjoys strong approximation guarantees and is shown to perform well in numerical experiments with the game of Tetris and queueing network control problem. Next, we consider optimal stopping problems with applications to pricing of high-dimensional American options. We introduce the pathwise optimization (PO) method: a new convex optimization procedure to produce upper and lower bounds on the optimal value (the `price') of high-dimensional optimal stopping problems. The PO method builds on a dual characterization of optimal stopping problems as optimization problems over the space of martingales, which we dub the martingale duality approach. We demonstrate via numerical experiments that the PO method produces upper bounds and lower bounds (via suboptimal exercise policies) of a quality comparable with state-of-the-art approaches. Further, we develop an approximation theory relevant to martingale duality approaches in general and the PO method in particular. Finally, we consider a broad class of MDPs and introduce a new tractable method for computing bounds by consider information relaxation and introducing penalty. The method delivers tight bounds by identifying the best penalty function among a parameterized class of penalty functions. We implement our method on a high-dimensional financial application, namely, optimal execution and demonstrate the practical value of the method vis-a-vis competing methods available in the literature. In addition, we provide theory to show that bounds generated by our method are provably tighter than some of the other available approaches

A primal-dual algorithm for BSDEs

We generalize the primal-dual methodology, which is popular in the pricing of early-exercise options, to a backward dynamic programming equation associated with time discretization schemes of (reflected) backward stochastic differential equations (BSDEs). Taking as an input some approximate solution of the backward dynamic program, which was pre-computed, e.g., by least-squares Monte Carlo, our methodology allows to construct a confidence interval for the unknown true solution of the time discretized (reflected) BSDE at time 0. We numerically demonstrate the practical applicability of our method in two five-dimensional nonlinear pricing problems where tight price bounds were previously unavailable

A central limit theorem for temporally non-homogenous Markov chains with applications to dynamic programming

We prove a central limit theorem for a class of additive processes that arise naturally in the theory of finite horizon Markov decision problems. The main theorem generalizes a classic result of Dobrushin (1956) for temporally non-homogeneous Markov chains, and the principal innovation is that here the summands are permitted to depend on both the current state and a bounded number of future states of the chain. We show through several examples that this added flexibility gives one a direct path to asymptotic normality of the optimal total reward of finite horizon Markov decision problems. The same examples also explain why such results are not easily obtained by alternative Markovian techniques such as enlargement of the state space.Comment: 27 pages, 1 figur

Bounds for Markov Decision Processes

We consider the problem of producing lower bounds on the optimal cost-to-go function of a Markov decision problem. We present two approaches to this problem: one based on the methodology of approximate linear programming (ALP) and another based on the so-called martingale duality approach. We show that these two approaches are intimately connected. Exploring this connection leads us to the problem of finding "optimal" martingale penalties within the martingale duality approach which we dub the pathwise optimization (PO) problem. We show interesting cases where the PO problem admits a tractable solution and establish that these solutions produce tighter approximations than the ALP approach. © 2013 The Institute of Electrical and Electronics Engineers, Inc