Approximate Solutions To Constrained Risk-Sensitive Markov Decision Processes

This paper considers the problem of finding near-optimal Markovian randomized (MR) policies for finite-state-action, infinite-horizon, constrained risk-sensitive Markov decision processes (CRSMDPs). Constraints are in the form of standard expected discounted cost functions as well as expected risk-sensitive discounted cost functions over finite and infinite horizons. The main contribution is to show that the problem possesses a solution if it is feasible, and to provide two methods for finding an approximate solution in the form of an ultimately stationary (US) MR policy. The latter is achieved through two approximating finite-horizon CRSMDPs which are constructed from the original CRSMDP by time-truncating the original objective and constraint cost functions, and suitably perturbing the constraint upper bounds. The first approximation gives a US policy which is $\epsilon$-optimal and feasible for the original problem, while the second approximation gives a near-optimal US policy whose violation of the original constraints is bounded above by a specified $\epsilon$. A key step in the proofs is an appropriate choice of a metric that makes the set of infinite-horizon MR policies and the feasible regions of the three CRSMDPs compact, and the objective and constraint functions continuous. A linear-programming-based formulation for solving the approximating finite-horizon CRSMDPs is also given.Comment: 38 page

A method for pricing American options using semi-infinite linear programming

We introduce a new approach for the numerical pricing of American options. The main idea is to choose a finite number of suitable excessive functions (randomly) and to find the smallest majorant of the gain function in the span of these functions. The resulting problem is a linear semi-infinite programming problem, that can be solved using standard algorithms. This leads to good upper bounds for the original problem. For our algorithms no discretization of space and time and no simulation is necessary. Furthermore it is applicable even for high-dimensional problems. The algorithm provides an approximation of the value not only for one starting point, but for the complete value function on the continuation set, so that the optimal exercise region and e.g. the Greeks can be calculated. We apply the algorithm to (one- and) multidimensional diffusions and to L\'evy processes, and show it to be fast and accurate

Adaptive Horizon Model Predictive Control and Al'brekht's Method

A standard way of finding a feedback law that stabilizes a control system to an operating point is to recast the problem as an infinite horizon optimal control problem. If the optimal cost and the optmal feedback can be found on a large domain around the operating point then a Lyapunov argument can be used to verify the asymptotic stability of the closed loop dynamics. The problem with this approach is that is usually very difficult to find the optimal cost and the optmal feedback on a large domain for nonlinear problems with or without constraints. Hence the increasing interest in Model Predictive Control (MPC). In standard MPC a finite horizon optimal control problem is solved in real time but just at the current state, the first control action is implimented, the system evolves one time step and the process is repeated. A terminal cost and terminal feedback found by Al'brekht's methoddefined in a neighborhood of the operating point is used to shorten the horizon and thereby make the nonlinear programs easier to solve because they have less decision variables. Adaptive Horizon Model Predictive Control (AHMPC) is a scheme for varying the horizon length of Model Predictive Control (MPC) as needed. Its goal is to achieve stabilization with horizons as small as possible so that MPC methods can be used on faster and/or more complicated dynamic processes.Comment: arXiv admin note: text overlap with arXiv:1602.0861

Representation of homothetic forward performance processes in stochastic factor models via ergodic and infinite horizon BSDE

In an incomplete market, with incompleteness stemming from stochastic factors imperfectly correlated with the underlying stocks, we derive representations of homothetic (power, exponential and logarithmic) forward performance processes in factor-form using ergodic BSDE. We also develop a connection between the forward processes and infinite horizon BSDE, and, moreover, with risk-sensitive optimization. In addition, we develop a connection, for large time horizons, with a family of classical homothetic value function processes with random endowments.Comment: 34 page

Analysis of unconstrained nonlinear MPC schemes with time varying control horizon

For discrete time nonlinear systems satisfying an exponential or finite time controllability assumption, we present an analytical formula for a suboptimality estimate for model predictive control schemes without stabilizing terminal constraints. Based on our formula, we perform a detailed analysis of the impact of the optimization horizon and the possibly time varying control horizon on stability and performance of the closed loop

A receding horizon generalization of pointwise min-norm controllers

Control Lyapunov functions (CLFs) are used in conjunction with receding horizon control to develop a new class of receding horizon control schemes. In the process, strong connections between the seemingly disparate approaches are revealed, leading to a unified picture that ties together the notions of pointwise min-norm, receding horizon, and optimal control. This framework is used to develop a CLF based receding horizon scheme, of which a special case provides an appropriate extension of Sontag's formula. The scheme is first presented as an idealized continuous-time receding horizon control law. The issue of implementation under discrete-time sampling is then discussed as a modification. These schemes are shown to possess a number of desirable theoretical and implementation properties. An example is provided, demonstrating their application to a nonlinear control problem. Finally, stronger connections to both optimal and pointwise min-norm control are proved