Linear Hamilton Jacobi Bellman Equations in High Dimensions

The Hamilton Jacobi Bellman Equation (HJB) provides the globally optimal solution to large classes of control problems. Unfortunately, this generality comes at a price, the calculation of such solutions is typically intractible for systems with more than moderate state space size due to the curse of dimensionality. This work combines recent results in the structure of the HJB, and its reduction to a linear Partial Differential Equation (PDE), with methods based on low rank tensor representations, known as a separated representations, to address the curse of dimensionality. The result is an algorithm to solve optimal control problems which scales linearly with the number of states in a system, and is applicable to systems that are nonlinear with stochastic forcing in finite-horizon, average cost, and first-exit settings. The method is demonstrated on inverted pendulum, VTOL aircraft, and quadcopter models, with system dimension two, six, and twelve respectively.Comment: 8 pages. Accepted to CDC 201

A Rotating-Grid Upwind Fast Sweeping Scheme for a Class of Hamilton-Jacobi Equations

We present a fast sweeping method for a class of Hamilton-Jacobi equations that arise from time-independent problems in optimal control theory. The basic method in two dimensions uses a four point stencil and is extremely simple to implement. We test our basic method against Eikonal equations in different norms, and then suggest a general method for rotating the grid and using additional approximations to the derivatives in different directions in order to more accurately capture characteristic flow. We display the utility of our method by applying it to relevant problems from engineering

High-order filtered schemes for time-dependent second order HJB equations

In this paper, we present and analyse a class of "filtered" numerical schemes for second order Hamilton-Jacobi-Bellman equations. Our approach follows the ideas introduced in B.D. Froese and A.M. Oberman, Convergent filtered schemes for the Monge-Amp\`ere partial differential equation, SIAM J. Numer. Anal., 51(1):423--444, 2013, and more recently applied by other authors to stationary or time-dependent first order Hamilton-Jacobi equations. For high order approximation schemes (where "high" stands for greater than one), the inevitable loss of monotonicity prevents the use of the classical theoretical results for convergence to viscosity solutions. The work introduces a suitable local modification of these schemes by "filtering" them with a monotone scheme, such that they can be proven convergent and still show an overall high order behaviour for smooth enough solutions. We give theoretical proofs of these claims and illustrate the behaviour with numerical tests from mathematical finance, focussing also on the use of backward difference formulae (BDF) for constructing the high order schemes.Comment: 27 pages, 16 figures, 4 table

Convergent finite difference methods for one-dimensional fully nonlinear second order partial differential equations

This paper develops a new framework for designing and analyzing convergent finite difference methods for approximating both classical and viscosity solutions of second order fully nonlinear partial differential equations (PDEs) in 1-D. The goal of the paper is to extend the successful framework of monotone, consistent, and stable finite difference methods for first order fully nonlinear Hamilton-Jacobi equations to second order fully nonlinear PDEs such as Monge-Amp\`ere and Bellman type equations. New concepts of consistency, generalized monotonicity, and stability are introduced; among them, the generalized monotonicity and consistency, which are easier to verify in practice, are natural extensions of the corresponding notions of finite difference methods for first order fully nonlinear Hamilton-Jacobi equations. The main component of the proposed framework is the concept of "numerical operator", and the main idea used to design consistent, monotone and stable finite difference methods is the concept of "numerical moment". These two new concepts play the same roles as the "numerical Hamiltonian" and the "numerical viscosity" play in the finite difference framework for first order fully nonlinear Hamilton-Jacobi equations. In the paper, two classes of consistent and monotone finite difference methods are proposed for second order fully nonlinear PDEs. The first class contains Lax-Friedrichs-like methods which also are proved to be stable and the second class contains Godunov-like methods. Numerical results are also presented to gauge the performance of the proposed finite difference methods and to validate the theoretical results of the paper.Comment: 23 pages, 8 figues, 11 table

Error estimates for a tree structure algorithm solving finite horizon control problems

In the Dynamic Programming approach to optimal control problems a crucial role is played by the value function that is characterized as the unique viscosity solution of a Hamilton-Jacobi-Bellman (HJB) equation. It is well known that this approach suffers of the "curse of dimensionality" and this limitation has reduced its practical in real world applications. Here we analyze a dynamic programming algorithm based on a tree structure. The tree is built by the time discrete dynamics avoiding in this way the use of a fixed space grid which is the bottleneck for high-dimensional problems, this also drops the projection on the grid in the approximation of the value function. We present some error estimates for a first order approximation based on the tree-structure algorithm. Moreover, we analyze a pruning technique for the tree to reduce the complexity and minimize the computational effort. Finally, we present some numerical tests