Piecewise Constant Policy Approximations to Hamilton-Jacobi-Bellman Equations

An advantageous feature of piecewise constant policy timestepping for Hamilton-Jacobi-Bellman (HJB) equations is that different linear approximation schemes, and indeed different meshes, can be used for the resulting linear equations for different control parameters. Standard convergence analysis suggests that monotone (i.e., linear) interpolation must be used to transfer data between meshes. Using the equivalence to a switching system and an adaptation of the usual arguments based on consistency, stability and monotonicity, we show that if limited, potentially higher order interpolation is used for the mesh transfer, convergence is guaranteed. We provide numerical tests for the mean-variance optimal investment problem and the uncertain volatility option pricing model, and compare the results to published test cases

Boundary Treatment and Multigrid Preconditioning for Semi-Lagrangian Schemes Applied to Hamilton-Jacobi-Bellman Equations

We analyse two practical aspects that arise in the numerical solution of Hamilton-Jacobi-Bellman (HJB) equations by a particular class of monotone approximation schemes known as semi-Lagrangian schemes. These schemes make use of a wide stencil to achieve convergence and result in discretization matrices that are less sparse and less local than those coming from standard finite difference schemes. This leads to computational difficulties not encountered there. In particular, we consider the overstepping of the domain boundary and analyse the accuracy and stability of stencil truncation. This truncation imposes a stricter CFL condition for explicit schemes in the vicinity of boundaries than in the interior, such that implicit schemes become attractive. We then study the use of geometric, algebraic and aggregation-based multigrid preconditioners to solve the resulting discretised systems from implicit time stepping schemes efficiently. Finally, we illustrate the performance of these techniques numerically for benchmark test cases from the literature

An efficient method for multiobjective optimal control and optimal control subject to integral constraints

We introduce a new and efficient numerical method for multicriterion optimal control and single criterion optimal control under integral constraints. The approach is based on extending the state space to include information on a "budget" remaining to satisfy each constraint; the augmented Hamilton-Jacobi-Bellman PDE is then solved numerically. The efficiency of our approach hinges on the causality in that PDE, i.e., the monotonicity of characteristic curves in one of the newly added dimensions. A semi-Lagrangian "marching" method is used to approximate the discontinuous viscosity solution efficiently. We compare this to a recently introduced "weighted sum" based algorithm for the same problem. We illustrate our method using examples from flight path planning and robotic navigation in the presence of friendly and adversarial observers.Comment: The final version accepted by J. Comp. Math. : 41 pages, 14 figures. Since the previous version: typos fixed, formatting improved, one mistake in bibliography correcte

Can local single-pass methods solve any stationary Hamilton-Jacobi-Bellman equation?

The use of local single-pass methods (like, e.g., the Fast Marching method) has become popular in the solution of some Hamilton-Jacobi equations. The prototype of these equations is the eikonal equation, for which the methods can be applied saving CPU time and possibly memory allocation. Then, some natural questions arise: can local single-pass methods solve any Hamilton-Jacobi equation? If not, where the limit should be set? This paper tries to answer these questions. In order to give a complete picture, we present an overview of some fast methods available in literature and we briefly analyze their main features. We also introduce some numerical tools and provide several numerical tests which are intended to exhibit the limitations of the methods. We show that the construction of a local single-pass method for general Hamilton-Jacobi equations is very hard, if not impossible. Nevertheless, some special classes of problems can be actually solved, making local single-pass methods very useful from the practical point of view.Comment: 19 page