Polynomial approximation of high-dimensional Hamilton–Jacobi–Bellman equations and applications to feedback control of semilinear parabolic PDES

© 2018 Society for Industrial and Applied Mathematics. A procedure for the numerical approximation of high-dimensional Hamilton–Jacobi–Bellman (HJB) equations associated to optimal feedback control problems for semilinear parabolic equations is proposed. Its main ingredients are a pseudospectral collocation approximation of the PDE dynamics and an iterative method for the nonlinear HJB equation associated to the feedback synthesis. The latter is known as the successive Galerkin approximation. It can also be interpreted as Newton iteration for the HJB equation. At every step, the associated linear generalized HJB equation is approximated via a separable polynomial approximation ansatz. Stabilizing feedback controls are obtained from solutions to the HJB equations for systems of dimension up to fourteen

Piecewise Constant Policy Approximations to Hamilton-Jacobi-Bellman Equations

An advantageous feature of piecewise constant policy timestepping for Hamilton-Jacobi-Bellman (HJB) equations is that different linear approximation schemes, and indeed different meshes, can be used for the resulting linear equations for different control parameters. Standard convergence analysis suggests that monotone (i.e., linear) interpolation must be used to transfer data between meshes. Using the equivalence to a switching system and an adaptation of the usual arguments based on consistency, stability and monotonicity, we show that if limited, potentially higher order interpolation is used for the mesh transfer, convergence is guaranteed. We provide numerical tests for the mean-variance optimal investment problem and the uncertain volatility option pricing model, and compare the results to published test cases

High-order filtered schemes for time-dependent second order HJB equations

In this paper, we present and analyse a class of "filtered" numerical schemes for second order Hamilton-Jacobi-Bellman equations. Our approach follows the ideas introduced in B.D. Froese and A.M. Oberman, Convergent filtered schemes for the Monge-Amp\`ere partial differential equation, SIAM J. Numer. Anal., 51(1):423--444, 2013, and more recently applied by other authors to stationary or time-dependent first order Hamilton-Jacobi equations. For high order approximation schemes (where "high" stands for greater than one), the inevitable loss of monotonicity prevents the use of the classical theoretical results for convergence to viscosity solutions. The work introduces a suitable local modification of these schemes by "filtering" them with a monotone scheme, such that they can be proven convergent and still show an overall high order behaviour for smooth enough solutions. We give theoretical proofs of these claims and illustrate the behaviour with numerical tests from mathematical finance, focussing also on the use of backward difference formulae (BDF) for constructing the high order schemes.Comment: 27 pages, 16 figures, 4 table

Boundary control of parabolic PDE using adaptive dynamic programming

In this dissertation, novel adaptive/approximate dynamic programming (ADP) based state and output feedback control methods are presented for distributed parameter systems (DPS) which are expressed as uncertain parabolic partial differential equations (PDEs) in one and two dimensional domains. In the first step, the output feedback control design using an early lumping method is introduced after model reduction. Subsequently controllers were developed in four stages; Unlike current approaches in the literature, state and output feedback approaches were designed without utilizing model reduction for uncertain linear, coupled nonlinear and two-dimensional parabolic PDEs, respectively. In all of these techniques, the infinite horizon cost function was considered and controller design was obtained in a forward-in-time and online manner without solving the algebraic Riccati equation (ARE) or using value and policy iterations techniques. Providing the stability analysis in the original infinite dimensional domain was a major challenge. Using Lyapunov criterion, the ultimate boundedness (UB) result was demonstrated for the regulation of closed-loop system using all the techniques developed herein. Moreover, due to distributed and large scale nature of state space, pure state feedback control design for DPS has proven to be practically obsolete. Therefore, output feedback design using limited point sensors in the domain or at boundaries are introduced. In the final two papers, the developed state feedback ADP control method was extended to regulate multi-dimensional and more complicated nonlinear parabolic PDE dynamics --Abstract, page iv

Robust feedback control of nonlinear PDEs by numerical approximation of high-dimensional Hamilton-Jacobi-Isaacs equations

Copyright © by SIAM. We propose an approach for the synthesis of robust and optimal feedback controllers for nonlinear PDEs. Our approach considers the approximation of infinite-dimensional control systems by a pseudospectral collocation method, leading to high-dimensional nonlinear dynamics. For the reducedorder model, we construct a robust feedback control based on the H∞ control method, which requires the solution of an associated high-dimensional Hamilton-Jacobi-Isaacs nonlinear PDE. The dimensionality of the Isaacs PDE is tackled by means of a separable representation of the control system, and a polynomial approximation ansatz for the corresponding value function. Our method proves to be effective for the robust stabilization of nonlinear dynamics up to dimension d ≈ 12. We assess the robustness and optimality features of our design over a class of nonlinear parabolic PDEs, including nonlinear advection and reaction terms. The proposed design yields a feedback controller achieving optimal stabilization and disturbance rejection properties, along with providing a modeling framework for the robust control of PDEs under parametric uncertainties

Tensor Decomposition Methods for High-dimensional Hamilton--Jacobi--Bellman Equations

A tensor decomposition approach for the solution of high-dimensional, fully nonlin-ear Hamilton-Jacobi-Bellman equations arising in optimal feedback control of nonlinear dynamics is presented. The method combines a tensor train approximation for the value function together with a Newton-like iterative method for the solution of the resulting nonlinear system. The tensor approximation leads to a polynomial scaling with respect to the dimension, partially circumventing the curse of dimensionality. A convergence analysis for the linear-quadratic optimal control problem is presented. For nonlinear dynamics, the effectiveness of the high-dimensional control synthesis method is assessed in the optimal feedback stabilization of the Allen-Cahn and Fokker-Planck equations with a hundred of variables. 1. Introduction. Richard Bellman first coined the expression "curse of dimen-sionality" when referring to the overwhelming computational complexity associated to the solution of multi-stage decision processes through dynamic programming, what is nowadays known as Bellman's equation. More than 60 years down the road, the curse of dimensionality has become ubiquitous in different fields such as numerical analysis, compressed sensing and statistical machine learning. However, it is in the computation of optimal feedback policies for the control of dynamical systems where its meaning continues to be most evident. Here, the curse of dimensionality arises since the synthesis of optimal feedback laws by dynamic programming techniques demands the solution of a Hamilton-Jacobi-Bellman (HJB) fully nonlinear Partial Differential Equation (PDE) cast over the state space of the dynamics. This intrinsic relation between the dimensions of the state space of the control system and the domain of the HJB PDE generates computational challenges of formidable complexity even for relatively simple dynamical systems 1. Much of the research in control revolves around circumventing this barrier through different trade-offs between dimensional-ity, performance, and optimality of the control design. Prominent examples of the research landscape shaped by the curse of dimensionality include model order reduction , model predictive control, suboptimal feedback design, reinforcement learning and distributed control. However, the effective computational solution of dynamic programming equations of arbitrarily high dimensions through deterministic methods remains an open quest with fundamental implications in optimal control design

Boolean Delay Equations: A simple way of looking at complex systems

Boolean Delay Equations (BDEs) are semi-discrete dynamical models with Boolean-valued variables that evolve in continuous time. Systems of BDEs can be classified into conservative or dissipative, in a manner that parallels the classification of ordinary or partial differential equations. Solutions to certain conservative BDEs exhibit growth of complexity in time. They represent therewith metaphors for biological evolution or human history. Dissipative BDEs are structurally stable and exhibit multiple equilibria and limit cycles, as well as more complex, fractal solution sets, such as Devil's staircases and ``fractal sunbursts``. All known solutions of dissipative BDEs have stationary variance. BDE systems of this type, both free and forced, have been used as highly idealized models of climate change on interannual, interdecadal and paleoclimatic time scales. BDEs are also being used as flexible, highly efficient models of colliding cascades in earthquake modeling and prediction, as well as in genetics. In this paper we review the theory of systems of BDEs and illustrate their applications to climatic and solid earth problems. The former have used small systems of BDEs, while the latter have used large networks of BDEs. We moreover introduce BDEs with an infinite number of variables distributed in space (``partial BDEs``) and discuss connections with other types of dynamical systems, including cellular automata and Boolean networks. This research-and-review paper concludes with a set of open questions.Comment: Latex, 67 pages with 15 eps figures. Revised version, in particular the discussion on partial BDEs is updated and enlarge