166 research outputs found

    Linear Hamilton Jacobi Bellman Equations in High Dimensions

    Get PDF
    The Hamilton Jacobi Bellman Equation (HJB) provides the globally optimal solution to large classes of control problems. Unfortunately, this generality comes at a price, the calculation of such solutions is typically intractible for systems with more than moderate state space size due to the curse of dimensionality. This work combines recent results in the structure of the HJB, and its reduction to a linear Partial Differential Equation (PDE), with methods based on low rank tensor representations, known as a separated representations, to address the curse of dimensionality. The result is an algorithm to solve optimal control problems which scales linearly with the number of states in a system, and is applicable to systems that are nonlinear with stochastic forcing in finite-horizon, average cost, and first-exit settings. The method is demonstrated on inverted pendulum, VTOL aircraft, and quadcopter models, with system dimension two, six, and twelve respectively.Comment: 8 pages. Accepted to CDC 201

    Verification Theorems for Stochastic Optimal Control Problems via a Time Dependent Fukushima - Dirichlet Decomposition

    Get PDF
    This paper is devoted to present a method of proving verification theorems for stochastic optimal control of finite dimensional diffusion processes without control in the diffusion term. The value function is assumed to be continuous in time and once differentiable in the space variable (C0,1C^{0,1}) instead of once differentiable in time and twice in space (C1,2C^{1,2}), like in the classical results. The results are obtained using a time dependent Fukushima - Dirichlet decomposition proved in a companion paper by the same authors using stochastic calculus via regularization. Applications, examples and comparison with other similar results are also given.Comment: 34 pages. To appear: Stochastic Processes and Their Application

    On a parabolic Hamilton-Jacobi-Bellman equation degenerating at the boundary

    Get PDF
    We derive the long time asymptotic of solutions to an evolutive Hamilton-Jacobi-Bellman equation in a bounded smooth domain, in connection with ergodic problems recently studied in \cite{bcr}. Our main assumption is an appropriate degeneracy condition on the operator at the boundary. This condition is related to the characteristic boundary points for linear operators as well as to the irrelevant points for the generalized Dirichlet problem, and implies in particular that no boundary datum has to be imposed. We prove that there exists a constant cc such that the solutions of the evolutive problem converge uniformly, in the reference frame moving with constant velocity cc, to a unique steady state solving a suitable ergodic problem.Comment: 12p

    A neural network based policy iteration algorithm with global H2H^2-superlinear convergence for stochastic games on domains

    Full text link
    In this work, we propose a class of numerical schemes for solving semilinear Hamilton-Jacobi-Bellman-Isaacs (HJBI) boundary value problems which arise naturally from exit time problems of diffusion processes with controlled drift. We exploit policy iteration to reduce the semilinear problem into a sequence of linear Dirichlet problems, which are subsequently approximated by a multilayer feedforward neural network ansatz. We establish that the numerical solutions converge globally in the H2H^2-norm, and further demonstrate that this convergence is superlinear, by interpreting the algorithm as an inexact Newton iteration for the HJBI equation. Moreover, we construct the optimal feedback controls from the numerical value functions and deduce convergence. The numerical schemes and convergence results are then extended to HJBI boundary value problems corresponding to controlled diffusion processes with oblique boundary reflection. Numerical experiments on the stochastic Zermelo navigation problem are presented to illustrate the theoretical results and to demonstrate the effectiveness of the method.Comment: Additional numerical experiments have been included (on Pages 27-31) to show the proposed algorithm achieves a more stable and more rapid convergence than the existing neural network based methods within similar computational tim

    Bounded-From-Below Solutions of the Hamilton-Jacobi Equation for Optimal Control Problems with Exit Times: Vanishing Lagrangians, Eikonal Equations, and Shape-From-Shading

    Get PDF
    We study the Hamilton-Jacobi equation for undiscounted exit time control problems with general nonnegative Lagrangians using the dynamic programming approach. We prove theorems characterizing the value function as the unique bounded-from-below viscosity solution of the Hamilton-Jacobi equation which is null on the target. The result applies to problems with the property that all trajectories satisfying a certain integral condition must stay in a bounded set. We allow problems for which the Lagrangian is not uniformly bounded below by positive constants, in which the hypotheses of the known uniqueness results for Hamilton-Jacobi equations are not satisfied. We apply our theorems to eikonal equations from geometric optics, shape-from-shading equations from image processing, and variants of the Fuller Problem.Comment: 29 pages, 0 figures, accepted for publication in NoDEA Nonlinear Differential Equations and Applications on July 29, 200

    Lyapunov stabilizability of controlled diffusions via a superoptimality principle for viscosity solutions

    Full text link
    We prove optimality principles for semicontinuous bounded viscosity solutions of Hamilton-Jacobi-Bellman equations. In particular we provide a representation formula for viscosity supersolutions as value functions of suitable obstacle control problems. This result is applied to extend the Lyapunov direct method for stability to controlled Ito stochastic differential equations. We define the appropriate concept of Lyapunov function to study the stochastic open loop stabilizability in probability and the local and global asymptotic stabilizability (or asymptotic controllability). Finally we illustrate the theory with some examples.Comment: 22 page
    • …
    corecore