166 research outputs found
Linear Hamilton Jacobi Bellman Equations in High Dimensions
The Hamilton Jacobi Bellman Equation (HJB) provides the globally optimal
solution to large classes of control problems. Unfortunately, this generality
comes at a price, the calculation of such solutions is typically intractible
for systems with more than moderate state space size due to the curse of
dimensionality. This work combines recent results in the structure of the HJB,
and its reduction to a linear Partial Differential Equation (PDE), with methods
based on low rank tensor representations, known as a separated representations,
to address the curse of dimensionality. The result is an algorithm to solve
optimal control problems which scales linearly with the number of states in a
system, and is applicable to systems that are nonlinear with stochastic forcing
in finite-horizon, average cost, and first-exit settings. The method is
demonstrated on inverted pendulum, VTOL aircraft, and quadcopter models, with
system dimension two, six, and twelve respectively.Comment: 8 pages. Accepted to CDC 201
Verification Theorems for Stochastic Optimal Control Problems via a Time Dependent Fukushima - Dirichlet Decomposition
This paper is devoted to present a method of proving verification theorems
for stochastic optimal control of finite dimensional diffusion processes
without control in the diffusion term. The value function is assumed to be
continuous in time and once differentiable in the space variable ()
instead of once differentiable in time and twice in space (), like in
the classical results. The results are obtained using a time dependent
Fukushima - Dirichlet decomposition proved in a companion paper by the same
authors using stochastic calculus via regularization. Applications, examples
and comparison with other similar results are also given.Comment: 34 pages. To appear: Stochastic Processes and Their Application
On a parabolic Hamilton-Jacobi-Bellman equation degenerating at the boundary
We derive the long time asymptotic of solutions to an evolutive
Hamilton-Jacobi-Bellman equation in a bounded smooth domain, in connection with
ergodic problems recently studied in \cite{bcr}. Our main assumption is an
appropriate degeneracy condition on the operator at the boundary. This
condition is related to the characteristic boundary points for linear operators
as well as to the irrelevant points for the generalized Dirichlet problem, and
implies in particular that no boundary datum has to be imposed. We prove that
there exists a constant such that the solutions of the evolutive problem
converge uniformly, in the reference frame moving with constant velocity ,
to a unique steady state solving a suitable ergodic problem.Comment: 12p
A neural network based policy iteration algorithm with global -superlinear convergence for stochastic games on domains
In this work, we propose a class of numerical schemes for solving semilinear
Hamilton-Jacobi-Bellman-Isaacs (HJBI) boundary value problems which arise
naturally from exit time problems of diffusion processes with controlled drift.
We exploit policy iteration to reduce the semilinear problem into a sequence of
linear Dirichlet problems, which are subsequently approximated by a multilayer
feedforward neural network ansatz. We establish that the numerical solutions
converge globally in the -norm, and further demonstrate that this
convergence is superlinear, by interpreting the algorithm as an inexact Newton
iteration for the HJBI equation. Moreover, we construct the optimal feedback
controls from the numerical value functions and deduce convergence. The
numerical schemes and convergence results are then extended to HJBI boundary
value problems corresponding to controlled diffusion processes with oblique
boundary reflection. Numerical experiments on the stochastic Zermelo navigation
problem are presented to illustrate the theoretical results and to demonstrate
the effectiveness of the method.Comment: Additional numerical experiments have been included (on Pages 27-31)
to show the proposed algorithm achieves a more stable and more rapid
convergence than the existing neural network based methods within similar
computational tim
Bounded-From-Below Solutions of the Hamilton-Jacobi Equation for Optimal Control Problems with Exit Times: Vanishing Lagrangians, Eikonal Equations, and Shape-From-Shading
We study the Hamilton-Jacobi equation for undiscounted exit time control
problems with general nonnegative Lagrangians using the dynamic programming
approach. We prove theorems characterizing the value function as the unique
bounded-from-below viscosity solution of the Hamilton-Jacobi equation which is
null on the target. The result applies to problems with the property that all
trajectories satisfying a certain integral condition must stay in a bounded
set. We allow problems for which the Lagrangian is not uniformly bounded below
by positive constants, in which the hypotheses of the known uniqueness results
for Hamilton-Jacobi equations are not satisfied. We apply our theorems to
eikonal equations from geometric optics, shape-from-shading equations from
image processing, and variants of the Fuller Problem.Comment: 29 pages, 0 figures, accepted for publication in NoDEA Nonlinear
Differential Equations and Applications on July 29, 200
Lyapunov stabilizability of controlled diffusions via a superoptimality principle for viscosity solutions
We prove optimality principles for semicontinuous bounded viscosity solutions
of Hamilton-Jacobi-Bellman equations. In particular we provide a representation
formula for viscosity supersolutions as value functions of suitable obstacle
control problems. This result is applied to extend the Lyapunov direct method
for stability to controlled Ito stochastic differential equations. We define
the appropriate concept of Lyapunov function to study the stochastic open loop
stabilizability in probability and the local and global asymptotic
stabilizability (or asymptotic controllability). Finally we illustrate the
theory with some examples.Comment: 22 page
- …