655 research outputs found
An Efficient Policy Iteration Algorithm for Dynamic Programming Equations
We present an accelerated algorithm for the solution of static
Hamilton-Jacobi-Bellman equations related to optimal control problems. Our
scheme is based on a classic policy iteration procedure, which is known to have
superlinear convergence in many relevant cases provided the initial guess is
sufficiently close to the solution. In many cases, this limitation degenerates
into a behavior similar to a value iteration method, with an increased
computation time. The new scheme circumvents this problem by combining the
advantages of both algorithms with an efficient coupling. The method starts
with a value iteration phase and then switches to a policy iteration procedure
when a certain error threshold is reached. A delicate point is to determine
this threshold in order to avoid cumbersome computation with the value
iteration and, at the same time, to be reasonably sure that the policy
iteration method will finally converge to the optimal solution. We analyze the
methods and efficient coupling in a number of examples in dimension two, three
and four illustrating its properties
An adaptive POD approximation method for the control of advection-diffusion equations
We present an algorithm for the approximation of a finite horizon optimal
control problem for advection-diffusion equations. The method is based on the
coupling between an adaptive POD representation of the solution and a Dynamic
Programming approximation scheme for the corresponding evolutive
Hamilton-Jacobi equation. We discuss several features regarding the adaptivity
of the method, the role of error estimate indicators to choose a time
subdivision of the problem and the computation of the basis functions. Some
test problems are presented to illustrate the method.Comment: 17 pages, 18 figure
Error estimates for a tree structure algorithm solving finite horizon control problems
In the Dynamic Programming approach to optimal control problems a crucial
role is played by the value function that is characterized as the unique
viscosity solution of a Hamilton-Jacobi-Bellman (HJB) equation. It is well
known that this approach suffers of the "curse of dimensionality" and this
limitation has reduced its practical in real world applications. Here we
analyze a dynamic programming algorithm based on a tree structure. The tree is
built by the time discrete dynamics avoiding in this way the use of a fixed
space grid which is the bottleneck for high-dimensional problems, this also
drops the projection on the grid in the approximation of the value function. We
present some error estimates for a first order approximation based on the
tree-structure algorithm. Moreover, we analyze a pruning technique for the tree
to reduce the complexity and minimize the computational effort. Finally, we
present some numerical tests
Linear Hamilton Jacobi Bellman Equations in High Dimensions
The Hamilton Jacobi Bellman Equation (HJB) provides the globally optimal
solution to large classes of control problems. Unfortunately, this generality
comes at a price, the calculation of such solutions is typically intractible
for systems with more than moderate state space size due to the curse of
dimensionality. This work combines recent results in the structure of the HJB,
and its reduction to a linear Partial Differential Equation (PDE), with methods
based on low rank tensor representations, known as a separated representations,
to address the curse of dimensionality. The result is an algorithm to solve
optimal control problems which scales linearly with the number of states in a
system, and is applicable to systems that are nonlinear with stochastic forcing
in finite-horizon, average cost, and first-exit settings. The method is
demonstrated on inverted pendulum, VTOL aircraft, and quadcopter models, with
system dimension two, six, and twelve respectively.Comment: 8 pages. Accepted to CDC 201
A HJB-POD approach for the control of nonlinear PDEs on a tree structure
The Dynamic Programming approach allows to compute a feedback control for
nonlinear problems, but suffers from the curse of dimensionality. The
computation of the control relies on the resolution of a nonlinear PDE, the
Hamilton-Jacobi-Bellman equation, with the same dimension of the original
problem. Recently, a new numerical method to compute the value function on a
tree structure has been introduced. The method allows to work without a
structured grid and avoids any interpolation.
Here, we aim to test the algorithm for nonlinear two dimensional PDEs. We
apply model order reduction to decrease the computational complexity since the
tree structure algorithm requires to solve many PDEs. Furthermore, we prove an
error estimate which guarantees the convergence of the proposed method.
Finally, we show efficiency of the method through numerical tests
Linearly Solvable Stochastic Control Lyapunov Functions
This paper presents a new method for synthesizing stochastic control Lyapunov
functions for a class of nonlinear stochastic control systems. The technique
relies on a transformation of the classical nonlinear Hamilton-Jacobi-Bellman
partial differential equation to a linear partial differential equation for a
class of problems with a particular constraint on the stochastic forcing. This
linear partial differential equation can then be relaxed to a linear
differential inclusion, allowing for relaxed solutions to be generated using
sum of squares programming. The resulting relaxed solutions are in fact
viscosity super/subsolutions, and by the maximum principle are pointwise upper
and lower bounds to the underlying value function, even for coarse polynomial
approximations. Furthermore, the pointwise upper bound is shown to be a
stochastic control Lyapunov function, yielding a method for generating
nonlinear controllers with pointwise bounded distance from the optimal cost
when using the optimal controller. These approximate solutions may be computed
with non-increasing error via a hierarchy of semidefinite optimization
problems. Finally, this paper develops a-priori bounds on trajectory
suboptimality when using these approximate value functions, as well as
demonstrates that these methods, and bounds, can be applied to a more general
class of nonlinear systems not obeying the constraint on stochastic forcing.
Simulated examples illustrate the methodology.Comment: Published in SIAM Journal of Control and Optimizatio
- …