231 research outputs found
Information-Theoretic Stochastic Optimal Control via Incremental Sampling-based Algorithms
This paper considers optimal control of dynamical systems which are
represented by nonlinear stochastic differential equations. It is well-known
that the optimal control policy for this problem can be obtained as a function
of a value function that satisfies a nonlinear partial differential equation,
namely, the Hamilton-Jacobi-Bellman equation. This nonlinear PDE must be solved
backwards in time, and this computation is intractable for large scale systems.
Under certain assumptions, and after applying a logarithmic transformation, an
alternative characterization of the optimal policy can be given in terms of a
path integral. Path Integral (PI) based control methods have recently been
shown to provide elegant solutions to a broad class of stochastic optimal
control problems. One of the implementation challenges with this formalism is
the computation of the expectation of a cost functional over the trajectories
of the unforced dynamics. Computing such expectation over trajectories that are
sampled uniformly may induce numerical instabilities due to the exponentiation
of the cost. Therefore, sampling of low-cost trajectories is essential for the
practical implementation of PI-based methods. In this paper, we use incremental
sampling-based algorithms to sample useful trajectories from the unforced
system dynamics, and make a novel connection between Rapidly-exploring Random
Trees (RRTs) and information-theoretic stochastic optimal control. We show the
results from the numerical implementation of the proposed approach to several
examples.Comment: 18 page
Optimal Covariance Steering for Continuous-Time Linear Stochastic Systems With Additive Noise
In this paper, we study the problem of how to optimally steer the state
covariance of a general continuous-time linear stochastic system over a finite
time interval subject to additive noise. Optimality here means reaching a
target state covariance with minimal control energy. The additive noise may
include a combination of white Gaussian noise and abrupt "jump noise" that is
discontinuous in time. We first establish the controllability of the state
covariance for linear time-varying stochastic systems. We then turn to the
derivation of the optimal control, which entails solving two dynamically
coupled matrix ordinary differential equations (ODEs) with split boundary
conditions. We show the existence and uniqueness of the solution to these
coupled matrix ODEs, and thus those of the optimal control.Comment: 8 pages, 2 figure
Batch Belief Trees for Motion Planning Under Uncertainty
In this work, we develop the Batch Belief Trees (BBT) algorithm for motion
planning under motion and sensing uncertainties. The algorithm interleaves
between batch sampling, building a graph of nominal trajectories in the state
space, and searching over the graph to find belief space motion plans. By
searching over the graph, BBT finds sophisticated plans that will visit (and
revisit) information-rich regions to reduce uncertainty. One of the key
benefits of this algorithm is the modified interplay between exploration and
exploitation. Instead of an exhaustive search (exploitation) after one
exploration step, the proposed algorithm uses batch samples to explore the
state space and, in addition, does not require exhaustive search before the
next iteration of batch sampling, which adds flexibility.The algorithm finds
motion plans that converge to the optimal one as more samples are added to the
graph. We test BBT in different planning environments. Our numerical
investigation confirms that BBT finds non-trivial motion plans and is faster
compared with previous similar methods
- …