25,871 research outputs found
Information-Theoretic Stochastic Optimal Control via Incremental Sampling-based Algorithms
This paper considers optimal control of dynamical systems which are
represented by nonlinear stochastic differential equations. It is well-known
that the optimal control policy for this problem can be obtained as a function
of a value function that satisfies a nonlinear partial differential equation,
namely, the Hamilton-Jacobi-Bellman equation. This nonlinear PDE must be solved
backwards in time, and this computation is intractable for large scale systems.
Under certain assumptions, and after applying a logarithmic transformation, an
alternative characterization of the optimal policy can be given in terms of a
path integral. Path Integral (PI) based control methods have recently been
shown to provide elegant solutions to a broad class of stochastic optimal
control problems. One of the implementation challenges with this formalism is
the computation of the expectation of a cost functional over the trajectories
of the unforced dynamics. Computing such expectation over trajectories that are
sampled uniformly may induce numerical instabilities due to the exponentiation
of the cost. Therefore, sampling of low-cost trajectories is essential for the
practical implementation of PI-based methods. In this paper, we use incremental
sampling-based algorithms to sample useful trajectories from the unforced
system dynamics, and make a novel connection between Rapidly-exploring Random
Trees (RRTs) and information-theoretic stochastic optimal control. We show the
results from the numerical implementation of the proposed approach to several
examples.Comment: 18 page
Topology-Guided Path Integral Approach for Stochastic Optimal Control in Cluttered Environment
This paper addresses planning and control of robot motion under uncertainty
that is formulated as a continuous-time, continuous-space stochastic optimal
control problem, by developing a topology-guided path integral control method.
The path integral control framework, which forms the backbone of the proposed
method, re-writes the Hamilton-Jacobi-Bellman equation as a statistical
inference problem; the resulting inference problem is solved by a sampling
procedure that computes the distribution of controlled trajectories around the
trajectory by the passive dynamics. For motion control of robots in a highly
cluttered environment, however, this sampling can easily be trapped in a local
minimum unless the sample size is very large, since the global optimality of
local minima depends on the degree of uncertainty. Thus, a homology-embedded
sampling-based planner that identifies many (potentially) local-minimum
trajectories in different homology classes is developed to aid the sampling
process. In combination with a receding-horizon fashion of the optimal control
the proposed method produces a dynamically feasible and collision-free motion
plans without being trapped in a local minimum. Numerical examples on a
synthetic toy problem and on quadrotor control in a complex obstacle field
demonstrate the validity of the proposed method.Comment: arXiv admin note: text overlap with arXiv:1510.0534
Hybrid Deterministic-Stochastic Methods for Data Fitting
Many structured data-fitting applications require the solution of an
optimization problem involving a sum over a potentially large number of
measurements. Incremental gradient algorithms offer inexpensive iterations by
sampling a subset of the terms in the sum. These methods can make great
progress initially, but often slow as they approach a solution. In contrast,
full-gradient methods achieve steady convergence at the expense of evaluating
the full objective and gradient on each iteration. We explore hybrid methods
that exhibit the benefits of both approaches. Rate-of-convergence analysis
shows that by controlling the sample size in an incremental gradient algorithm,
it is possible to maintain the steady convergence rates of full-gradient
methods. We detail a practical quasi-Newton implementation based on this
approach. Numerical experiments illustrate its potential benefits.Comment: 26 pages. Revised proofs of Theorems 2.6 and 3.1, results unchange
Sequential Monte Carlo Methods for Option Pricing
In the following paper we provide a review and development of sequential
Monte Carlo (SMC) methods for option pricing. SMC are a class of Monte
Carlo-based algorithms, that are designed to approximate expectations w.r.t a
sequence of related probability measures. These approaches have been used,
successfully, for a wide class of applications in engineering, statistics,
physics and operations research. SMC methods are highly suited to many option
pricing problems and sensitivity/Greek calculations due to the nature of the
sequential simulation. However, it is seldom the case that such ideas are
explicitly used in the option pricing literature. This article provides an
up-to date review of SMC methods, which are appropriate for option pricing. In
addition, it is illustrated how a number of existing approaches for option
pricing can be enhanced via SMC. Specifically, when pricing the arithmetic
Asian option w.r.t a complex stochastic volatility model, it is shown that SMC
methods provide additional strategies to improve estimation.Comment: 37 Pages, 2 Figure
- …