95,329 research outputs found
Implicit sampling for path integral control, Monte Carlo localization, and SLAM
The applicability and usefulness of implicit sampling in stochastic optimal
control, stochastic localization, and simultaneous localization and mapping
(SLAM), is explored; implicit sampling is a recently-developed
variationally-enhanced sampling method. The theory is illustrated with
examples, and it is found that implicit sampling is significantly more
efficient than current Monte Carlo methods in test problems for all three
applications
Reducing Reparameterization Gradient Variance
Optimization with noisy gradients has become ubiquitous in statistics and
machine learning. Reparameterization gradients, or gradient estimates computed
via the "reparameterization trick," represent a class of noisy gradients often
used in Monte Carlo variational inference (MCVI). However, when these gradient
estimators are too noisy, the optimization procedure can be slow or fail to
converge. One way to reduce noise is to use more samples for the gradient
estimate, but this can be computationally expensive. Instead, we view the noisy
gradient as a random variable, and form an inexpensive approximation of the
generating procedure for the gradient sample. This approximation has high
correlation with the noisy gradient by construction, making it a useful control
variate for variance reduction. We demonstrate our approach on non-conjugate
multi-level hierarchical models and a Bayesian neural net where we observed
gradient variance reductions of multiple orders of magnitude (20-2,000x)
Path integrals and symmetry breaking for optimal control theory
This paper considers linear-quadratic control of a non-linear dynamical
system subject to arbitrary cost. I show that for this class of stochastic
control problems the non-linear Hamilton-Jacobi-Bellman equation can be
transformed into a linear equation. The transformation is similar to the
transformation used to relate the classical Hamilton-Jacobi equation to the
Schr\"odinger equation. As a result of the linearity, the usual backward
computation can be replaced by a forward diffusion process, that can be
computed by stochastic integration or by the evaluation of a path integral. It
is shown, how in the deterministic limit the PMP formalism is recovered. The
significance of the path integral approach is that it forms the basis for a
number of efficient computational methods, such as MC sampling, the Laplace
approximation and the variational approximation. We show the effectiveness of
the first two methods in number of examples. Examples are given that show the
qualitative difference between stochastic and deterministic control and the
occurrence of symmetry breaking as a function of the noise.Comment: 21 pages, 6 figures, submitted to JSTA
- …