5,481 research outputs found
Wormhole Hamiltonian Monte Carlo
In machine learning and statistics, probabilistic inference involving
multimodal distributions is quite difficult. This is especially true in high
dimensional problems, where most existing algorithms cannot easily move from
one mode to another. To address this issue, we propose a novel Bayesian
inference approach based on Markov Chain Monte Carlo. Our method can
effectively sample from multimodal distributions, especially when the dimension
is high and the modes are isolated. To this end, it exploits and modifies the
Riemannian geometric properties of the target distribution to create
\emph{wormholes} connecting modes in order to facilitate moving between them.
Further, our proposed method uses the regeneration technique in order to adapt
the algorithm by identifying new modes and updating the network of wormholes
without affecting the stationary distribution. To find new modes, as opposed to
rediscovering those previously identified, we employ a novel mode searching
algorithm that explores a \emph{residual energy} function obtained by
subtracting an approximate Gaussian mixture density (based on previously
discovered modes) from the target density function
Monte Carlo Bayesian Reinforcement Learning
Bayesian reinforcement learning (BRL) encodes prior knowledge of the world in
a model and represents uncertainty in model parameters by maintaining a
probability distribution over them. This paper presents Monte Carlo BRL
(MC-BRL), a simple and general approach to BRL. MC-BRL samples a priori a
finite set of hypotheses for the model parameter values and forms a discrete
partially observable Markov decision process (POMDP) whose state space is a
cross product of the state space for the reinforcement learning task and the
sampled model parameter space. The POMDP does not require conjugate
distributions for belief representation, as earlier works do, and can be solved
relatively easily with point-based approximation algorithms. MC-BRL naturally
handles both fully and partially observable worlds. Theoretical and
experimental results show that the discrete POMDP approximates the underlying
BRL task well with guaranteed performance.Comment: Appears in Proceedings of the 29th International Conference on
Machine Learning (ICML 2012
Small-world MCMC and convergence to multi-modal distributions: From slow mixing to fast mixing
We compare convergence rates of Metropolis--Hastings chains to multi-modal
target distributions when the proposal distributions can be of ``local'' and
``small world'' type. In particular, we show that by adding occasional
long-range jumps to a given local proposal distribution, one can turn a chain
that is ``slowly mixing'' (in the complexity of the problem) into a chain that
is ``rapidly mixing.'' To do this, we obtain spectral gap estimates via a new
state decomposition theorem and apply an isoperimetric inequality for
log-concave probability measures. We discuss potential applicability of our
result to Metropolis-coupled Markov chain Monte Carlo schemes.Comment: Published at http://dx.doi.org/10.1214/105051606000000772 in the
Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute
of Mathematical Statistics (http://www.imstat.org
Variational Hamiltonian Monte Carlo via Score Matching
Traditionally, the field of computational Bayesian statistics has been
divided into two main subfields: variational methods and Markov chain Monte
Carlo (MCMC). In recent years, however, several methods have been proposed
based on combining variational Bayesian inference and MCMC simulation in order
to improve their overall accuracy and computational efficiency. This marriage
of fast evaluation and flexible approximation provides a promising means of
designing scalable Bayesian inference methods. In this paper, we explore the
possibility of incorporating variational approximation into a state-of-the-art
MCMC method, Hamiltonian Monte Carlo (HMC), to reduce the required gradient
computation in the simulation of Hamiltonian flow, which is the bottleneck for
many applications of HMC in big data problems. To this end, we use a {\it
free-form} approximation induced by a fast and flexible surrogate function
based on single-hidden layer feedforward neural networks. The surrogate
provides sufficiently accurate approximation while allowing for fast
exploration of parameter space, resulting in an efficient approximate inference
algorithm. We demonstrate the advantages of our method on both synthetic and
real data problems
CMBfit: Rapid WMAP likelihood calculations with normal parameters
We present a method for ultra-fast confrontation of the WMAP cosmic microwave
background observations with theoretical models, implemented as a publicly
available software package called CMBfit, useful for anyone wishing to measure
cosmological parameters by combining WMAP with other observations. The method
takes advantage of the underlying physics by transforming into a set of
parameters where the WMAP likelihood surface is accurately fit by the
exponential of a quartic or sextic polynomial. Building on previous physics
based approximations by Hu et.al., Kosowsky et.al. and Chu et.al., it combines
their speed with precision cosmology grade accuracy. A Fortran code for
computing the WMAP likelihood for a given set of parameters is provided,
pre-calibrated against CMBfast, accurate to Delta lnL ~ 0.05 over the entire
2sigma region of the parameter space for 6 parameter ``vanilla'' Lambda CDM
models. We also provide 7-parameter fits including spatial curvature,
gravitational waves and a running spectral index.Comment: 14 pages, 8 figures, References added, accepted for publication in
Phys.Rev.D., a Fortran code can be downloaded from
http://space.mit.edu/home/tegmark/cmbfit
- …