2,151 research outputs found
Randomized Hamiltonian Monte Carlo as Scaling Limit of the Bouncy Particle Sampler and Dimension-Free Convergence Rates
The Bouncy Particle Sampler is a Markov chain Monte Carlo method based on a
nonreversible piecewise deterministic Markov process. In this scheme, a
particle explores the state space of interest by evolving according to a linear
dynamics which is altered by bouncing on the hyperplane tangent to the gradient
of the negative log-target density at the arrival times of an inhomogeneous
Poisson Process (PP) and by randomly perturbing its velocity at the arrival
times of an homogeneous PP. Under regularity conditions, we show here that the
process corresponding to the first component of the particle and its
corresponding velocity converges weakly towards a Randomized Hamiltonian Monte
Carlo (RHMC) process as the dimension of the ambient space goes to infinity.
RHMC is another piecewise deterministic non-reversible Markov process where a
Hamiltonian dynamics is altered at the arrival times of a homogeneous PP by
randomly perturbing the momentum component. We then establish dimension-free
convergence rates for RHMC for strongly log-concave targets with bounded
Hessians using coupling ideas and hypocoercivity techniques.Comment: 47 pages, 2 figure
Diffusion limits of the random walk Metropolis algorithm in high dimensions
Diffusion limits of MCMC methods in high dimensions provide a useful
theoretical tool for studying computational complexity. In particular, they
lead directly to precise estimates of the number of steps required to explore
the target measure, in stationarity, as a function of the dimension of the
state space. However, to date such results have mainly been proved for target
measures with a product structure, severely limiting their applicability. The
purpose of this paper is to study diffusion limits for a class of naturally
occurring high-dimensional measures found from the approximation of measures on
a Hilbert space which are absolutely continuous with respect to a Gaussian
reference measure. The diffusion limit of a random walk Metropolis algorithm to
an infinite-dimensional Hilbert space valued SDE (or SPDE) is proved,
facilitating understanding of the computational complexity of the algorithm.Comment: Published in at http://dx.doi.org/10.1214/10-AAP754 the Annals of
Applied Probability (http://www.imstat.org/aap/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Multi-level Monte Carlo for continuous time Markov chains, with applications in biochemical kinetics
We show how to extend a recently proposed multi-level Monte Carlo approach to
the continuous time Markov chain setting, thereby greatly lowering the
computational complexity needed to compute expected values of functions of the
state of the system to a specified accuracy. The extension is non-trivial,
exploiting a coupling of the requisite processes that is easy to simulate while
providing a small variance for the estimator. Further, and in a stark departure
from other implementations of multi-level Monte Carlo, we show how to produce
an unbiased estimator that is significantly less computationally expensive than
the usual unbiased estimator arising from exact algorithms in conjunction with
crude Monte Carlo. We thereby dramatically improve, in a quantifiable manner,
the basic computational complexity of current approaches that have many names
and variants across the scientific literature, including the
Bortz-Kalos-Lebowitz algorithm, discrete event simulation, dynamic Monte Carlo,
kinetic Monte Carlo, the n-fold way, the next reaction method,the
residence-time algorithm, the stochastic simulation algorithm, Gillespie's
algorithm, and tau-leaping. The new algorithm applies generically, but we also
give an example where the coupling idea alone, even without a multi-level
discretization, can be used to improve efficiency by exploiting system
structure. Stochastically modeled chemical reaction networks provide a very
important application for this work. Hence, we use this context for our
notation, terminology, natural scalings, and computational examples.Comment: Improved description of the constants in statement of Theorem
Bayesian Reinforcement Learning via Deep, Sparse Sampling
We address the problem of Bayesian reinforcement learning using efficient
model-based online planning. We propose an optimism-free Bayes-adaptive
algorithm to induce deeper and sparser exploration with a theoretical bound on
its performance relative to the Bayes optimal policy, with a lower
computational complexity. The main novelty is the use of a candidate policy
generator, to generate long-term options in the planning tree (over beliefs),
which allows us to create much sparser and deeper trees. Experimental results
on different environments show that in comparison to the state-of-the-art, our
algorithm is both computationally more efficient, and obtains significantly
higher reward in discrete environments.Comment: Published in AISTATS 202
Data-driven modelling of biological multi-scale processes
Biological processes involve a variety of spatial and temporal scales. A
holistic understanding of many biological processes therefore requires
multi-scale models which capture the relevant properties on all these scales.
In this manuscript we review mathematical modelling approaches used to describe
the individual spatial scales and how they are integrated into holistic models.
We discuss the relation between spatial and temporal scales and the implication
of that on multi-scale modelling. Based upon this overview over
state-of-the-art modelling approaches, we formulate key challenges in
mathematical and computational modelling of biological multi-scale and
multi-physics processes. In particular, we considered the availability of
analysis tools for multi-scale models and model-based multi-scale data
integration. We provide a compact review of methods for model-based data
integration and model-based hypothesis testing. Furthermore, novel approaches
and recent trends are discussed, including computation time reduction using
reduced order and surrogate models, which contribute to the solution of
inference problems. We conclude the manuscript by providing a few ideas for the
development of tailored multi-scale inference methods.Comment: This manuscript will appear in the Journal of Coupled Systems and
Multiscale Dynamics (American Scientific Publishers
- ā¦