6,911 research outputs found
Hamiltonian ABC
Approximate Bayesian computation (ABC) is a powerful and elegant framework
for performing inference in simulation-based models. However, due to the
difficulty in scaling likelihood estimates, ABC remains useful for relatively
low-dimensional problems. We introduce Hamiltonian ABC (HABC), a set of
likelihood-free algorithms that apply recent advances in scaling Bayesian
learning using Hamiltonian Monte Carlo (HMC) and stochastic gradients. We find
that a small number forward simulations can effectively approximate the ABC
gradient, allowing Hamiltonian dynamics to efficiently traverse parameter
spaces. We also describe a new simple yet general approach of incorporating
random seeds into the state of the Markov chain, further reducing the random
walk behavior of HABC. We demonstrate HABC on several typical ABC problems, and
show that HABC samples comparably to regular Bayesian inference using true
gradients on a high-dimensional problem from machine learning.Comment: Submission to UAI 201
A spatial analysis of multivariate output from regional climate models
Climate models have become an important tool in the study of climate and
climate change, and ensemble experiments consisting of multiple climate-model
runs are used in studying and quantifying the uncertainty in climate-model
output. However, there are often only a limited number of model runs available
for a particular experiment, and one of the statistical challenges is to
characterize the distribution of the model output. To that end, we have
developed a multivariate hierarchical approach, at the heart of which is a new
representation of a multivariate Markov random field. This approach allows for
flexible modeling of the multivariate spatial dependencies, including the
cross-dependencies between variables. We demonstrate this statistical model on
an ensemble arising from a regional-climate-model experiment over the western
United States, and we focus on the projected change in seasonal temperature and
precipitation over the next 50 years.Comment: Published in at http://dx.doi.org/10.1214/10-AOAS369 the Annals of
Applied Statistics (http://www.imstat.org/aoas/) by the Institute of
Mathematical Statistics (http://www.imstat.org
An empirical Bayes approach to identification of modules in dynamic networks
We present a new method of identifying a specific module in a dynamic
network, possibly with feedback loops. Assuming known topology, we express the
dynamics by an acyclic network composed of two blocks where the first block
accounts for the relation between the known reference signals and the input to
the target module, while the second block contains the target module. Using an
empirical Bayes approach, we model the first block as a Gaussian vector with
covariance matrix (kernel) given by the recently introduced stable spline
kernel. The parameters of the target module are estimated by solving a marginal
likelihood problem with a novel iterative scheme based on the
Expectation-Maximization algorithm. Additionally, we extend the method to
include additional measurements downstream of the target module. Using Markov
Chain Monte Carlo techniques, it is shown that the same iterative scheme can
solve also this formulation. Numerical experiments illustrate the effectiveness
of the proposed methods
Data-Driven Model Reduction for the Bayesian Solution of Inverse Problems
One of the major challenges in the Bayesian solution of inverse problems
governed by partial differential equations (PDEs) is the computational cost of
repeatedly evaluating numerical PDE models, as required by Markov chain Monte
Carlo (MCMC) methods for posterior sampling. This paper proposes a data-driven
projection-based model reduction technique to reduce this computational cost.
The proposed technique has two distinctive features. First, the model reduction
strategy is tailored to inverse problems: the snapshots used to construct the
reduced-order model are computed adaptively from the posterior distribution.
Posterior exploration and model reduction are thus pursued simultaneously.
Second, to avoid repeated evaluations of the full-scale numerical model as in a
standard MCMC method, we couple the full-scale model and the reduced-order
model together in the MCMC algorithm. This maintains accurate inference while
reducing its overall computational cost. In numerical experiments considering
steady-state flow in a porous medium, the data-driven reduced-order model
achieves better accuracy than a reduced-order model constructed using the
classical approach. It also improves posterior sampling efficiency by several
orders of magnitude compared to a standard MCMC method
- …