932 research outputs found
Patterns of Scalable Bayesian Inference
Datasets are growing not just in size but in complexity, creating a demand
for rich models and quantification of uncertainty. Bayesian methods are an
excellent fit for this demand, but scaling Bayesian inference is a challenge.
In response to this challenge, there has been considerable recent work based on
varying assumptions about model structure, underlying computational resources,
and the importance of asymptotic correctness. As a result, there is a zoo of
ideas with few clear overarching principles.
In this paper, we seek to identify unifying principles, patterns, and
intuitions for scaling Bayesian inference. We review existing work on utilizing
modern computing resources with both MCMC and variational approximation
techniques. From this taxonomy of ideas, we characterize the general principles
that have proven successful for designing scalable inference procedures and
comment on the path forward
Generalized Direct Sampling for Hierarchical Bayesian Models
We develop a new method to sample from posterior distributions in
hierarchical models without using Markov chain Monte Carlo. This method, which
is a variant of importance sampling ideas, is generally applicable to
high-dimensional models involving large data sets. Samples are independent, so
they can be collected in parallel, and we do not need to be concerned with
issues like chain convergence and autocorrelation. Additionally, the method can
be used to compute marginal likelihoods
Estimating the granularity coefficient of a Potts-Markov random field within an MCMC algorithm
This paper addresses the problem of estimating the Potts parameter B jointly
with the unknown parameters of a Bayesian model within a Markov chain Monte
Carlo (MCMC) algorithm. Standard MCMC methods cannot be applied to this problem
because performing inference on B requires computing the intractable
normalizing constant of the Potts model. In the proposed MCMC method the
estimation of B is conducted using a likelihood-free Metropolis-Hastings
algorithm. Experimental results obtained for synthetic data show that
estimating B jointly with the other unknown parameters leads to estimation
results that are as good as those obtained with the actual value of B. On the
other hand, assuming that the value of B is known can degrade estimation
performance significantly if this value is incorrect. To illustrate the
interest of this method, the proposed algorithm is successfully applied to real
bidimensional SAR and tridimensional ultrasound images
- …