4,590 research outputs found
Pseudo-Marginal Slice Sampling
Markov chain Monte Carlo (MCMC) methods asymptotically sample from complex
probability distributions. The pseudo-marginal MCMC framework only requires an
unbiased estimator of the unnormalized probability distribution function to
construct a Markov chain. However, the resulting chains are harder to tune to a
target distribution than conventional MCMC, and the types of updates available
are limited. We describe a general way to clamp and update the random numbers
used in a pseudo-marginal method's unbiased estimator. In this framework we can
use slice sampling and other adaptive methods. We obtain more robust Markov
chains, which often mix more quickly.Comment: 9 pages, 6 figures, 1 table. Version 2 includes citations to
closely-related work released on arXiv since version
Driving Markov chain Monte Carlo with a dependent random stream
Markov chain Monte Carlo is a widely-used technique for generating a
dependent sequence of samples from complex distributions. Conventionally, these
methods require a source of independent random variates. Most implementations
use pseudo-random numbers instead because generating true independent variates
with a physical system is not straightforward. In this paper we show how to
modify some commonly used Markov chains to use a dependent stream of random
numbers in place of independent uniform variates. The resulting Markov chains
have the correct invariant distribution without requiring detailed knowledge of
the stream's dependencies or even its marginal distribution. As a side-effect,
sometimes far fewer random numbers are required to obtain accurate results.Comment: 16 pages, 4 figure
Adaptive Multiple Importance Sampling for Gaussian Processes
In applications of Gaussian processes where quantification of uncertainty is
a strict requirement, it is necessary to accurately characterize the posterior
distribution over Gaussian process covariance parameters. Normally, this is
done by means of standard Markov chain Monte Carlo (MCMC) algorithms. Motivated
by the issues related to the complexity of calculating the marginal likelihood
that can make MCMC algorithms inefficient, this paper develops an alternative
inference framework based on Adaptive Multiple Importance Sampling (AMIS). This
paper studies the application of AMIS in the case of a Gaussian likelihood, and
proposes the Pseudo-Marginal AMIS for non-Gaussian likelihoods, where the
marginal likelihood is unbiasedly estimated. The results suggest that the
proposed framework outperforms MCMC-based inference of covariance parameters in
a wide range of scenarios and remains competitive for moderately large
dimensional parameter spaces.Comment: 27 page
Pseudo-marginal Bayesian inference for Gaussian process latent variable models
A Bayesian inference framework for supervised Gaussian process latent variable models is introduced. The framework overcomes the high correlations between latent variables and hyperparameters by collapsing the statistical model through approximate integration of the latent variables. Using an unbiased pseudo estimate for the marginal likelihood, the exact hyperparameter posterior can then be explored using collapsed Gibbs sampling and, conditional on these samples, the exact latent posterior can be explored through elliptical slice sampling. The framework is tested on both simulated and real examples. When compared with the standard approach based on variational inference, this approach leads to significant improvements in the predictive accuracy and quantification of uncertainty, as well as a deeper insight into the challenges of performing inference in this class of models
Premium: An R package for profile regression mixture models using dirichlet processes
PReMiuM is a recently developed R package for Bayesian clustering using a Dirichlet process mixture model. This model is an alternative to regression models, nonparametrically linking a response vector to covariate data through cluster membership (Molitor, Papathomas, Jerrett, and Richardson 2010). The package allows binary, categorical, count and continuous response, as well as continuous and discrete covariates. Additionally, predictions may be made for the response, and missing values for the covariates are handled. Several samplers and label switching moves are implemented along with diagnostic tools to assess convergence. A number of R functions for post-processing of the output are also provided. In addition to fitting mixtures, it may additionally be of interest to determine which covariates actively drive the mixture components. This is implemented in the package as variable selection
Accelerating Metropolis-Hastings algorithms: Delayed acceptance with prefetching
MCMC algorithms such as Metropolis-Hastings algorithms are slowed down by the
computation of complex target distributions as exemplified by huge datasets. We
offer in this paper an approach to reduce the computational costs of such
algorithms by a simple and universal divide-and-conquer strategy. The idea
behind the generic acceleration is to divide the acceptance step into several
parts, aiming at a major reduction in computing time that outranks the
corresponding reduction in acceptance probability. The division decomposes the
"prior x likelihood" term into a product such that some of its components are
much cheaper to compute than others. Each of the components can be sequentially
compared with a uniform variate, the first rejection signalling that the
proposed value is considered no further, This approach can in turn be
accelerated as part of a prefetching algorithm taking advantage of the parallel
abilities of the computer at hand. We illustrate those accelerating features on
a series of toy and realistic examples.Comment: 20 pages, 12 figures, 2 tables, submitte
- …