1,658 research outputs found
Sequential Monte Carlo EM for multivariate probit models
Multivariate probit models (MPM) have the appealing feature of capturing some
of the dependence structure between the components of multidimensional binary
responses. The key for the dependence modelling is the covariance matrix of an
underlying latent multivariate Gaussian. Most approaches to MLE in multivariate
probit regression rely on MCEM algorithms to avoid computationally intensive
evaluations of multivariate normal orthant probabilities. As an alternative to
the much used Gibbs sampler a new SMC sampler for truncated multivariate
normals is proposed. The algorithm proceeds in two stages where samples are
first drawn from truncated multivariate Student distributions and then
further evolved towards a Gaussian. The sampler is then embedded in a MCEM
algorithm. The sequential nature of SMC methods can be exploited to design a
fully sequential version of the EM, where the samples are simply updated from
one iteration to the next rather than resampled from scratch. Recycling the
samples in this manner significantly reduces the computational cost. An
alternative view of the standard conditional maximisation step provides the
basis for an iterative procedure to fully perform the maximisation needed in
the EM algorithm. The identifiability of MPM is also thoroughly discussed. In
particular, the likelihood invariance can be embedded in the EM algorithm to
ensure that constrained and unconstrained maximisation are equivalent. A simple
iterative procedure is then derived for either maximisation which takes
effectively no computational time. The method is validated by applying it to
the widely analysed Six Cities dataset and on a higher dimensional simulated
example. Previous approaches to the Six Cities overly restrict the parameter
space but, by considering the correct invariance, the maximum likelihood is
quite naturally improved when treating the full unrestricted model.Comment: 26 pages, 2 figures. In press, Computational Statistics & Data
Analysi
Computation of Gaussian orthant probabilities in high dimension
We study the computation of Gaussian orthant probabilities, i.e. the
probability that a Gaussian falls inside a quadrant. The
Geweke-Hajivassiliou-Keane (GHK) algorithm [Genz, 1992; Geweke, 1991;
Hajivassiliou et al., 1996; Keane, 1993], is currently used for integrals of
dimension greater than 10. In this paper we show that for Markovian covariances
GHK can be interpreted as the estimator of the normalizing constant of a state
space model using sequential importance sampling (SIS). We show for an AR(1)
the variance of the GHK, properly normalized, diverges exponentially fast with
the dimension. As an improvement we propose using a particle filter (PF). We
then generalize this idea to arbitrary covariance matrices using Sequential
Monte Carlo (SMC) with properly tailored MCMC moves. We show empirically that
this can lead to drastic improvements on currently used algorithms. We also
extend the framework to orthants of mixture of Gaussians (Student, Cauchy
etc.), and to the simulation of truncated Gaussians
Conjugate Bayes for probit regression via unified skew-normal distributions
Regression models for dichotomous data are ubiquitous in statistics. Besides
being useful for inference on binary responses, these methods serve also as
building blocks in more complex formulations, such as density regression,
nonparametric classification and graphical models. Within the Bayesian
framework, inference proceeds by updating the priors for the coefficients,
typically set to be Gaussians, with the likelihood induced by probit or logit
regressions for the responses. In this updating, the apparent absence of a
tractable posterior has motivated a variety of computational methods, including
Markov Chain Monte Carlo routines and algorithms which approximate the
posterior. Despite being routinely implemented, Markov Chain Monte Carlo
strategies face mixing or time-inefficiency issues in large p and small n
studies, whereas approximate routines fail to capture the skewness typically
observed in the posterior. This article proves that the posterior distribution
for the probit coefficients has a unified skew-normal kernel, under Gaussian
priors. Such a novel result allows efficient Bayesian inference for a wide
class of applications, especially in large p and small-to-moderate n studies
where state-of-the-art computational methods face notable issues. These
advances are outlined in a genetic study, and further motivate the development
of a wider class of conjugate priors for probit models along with methods to
obtain independent and identically distributed samples from the unified
skew-normal posterior
Hierarchical Bayesian sparse image reconstruction with application to MRFM
This paper presents a hierarchical Bayesian model to reconstruct sparse
images when the observations are obtained from linear transformations and
corrupted by an additive white Gaussian noise. Our hierarchical Bayes model is
well suited to such naturally sparse image applications as it seamlessly
accounts for properties such as sparsity and positivity of the image via
appropriate Bayes priors. We propose a prior that is based on a weighted
mixture of a positive exponential distribution and a mass at zero. The prior
has hyperparameters that are tuned automatically by marginalization over the
hierarchical Bayesian model. To overcome the complexity of the posterior
distribution, a Gibbs sampling strategy is proposed. The Gibbs samples can be
used to estimate the image to be recovered, e.g. by maximizing the estimated
posterior distribution. In our fully Bayesian approach the posteriors of all
the parameters are available. Thus our algorithm provides more information than
other previously proposed sparse reconstruction methods that only give a point
estimate. The performance of our hierarchical Bayesian sparse reconstruction
method is illustrated on synthetic and real data collected from a tobacco virus
sample using a prototype MRFM instrument.Comment: v2: final version; IEEE Trans. Image Processing, 200
Efficient Gaussian Sampling for Solving Large-Scale Inverse Problems using MCMC Methods
The resolution of many large-scale inverse problems using MCMC methods
requires a step of drawing samples from a high dimensional Gaussian
distribution. While direct Gaussian sampling techniques, such as those based on
Cholesky factorization, induce an excessive numerical complexity and memory
requirement, sequential coordinate sampling methods present a low rate of
convergence. Based on the reversible jump Markov chain framework, this paper
proposes an efficient Gaussian sampling algorithm having a reduced computation
cost and memory usage. The main feature of the algorithm is to perform an
approximate resolution of a linear system with a truncation level adjusted
using a self-tuning adaptive scheme allowing to achieve the minimal computation
cost. The connection between this algorithm and some existing strategies is
discussed and its efficiency is illustrated on a linear inverse problem of
image resolution enhancement.Comment: 20 pages, 10 figures, under review for journal publicatio
- âŠ