1,298 research outputs found
Gibbs Sampling using Anti-correlation Gaussian Data Augmentation, with Applications to L1-ball-type Models
L1-ball-type priors are a recent generalization of the spike-and-slab priors.
By transforming a continuous precursor distribution to the L1-ball boundary, it
induces exact zeros with positive prior and posterior probabilities. With great
flexibility in choosing the precursor and threshold distributions, we can
easily specify models under structured sparsity, such as those with dependent
probability for zeros and smoothness among the non-zeros. Motivated to
significantly accelerate the posterior computation, we propose a new data
augmentation that leads to a fast block Gibbs sampling algorithm. The latent
variable, named ``anti-correlation Gaussian'', cancels out the quadratic
exponent term in the latent Gaussian distribution, making the parameters of
interest conditionally independent so that they can be updated in a block.
Compared to existing algorithms such as the No-U-Turn sampler, the new blocked
Gibbs sampler has a very low computing cost per iteration and shows rapid
mixing of Markov chains. We establish the geometric ergodicity guarantee of the
algorithm in linear models. Further, we show useful extensions of our algorithm
for posterior estimation of general latent Gaussian models, such as those
involving multivariate truncated Gaussian or latent Gaussian process. Keywords:
Blocked Gibbs sampler; Fast Mixing of Markov Chains; Latent Gaussian Models;
Soft-thresholding
Sparse Linear Identifiable Multivariate Modeling
In this paper we consider sparse and identifiable linear latent variable
(factor) and linear Bayesian network models for parsimonious analysis of
multivariate data. We propose a computationally efficient method for joint
parameter and model inference, and model comparison. It consists of a fully
Bayesian hierarchy for sparse models using slab and spike priors (two-component
delta-function and continuous mixtures), non-Gaussian latent factors and a
stochastic search over the ordering of the variables. The framework, which we
call SLIM (Sparse Linear Identifiable Multivariate modeling), is validated and
bench-marked on artificial and real biological data sets. SLIM is closest in
spirit to LiNGAM (Shimizu et al., 2006), but differs substantially in
inference, Bayesian network structure learning and model comparison.
Experimentally, SLIM performs equally well or better than LiNGAM with
comparable computational complexity. We attribute this mainly to the stochastic
search strategy used, and to parsimony (sparsity and identifiability), which is
an explicit part of the model. We propose two extensions to the basic i.i.d.
linear framework: non-linear dependence on observed variables, called SNIM
(Sparse Non-linear Identifiable Multivariate modeling) and allowing for
correlations between latent variables, called CSLIM (Correlated SLIM), for the
temporal and/or spatial data. The source code and scripts are available from
http://cogsys.imm.dtu.dk/slim/.Comment: 45 pages, 17 figure
- …