1,458 research outputs found
Particle Gibbs Split-Merge Sampling for Bayesian Inference in Mixture Models
This paper presents a new Markov chain Monte Carlo method to sample from the
posterior distribution of conjugate mixture models. This algorithm relies on a
flexible split-merge procedure built using the particle Gibbs sampler. Contrary
to available split-merge procedures, the resulting so-called Particle Gibbs
Split-Merge sampler does not require the computation of a complex acceptance
ratio, is simple to implement using existing sequential Monte Carlo libraries
and can be parallelized. We investigate its performance experimentally on
synthetic problems as well as on geolocation and cancer genomics data. In all
these examples, the particle Gibbs split-merge sampler outperforms
state-of-the-art split-merge methods by up to an order of magnitude for a fixed
computational complexity
Variational approximation for mixtures of linear mixed models
Mixtures of linear mixed models (MLMMs) are useful for clustering grouped
data and can be estimated by likelihood maximization through the EM algorithm.
The conventional approach to determining a suitable number of components is to
compare different mixture models using penalized log-likelihood criteria such
as BIC.We propose fitting MLMMs with variational methods which can perform
parameter estimation and model selection simultaneously. A variational
approximation is described where the variational lower bound and parameter
updates are in closed form, allowing fast evaluation. A new variational greedy
algorithm is developed for model selection and learning of the mixture
components. This approach allows an automatic initialization of the algorithm
and returns a plausible number of mixture components automatically. In cases of
weak identifiability of certain model parameters, we use hierarchical centering
to reparametrize the model and show empirically that there is a gain in
efficiency by variational algorithms similar to that in MCMC algorithms.
Related to this, we prove that the approximate rate of convergence of
variational algorithms by Gaussian approximation is equal to that of the
corresponding Gibbs sampler which suggests that reparametrizations can lead to
improved convergence in variational algorithms as well.Comment: 36 pages, 5 figures, 2 tables, submitted to JCG
- …