Computational and Inferential Difficulties with Mixture Posterior Distributions

Abstract

This paper deals with both exploration and interpretation problems related to posterior distributions for mixture models. The specification of mixture posterior distributions means that the presence of k! modes is known immediate- ly. Standard Markov chain Monte Carlo techniques usually have difficulties with well-separated modes such as occur here; the Markov chain Monte Carlo sampler stays within a neighbourhood of a local mode and fails to visit other equally important modes. We show that exploration of these modes can be imposed on the Markov chain Monte Carlo sampler using tempered transitions based on Langevin algorithms. However, as the prior distribution does not distinguish between the different components, the posterior mixture distribution is symmetric and thus standard estimators such as posterior means cannot be used. Since this is also true for most non-symmetric priors, we propose alternatives for Bayesian inference for permutation invariant posteriors, including a clustering device and the call to appropriate loss functions. An important side-issue of this study is the highlighting of the flexibility and adaptability of Langevin Metropolis-Hastings algorithms as quasi-automated Markov chain Monte Carlo algorithms when the posterior distribution is known up to a constant