Sequential Monte Carlo Samplers For Nonparametric Bayesian Mixture Models

Tez (Doktora) -- İstanbul Teknik Üniversitesi, Fen Bilimleri Enstitüsü, 2012Thesis (PhD) -- İstanbul Technical University, Institute of Science and Technology, 2012Bu çalışmanın temel amacı, parametrik olmayan Bayesçi model seçim teknikleri içinde önemli bir yere sahip olan Dirichlet süreci karışım modelleri (DPM) için etkin ardışık Monte Carlo (SMC) örnekleyiciler tasarlamaktır. Tasarlanan algoritmalar, önerilen sınıf güncelleme metotları sayesinde, yeni gelen gözlemlerin ışığında parçacık gezingelerinde değişiklik yaparak gerçek DPM sonsal dağılımına daha iyi bir yaklaşıklık sağlamaktadır. Önerilen metot, DPM sonsal dağılımının çözümünde kullanılan diğer ardışık Monte Carlo örnekleyicileri genelleme özelliğe sahiptir. Tek ve çok boyutlu olasılık dağılımı kestirim problemlerinde yapılan değerlendirmelerde, özellikle sonsal dağılımın izole modlara sahip olduğu koşullarda, önerilen metodun klasik metotlara göre çok daha yüksek doğrulukta sonuca yakınsayabildiği görülmüştür. Ayrıca, manevralı hedeflerin takibinde ortaya atılan en yenilikçi modellerden biri olan değişken oranlı parçacık süzgeçleri (VRPF) tezde ele alınmış ve çoklu model yaklaşımları değişken oranlı modeller ile birleştirilerek, takip başarımını arttıran çoklu model değişken oranlı parçacık süzgeçleri (MM-VRPF) önerilmiştir. Çoklu model yaklaşımının manevralı hedef gezingelerini daha iyi modellediği benzetim sonuçları ile gösterilmiştir.In this thesis, we developed a novel online algorithm for posterior inference in Dirichlet Process Mixture (DPM) models that is based on the sequential Monte Carlo (SMC) samplers framework. The proposed method enables us to design new clustering update schemes, such as updating past trajectories of the particles in light of recent observations, and still ensures convergence to the true DPM posterior distribution asymptotically. Our method generalizes many sequential importance sampling based approaches and provides a computationally efficient improvement to particle filtering that is less prone to getting trapped in isolated modes. Performance has been evaluated in univariate and multivariate infinite Gaussian mixture density estimation problems. It is shown that the proposed algorithm outperforms conventional Monte Carlo approaches in terms of estimation variance and average log-marginal. Moreover, in this thesis we dealt with the maneuvering target tracking problem. We incorporated multiple model approach with the recently introduced variable rate particle filters (VRPF) in order to improve the tracking performance. The proposed variable rate model structure, referred as Multiple Model Variable Rate Particle Filter (MM-VRPF) results in a much more accurate tracking.DoktoraPh

Sampling from Dirichlet process mixture models with unknown concentration parameter: mixing issues in large data implementations

We consider the question of Markov chain Monte Carlo sampling from a general stick-breaking Dirichlet process mixture model, with concentration parameter (Formula presented.). This paper introduces a Gibbs sampling algorithm that combines the slice sampling approach of Walker (Communications in Statistics - Simulation and Computation 36:45-54, 2007) and the retrospective sampling approach of Papaspiliopoulos and Roberts (Biometrika 95(1):169-186, 2008). Our general algorithm is implemented as efficient open source C++ software, available as an R package, and is based on a blocking strategy similar to that suggested by Papaspiliopoulos (A note on posterior sampling from Dirichlet mixture models, 2008) and implemented by Yau et al. (Journal of the Royal Statistical Society, Series B (Statistical Methodology) 73:37-57, 2011). We discuss the difficulties of achieving good mixing in MCMC samplers of this nature in large data sets and investigate sensitivity to initialisation. We additionally consider the challenges when an additional layer of hierarchy is added such that joint inference is to be made on (Formula presented.). We introduce a new label-switching move and compute the marginal partition posterior to help to surmount these difficulties. Our work is illustrated using a profile regression (Molitor et al. Biostatistics 11(3):484-498, 2010) application, where we demonstrate good mixing behaviour for both synthetic and real examples. © 2014 The Author(s)

A sequential Monte Carlo algorithm with transformations for Bayesian model exploration: applications in population genetics

Given a statistical model that attempts to explain the data, calculating the Bayes’ posterior distribution of the models parameters is desirable. The marginal likelihood of the model is also of interest, which is used for model comparison. However, for most applications, only estimates of these two measurements can be obtained with a class of methods that give consistent estimates being Monte Carlo algorithms. This thesis attempts to improve both the process in inferring a high-dimensional posterior distribution and the corresponding model marginal likelihood, on the condition that we can define an ordered set of statistical models in which deterministic transformations between each adjacent model can be applied. We propose an adaption of the sequential Monte Carlo algorithm, which we term the “transformation Sequential Monte Carlo” algorithm. The key feature of this algorithm is by defining a series of target distributions, that make use of said mentioned model transformations, we aim to infer high dimensional models by using easier to estimate posteriors from lower dimensional models with a model transformation applied. Our proposed algorithm has advantages over many established MC methods. One notable advantage is that we can tailor the algorithm if we wish to update a posterior distribution by including additional observations, but these observations also correspond to a new parameter set that needs to be inferred. Alternatively it is useful where the parameter space can become too large to explore using basic MC methods, for example if there exists an exponential or factorial relationship with observation size and the number of discrete values, but using a lower dimensional model and incorporating it into the model exploration assists with convergence. We test these strengths of tSMC under three applications, which include two population genetics applications being ancestral reconstruction under the coalescent and the other being the Structure algorithm

Quasi-Newton Sequential Monte Carlo

Sequential Monte Carlo samplers represent a compelling approach to posterior inference in Bayesian models, due to being parallelisable and providing an unbiased estimate of the posterior normalising constant. In this work, we significantly accelerate sequential Monte Carlo samplers by adopting the L-BFGS Hessian approximation which represents the state-of-the-art in full-batch optimisation techniques. The L-BFGS Hessian approximation has only linear complexity in the parameter dimension and requires no additional posterior or gradient evaluations. The resulting sequential Monte Carlo algorithm is adaptive, parallelisable and well-suited to high-dimensional and multi-modal settings, which we demonstrate in numerical experiments on challenging posterior distributions