Mixtures of Skew-t Factor Analyzers

In this paper, we introduce a mixture of skew-t factor analyzers as well as a family of mixture models based thereon. The mixture of skew-t distributions model that we use arises as a limiting case of the mixture of generalized hyperbolic distributions. Like their Gaussian and t-distribution analogues, our mixture of skew-t factor analyzers are very well-suited to the model-based clustering of high-dimensional data. Imposing constraints on components of the decomposed covariance parameter results in the development of eight flexible models. The alternating expectation-conditional maximization algorithm is used for model parameter estimation and the Bayesian information criterion is used for model selection. The models are applied to both real and simulated data, giving superior clustering results compared to a well-established family of Gaussian mixture models

Parameter estimation of Gaussian hierarchical model using Gibbs sampling

Gibbs sampling method is an important tool used in parameter estimation for many probabilistic models. Specifically, for many scenarios, it is difficult to generate high-dimensional data samples from its joint distribution. The Gibbs sampling provides a way to draw high-dimensional data via the conditional distributions which are typically easier to sample. In this thesis, we study a simple generative model called Hierarchical Gaussian and an efficient method for computing its parameters using Gibbs sampling. In particular, we show that the Hierarchical Gaussian model admits closed form conditional distributions such that Gibbs sampling can be used effectively to draw the samples from the joint distribution, and perform parameter estimation

Inverse Covariance Estimation for High-Dimensional Data in Linear Time and Space: Spectral Methods for Riccati and Sparse Models

We propose maximum likelihood estimation for learning Gaussian graphical models with a Gaussian (ell_2^2) prior on the parameters. This is in contrast to the commonly used Laplace (ell_1) prior for encouraging sparseness. We show that our optimization problem leads to a Riccati matrix equation, which has a closed form solution. We propose an efficient algorithm that performs a singular value decomposition of the training data. Our algorithm is O(NT^2)-time and O(NT)-space for N variables and T samples. Our method is tailored to high-dimensional problems (N gg T), in which sparseness promoting methods become intractable. Furthermore, instead of obtaining a single solution for a specific regularization parameter, our algorithm finds the whole solution path. We show that the method has logarithmic sample complexity under the spiked covariance model. We also propose sparsification of the dense solution with provable performance guarantees. We provide techniques for using our learnt models, such as removing unimportant variables, computing likelihoods and conditional distributions. Finally, we show promising results in several gene expressions datasets.Comment: Appears in Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence (UAI2013

Efficient Cosmological Parameter Estimation with Hamiltonian Monte Carlo

Traditional Markov Chain Monte Carlo methods suffer from low acceptance rate, slow mixing and low efficiency in high dimensions. Hamiltonian Monte Carlo resolves this issue by avoiding the random walk. Hamiltonian Monte Carlo (HMC) is a Markov chain Monte Carlo (MCMC) technique built upon the basic principle of Hamiltonian mechanics. Hamiltonian dynamics allows the chain to move along trajectories of constant energy, taking large jumps in the parameter space with relatively inexpensive computations. This new technique improves the acceptance rate by a factor of 4 and boosts up the efficiency by at least a factor of D in a D-dimensional parameter space. Therefor shorter chains will be needed for a reliable parameter estimation comparing to a traditional MCMC chain yielding the same performance. Besides that, the HMC is well suited for sampling from non-Gaussian and curved distributions which are very hard to sample from using the traditional MCMC methods. The method is very simple to code and can be easily plugged into standard parameter estimation codes such as CosmoMC. In this paper we demonstrate how the HMC can be efficiently used in cosmological parameter estimation