1,001,725 research outputs found
Multidimensional Membership Mixture Models
We present the multidimensional membership mixture (M3) models where every
dimension of the membership represents an independent mixture model and each
data point is generated from the selected mixture components jointly. This is
helpful when the data has a certain shared structure. For example, three unique
means and three unique variances can effectively form a Gaussian mixture model
with nine components, while requiring only six parameters to fully describe it.
In this paper, we present three instantiations of M3 models (together with the
learning and inference algorithms): infinite, finite, and hybrid, depending on
whether the number of mixtures is fixed or not. They are built upon Dirichlet
process mixture models, latent Dirichlet allocation, and a combination
respectively. We then consider two applications: topic modeling and learning 3D
object arrangements. Our experiments show that our M3 models achieve better
performance using fewer topics than many classic topic models. We also observe
that topics from the different dimensions of M3 models are meaningful and
orthogonal to each other.Comment: 9 pages, 7 figure
Deep Gaussian Mixture Models
Deep learning is a hierarchical inference method formed by subsequent
multiple layers of learning able to more efficiently describe complex
relationships. In this work, Deep Gaussian Mixture Models are introduced and
discussed. A Deep Gaussian Mixture model (DGMM) is a network of multiple layers
of latent variables, where, at each layer, the variables follow a mixture of
Gaussian distributions. Thus, the deep mixture model consists of a set of
nested mixtures of linear models, which globally provide a nonlinear model able
to describe the data in a very flexible way. In order to avoid
overparameterized solutions, dimension reduction by factor models can be
applied at each layer of the architecture thus resulting in deep mixtures of
factor analysers.Comment: 19 pages, 4 figure
Adaptive Seeding for Gaussian Mixture Models
We present new initialization methods for the expectation-maximization
algorithm for multivariate Gaussian mixture models. Our methods are adaptions
of the well-known -means++ initialization and the Gonzalez algorithm.
Thereby we aim to close the gap between simple random, e.g. uniform, and
complex methods, that crucially depend on the right choice of hyperparameters.
Our extensive experiments indicate the usefulness of our methods compared to
common techniques and methods, which e.g. apply the original -means++ and
Gonzalez directly, with respect to artificial as well as real-world data sets.Comment: This is a preprint of a paper that has been accepted for publication
in the Proceedings of the 20th Pacific Asia Conference on Knowledge Discovery
and Data Mining (PAKDD) 2016. The final publication is available at
link.springer.com (http://link.springer.com/chapter/10.1007/978-3-319-31750-2
24
Testing for Homogeneity in Mixture Models
Statistical models of unobserved heterogeneity are typically formalized as
mixtures of simple parametric models and interest naturally focuses on testing
for homogeneity versus general mixture alternatives. Many tests of this type
can be interpreted as tests, as in Neyman (1959), and shown to be
locally, asymptotically optimal. These tests will be contrasted
with a new approach to likelihood ratio testing for general mixture models. The
latter tests are based on estimation of general nonparametric mixing
distribution with the Kiefer and Wolfowitz (1956) maximum likelihood estimator.
Recent developments in convex optimization have dramatically improved upon
earlier EM methods for computation of these estimators, and recent results on
the large sample behavior of likelihood ratios involving such estimators yield
a tractable form of asymptotic inference. Improvement in computation efficiency
also facilitates the use of a bootstrap methods to determine critical values
that are shown to work better than the asymptotic critical values in finite
samples. Consistency of the bootstrap procedure is also formally established.
We compare performance of the two approaches identifying circumstances in which
each is preferred
Identifiability of Large Phylogenetic Mixture Models
Phylogenetic mixture models are statistical models of character evolution
allowing for heterogeneity. Each of the classes in some unknown partition of
the characters may evolve by different processes, or even along different
trees. The fundamental question of whether parameters of such a model are
identifiable is difficult to address, due to the complexity of the
parameterization. We analyze mixture models on large trees, with many mixture
components, showing that both numerical and tree parameters are indeed
identifiable in these models when all trees are the same. We also explore the
extent to which our algebraic techniques can be employed to extend the result
to mixtures on different trees.Comment: 15 page
- …