47,481 research outputs found
Multidimensional Membership Mixture Models
We present the multidimensional membership mixture (M3) models where every
dimension of the membership represents an independent mixture model and each
data point is generated from the selected mixture components jointly. This is
helpful when the data has a certain shared structure. For example, three unique
means and three unique variances can effectively form a Gaussian mixture model
with nine components, while requiring only six parameters to fully describe it.
In this paper, we present three instantiations of M3 models (together with the
learning and inference algorithms): infinite, finite, and hybrid, depending on
whether the number of mixtures is fixed or not. They are built upon Dirichlet
process mixture models, latent Dirichlet allocation, and a combination
respectively. We then consider two applications: topic modeling and learning 3D
object arrangements. Our experiments show that our M3 models achieve better
performance using fewer topics than many classic topic models. We also observe
that topics from the different dimensions of M3 models are meaningful and
orthogonal to each other.Comment: 9 pages, 7 figure
The Importance of Being Clustered: Uncluttering the Trends of Statistics from 1970 to 2015
In this paper we retrace the recent history of statistics by analyzing all
the papers published in five prestigious statistical journals since 1970,
namely: Annals of Statistics, Biometrika, Journal of the American Statistical
Association, Journal of the Royal Statistical Society, series B and Statistical
Science. The aim is to construct a kind of "taxonomy" of the statistical papers
by organizing and by clustering them in main themes. In this sense being
identified in a cluster means being important enough to be uncluttered in the
vast and interconnected world of the statistical research. Since the main
statistical research topics naturally born, evolve or die during time, we will
also develop a dynamic clustering strategy, where a group in a time period is
allowed to migrate or to merge into different groups in the following one.
Results show that statistics is a very dynamic and evolving science, stimulated
by the rise of new research questions and types of data
The supervised hierarchical Dirichlet process
We propose the supervised hierarchical Dirichlet process (sHDP), a
nonparametric generative model for the joint distribution of a group of
observations and a response variable directly associated with that whole group.
We compare the sHDP with another leading method for regression on grouped data,
the supervised latent Dirichlet allocation (sLDA) model. We evaluate our method
on two real-world classification problems and two real-world regression
problems. Bayesian nonparametric regression models based on the Dirichlet
process, such as the Dirichlet process-generalised linear models (DP-GLM) have
previously been explored; these models allow flexibility in modelling nonlinear
relationships. However, until now, Hierarchical Dirichlet Process (HDP)
mixtures have not seen significant use in supervised problems with grouped data
since a straightforward application of the HDP on the grouped data results in
learnt clusters that are not predictive of the responses. The sHDP solves this
problem by allowing for clusters to be learnt jointly from the group structure
and from the label assigned to each group.Comment: 14 page
On the use of reproducing kernel Hilbert spaces in functional classification
The H\'ajek-Feldman dichotomy establishes that two Gaussian measures are
either mutually absolutely continuous with respect to each other (and hence
there is a Radon-Nikodym density for each measure with respect to the other
one) or mutually singular. Unlike the case of finite dimensional Gaussian
measures, there are non-trivial examples of both situations when dealing with
Gaussian stochastic processes. This paper provides:
(a) Explicit expressions for the optimal (Bayes) rule and the minimal
classification error probability in several relevant problems of supervised
binary classification of mutually absolutely continuous Gaussian processes. The
approach relies on some classical results in the theory of Reproducing Kernel
Hilbert Spaces (RKHS).
(b) An interpretation, in terms of mutual singularity, for the "near perfect
classification" phenomenon described by Delaigle and Hall (2012). We show that
the asymptotically optimal rule proposed by these authors can be identified
with the sequence of optimal rules for an approximating sequence of
classification problems in the absolutely continuous case.
(c) A new model-based method for variable selection in binary classification
problems, which arises in a very natural way from the explicit knowledge of the
RN-derivatives and the underlying RKHS structure. Different classifiers might
be used from the selected variables. In particular, the classical, linear
finite-dimensional Fisher rule turns out to be consistent under some standard
conditions on the underlying functional model
- …