15 research outputs found
From here to infinity - sparse finite versus Dirichlet process mixtures in model-based clustering
In model-based-clustering mixture models are used to group data points into
clusters. A useful concept introduced for Gaussian mixtures by Malsiner Walli
et al (2016) are sparse finite mixtures, where the prior distribution on the
weight distribution of a mixture with components is chosen in such a way
that a priori the number of clusters in the data is random and is allowed to be
smaller than with high probability. The number of cluster is then inferred
a posteriori from the data.
The present paper makes the following contributions in the context of sparse
finite mixture modelling. First, it is illustrated that the concept of sparse
finite mixture is very generic and easily extended to cluster various types of
non-Gaussian data, in particular discrete data and continuous multivariate data
arising from non-Gaussian clusters. Second, sparse finite mixtures are compared
to Dirichlet process mixtures with respect to their ability to identify the
number of clusters. For both model classes, a random hyper prior is considered
for the parameters determining the weight distribution. By suitable matching of
these priors, it is shown that the choice of this hyper prior is far more
influential on the cluster solution than whether a sparse finite mixture or a
Dirichlet process mixture is taken into consideration.Comment: Accepted versio
Effect fusion using model-based clustering
In social and economic studies many of the collected variables are measured
on a nominal scale, often with a large number of categories. The definition of
categories is usually not unambiguous and different classification schemes
using either a finer or a coarser grid are possible. Categorisation has an
impact when such a variable is included as covariate in a regression model: a
too fine grid will result in imprecise estimates of the corresponding effects,
whereas with a too coarse grid important effects will be missed, resulting in
biased effect estimates and poor predictive performance.
To achieve automatic grouping of levels with essentially the same effect, we
adopt a Bayesian approach and specify the prior on the level effects as a
location mixture of spiky normal components. Fusion of level effects is induced
by a prior on the mixture weights which encourages empty components.
Model-based clustering of the effects during MCMC sampling allows to
simultaneously detect categories which have essentially the same effect size
and identify variables with no effect at all. The properties of this approach
are investigated in simulation studies. Finally, the method is applied to
analyse effects of high-dimensional categorical predictors on income in
Austria
Semi-parametric Regression under Model Uncertainty: Economic Applications
Economic theory does not always specify the functional relationship between dependent and explanatory variables, or even isolate a particular set of covariates. This means that model uncertainty is pervasive in empirical economics. In this paper, we indicate how Bayesian semi-parametric regression methods in combination with stochastic search variable selection can be used to address two model uncertainties simultaneously: (i) the uncertainty with respect to the variables which should be included in the model and (ii) the uncertainty with respect to the functional form of their effects. The presented approach enables the simultaneous identification of robust linear and nonlinear effects. The additional insights gained are illustrated on applications in empirical economics, namely willingness to pay for housing, and cross-country growth regression
Identifying Mixtures of Mixtures Using Bayesian Estimation
The use of a finite mixture of normal distributions in model-based clustering
allows to capture non-Gaussian data clusters. However, identifying the clusters
from the normal components is challenging and in general either achieved by
imposing constraints on the model or by using post-processing procedures.
Within the Bayesian framework we propose a different approach based on sparse
finite mixtures to achieve identifiability. We specify a hierarchical prior
where the hyperparameters are carefully selected such that they are reflective
of the cluster structure aimed at. In addition this prior allows to estimate
the model using standard MCMC sampling methods. In combination with a
post-processing approach which resolves the label switching issue and results
in an identified model, our approach allows to simultaneously (1) determine the
number of clusters, (2) flexibly approximate the cluster distributions in a
semi-parametric way using finite mixtures of normals and (3) identify
cluster-specific parameters and classify observations. The proposed approach is
illustrated in two simulation studies and on benchmark data sets.Comment: 49 page
Bayesian Latent Class Analysis with Shrinkage Priors: An Application to the Hungarian Heart Disease Data
Latent class analysis explains dependency structures in multivariate categorical
data by assuming the presence of latent classes. We investigate the specification of suitable
priors for the Bayesian latent class model to determine the number of classes and perform
variable selection. Estimation is possible using standard tools implementing general purpose
Markov chain Monte Carlo sampling techniques such as the software JAGS. However, class
specific inference requires suitable post-processing in order to eliminate label switching. The
proposed Bayesian specification and analysis method is applied to the Hungarian heart disease
data set to determine the number of classes and identify relevant variables and results are
compared to those obtained with the standard prior for the component specific parameters
Model-based clustering based on sparse finite Gaussian mixtures
In the framework of Bayesian model-based clustering based on a finite mixture of Gaussian distributions, we present a joint approach to estimate the number of mixture components and identify cluster-relevant variables simultaneously as well as to obtain an identified model. Our approach consists in specifying sparse hierarchical priors on the mixture weights and component means. In a deliberately overfitting mixture model the sparse prior on the weights empties superfluous components during MCMC. A straightforward estimator for the true number of components is given by the most frequent number of non-empty components visited during MCMC sampling. Specifying a shrinkage prior, namely the normal gamma prior, on the component means leads to improved parameter estimates as well as identification of cluster-relevant variables. After estimating the mixture model using MCMC methods based on data augmentation and Gibbs sampling, an identified model is obtained by relabeling the MCMC output in the point process representation of the draws. This is performed using K-centroids cluster analysis based on the Mahalanobis distance. We evaluate our proposed strategy in a simulation setup with artificial data and by applying it to benchmark data sets. (authors' abstract
Model-based clustering based on sparse finite Gaussian mixtures
In the framework of Bayesian model-based clustering based on a finite mixture of Gaussian distributions, we present a joint approach to estimate the number of mixture components and identify cluster-relevant variables simultaneously as well as to obtain an identified model. Our approach consists in specifying sparse hierarchical priors on the mixture weights and component means. In a deliberately overfitting mixture model the sparse prior on the weights empties superfluous components during MCMC. A straightforward estimator for the true number of components is given by the most frequent number of non-empty components visited during MCMC sampling. Specifying a shrinkage prior, namely the normal gamma prior, on the component means leads to improved parameter estimates as well as identification of cluster-relevant variables. After estimating the mixture model using MCMC methods based on data augmentation and Gibbs sampling, an identified model is obtained by relabeling the MCMC output in the point process representation of the draws. This is performed using K-centroids cluster analysis based on the Mahalanobis distance. We evaluate our proposed strategy in a simulation setup with artificial data and by applying it to benchmark data sets. (authors' abstract