Model-based clustering for conditionally correlated categorical data

International audienceAn extension of the latent class model is presented for clustering categorical data by relaxing the classical ''class conditional independence assumption'' of variables. This model consists in grouping the variables into inter-independent and intra-dependent blocks, in order to consider the main intra-class correlations. The dependency between variables grouped inside the same block of a class is taken into account by mixing two extreme distributions, which are respectively the independence and the maximum dependency. When the variables are dependent given the class, this approach is expected to reduce the biases of the latent class model. Indeed, it produces a meaningful dependency model with only a few additional parameters. The parameters are estimated, by maximum likelihood, by means of an EM algorithm. Moreover, a Gibbs sampler is used for model selection in order to overcome the computational intractability of the combinatorial problems involved by the block structure search. Two applications on medical and biological data sets show the relevance of this new model. The results strengthen the view that this model is meaningful and that it reduces the biases induced by the conditional independence assumption of the latent class model.Nous proposons une extension du modèle des classes latentes pour la classification non supervisée de données catégorielles conditionnellement corrélées. Dans ce modèle, les variables sont regroupées en blocs inter-indépendants et intra-dépendants dans le but de prendre en compte les principales corrélations intra-classes. La dépendance entre les variables d'un même bloc est prise en compte par un mélange de deux distributions extrêmes, qui sont celles d'indépendance et de dépendance maximale. Dans le cas de données conditionnellement corrélées, on s'attend à ce que cette approche réduise les biais induits par le modèle des classes latentes et qu'il produise un modèle de dépendance facilement interprétable nécessitant peu de paramètres supplémentaires. L'estimation de ces derniers par maximum de vraisemblance est effectuée par un algorithme EM alors qu'un algorithme de Gibbs, permettant de résoudre les problèmes combinatoires dus à la recherche des blocs, est utilisé pour la sélection de modèle. Des applications sur des données sociologiques et biologiques permettent de mettre en avant l'intérêt du modèle proposé. Leurs résultats confortent l'idée que celui-ci est facilement interprétable et qu'il réduit les biais du modèle des classes latentes dus à l'hypothèse d'indépendance conditionnelle

Exploring dependence between categorical variables: benefits and limitations of using variable selection within Bayesian clustering in relation to log-linear modelling with interaction terms

This manuscript is concerned with relating two approaches that can be used to explore complex dependence structures between categorical variables, namely Bayesian partitioning of the covariate space incorporating a variable selection procedure that highlights the covariates that drive the clustering, and log-linear modelling with interaction terms. We derive theoretical results on this relation and discuss if they can be employed to assist log-linear model determination, demonstrating advantages and limitations with simulated and real data sets. The main advantage concerns sparse contingency tables. Inferences from clustering can potentially reduce the number of covariates considered and, subsequently, the number of competing log-linear models, making the exploration of the model space feasible. Variable selection within clustering can inform on marginal independence in general, thus allowing for a more efficient exploration of the log-linear model space. However, we show that the clustering structure is not informative on the existence of interactions in a consistent manner. This work is of interest to those who utilize log-linear models, as well as practitioners such as epidemiologists that use clustering models to reduce the dimensionality in the data and to reveal interesting patterns on how covariates combine.Comment: Preprin