Bayesian Feature Selection for Clustering Problems

Bayesian methods have been successfully used for feature selection in many supervised learning tasks. In this paper, the adaptation of such methods for unsupervised learning (clustering) is investigated. We adopt an algorithm that iterates between clustering (assuming that the number of clusters is unknown a priori) and feature selection. From this standpoint, two Bayesian approaches for feature selection are addressed: (i) Naïve Bayes Wrapper (NBW), and (ii) Markov Blanket Filter (MBF) obtained from the construction of Bayesian networks. Experiments in ten datasets illustrate the performance of each proposed method. Advantages of feature selection are demonstrated by comparing the results obtained from Bayesian feature selection with the results achieved without any kind of feature selection, i.e., using all the available features. In most of the performed experiments, NBW and MBF have allowed reducing the number of features, while providing good quality partitions in relation to those found by means of the full set of features. Also, NBW has outperformed its Bayesian feature selection counterpart (MBF) in most of the assessed datasets, mainly when the cardinality of the selected feature subset is taken into consideration.Feature selection, clustering, Naïve Bayes, Bayesian networks