Learning Selective Averaged One-Dependence Estimators for Probability Estimation

Naïve Bayes is a well-known effective and efficient classification algorithm, but its probability estimation performance is poor. Averaged One-Dependence Estimators, simply AODE, is a recently proposed semi-naïve Bayes algorithm and demonstrates significantly high classification accuracy at a modest cost. In many data mining applications, however, accurate probability estimation is more desirable when making optimal decisions. Usually, probability estimation performance is measured by conditional log likelihood (CLL). In this paper, we first study the probability estimation performance of AODE and compare it to naïve Bayes, Tree-Augumented naïve Bayes, CLLTree, C4.4 (the improved version of C4.5 for better probability estimation) and Support Vector Machines. From our experiments, we find that AODE performs significantly better than the algorithms used to compare except C4.4, and performs slightly better than C4.4 although its classification accuracy is significantly better than C4.5. We then propose an efficient forward greedy feature selection algorithm for AODE and use the CLL score for attribute selection. The experimental results show that our algorithm achieves substantially improvement over AODE and significantly outperforms C4.4. Our experiments are conducted on the basis of 36 UCI data sets that cover a wide range of domains and data characteristics and we run all the algorithms within the Weka platform. 1