A Preliminary MML Linear Classifier using Principal Components for Multiple Classes

Abstract In this paper we improve on the supervised classification method developed inKornienko et al. (2002) by the introduction of Principal Components Analysis to the inference process. We also extend the classifier from dealing with binomial(two-class) problems only to multinomial (multi-class) problems. The application to which the MML criterion has been applied in this paperis the classification of objects via a linear hyperplane, where the objects are able to come from any multi-class distribution. The inclusion of Principal Compo-nent Analysis to the original inference scheme reduces the bias present in the classifier's search technique. Such improvements lead to a method which, whencompared against three commercial Support Vector Machine (SVM) classifiers on Binary data, was found to be as good as the most successful SVM tested. Fur-thermore, the new scheme is able to classify objects of a multiclass distribution with just one hyperplane, whereas SVMs require several hyperplanes. Content Areas Machine Learning, Knowledge discovery and data mining