A hierarchical VQSVM for imbalanced data sets

First, a hierarchical modelling method, VQSVM, is introduced, and some remarks are discussed. Secondly the proposed VQSVM is applied to a nonstandard learning environment, imbalanced data sets. In cases of extremely imbalanced dataset with high dimensions, standard machine learning techniques tend to be overwhelmed by the large classes. The hierarchical VQSVM contains a set of local models i.e. codevectors produced by the Vector Quantization and a global model, i.e. Support Vector Machine, to rebalance datasets without significant information loss. Some issues, e.g. distortion and support vectors, have been discussed to address the trade-off between the information loss and undersampling rate. Experiments compare VQSVM with random resampling techniques on some imbalanced datasets with varied imbalance ratios, and results show that the performance of VQSVM is superior or equivalent to random resampling techniques, especially in case of extremely imbalanced large datasets. ©2007 IEEE

VQSVM: A case study for incorporating prior domain knowledge into inductive machine learning

When dealing with real-world problems, there is considerable amount of prior domain knowledge that can provide insights on various aspect of the problem. On the other hand, many machine learning methods rely solely on the data sets for their learning phase and do not take into account any explicitly expressed domain knowledge. This paper proposes a framework that investigates and enables the incorporation of prior domain knowledge with respect to three key characteristics of inductive machine learning algorithms: consistency, generalization and convergence. The framework is used to review, classify and analyse key existing approaches to incorporating domain knowledge into inductive machine learning, as well as to consider the risks of doing so. The paper also demonstrates the design of a novel hierarchical semi-parametric machine learning method, capable of incorporating prior domain knowledge. The method-VQSVM-extends the support vector machine (SVM) family of methods with vector quantization (VQ) techniques to address the problem of learning from imbalanced data sets. The paper presents the results of testing the method on a collection of imbalanced data sets with various imbalance ratios and various numbers of subclasses. The learning process of the VQSVM method utilizes some domain knowledge to solve problem of fitting imbalance data. The experiments in the paper demonstrate that enabling the incorporation of prior domain knowledge into the SVM framework is an effective way to overcome the sensitivity of SVM towards the imbalance ratio in a data set. © 2010 Elsevier B.V