Reinforced Angle-Based Multicategory Support Vector Machines

The Support Vector Machine (SVM) is a very popular classification tool with many successful applications. It was originally designed for binary problems with desirable theoretical properties. Although there exist various Multicategory SVM (MSVM) extensions in the literature, some challenges remain. In particular, most existing MSVMs make use of k classification functions for a k-class problem, and the corresponding optimization problems are typically handled by existing quadratic programming solvers. In this paper, we propose a new group of MSVMs, namely the Reinforced Angle-based MSVMs (RAMSVMs), using an angle-based prediction rule with k − 1 functions directly. We prove that RAMSVMs can enjoy Fisher consistency. Moreover, we show that the RAMSVM can be implemented using the very efficient coordinate descent algorithm on its dual problem. Numerical experiments demonstrate that our method is highly competitive in terms of computational speed, as well as classification prediction performance. Supplemental materials for the article are available online

Multicategory angle-based large-margin classification

Large-margin classifiers are popular methods for classification. Among existing simultaneous multicategory large-margin classifiers, a common approach is to learn k different functions for a k-class problem with a sum-to-zero constraint. Such a formulation can be inefficient. We propose a new multicategory angle-based large-margin classification framework. The proposed angle-based classifiers consider a simplex-based prediction rule without the sum-to-zero constraint, and enjoy more efficient computation. Many binary large-margin classifiers can be naturally generalized for multicategory problems through the angle-based framework. Theoretical and numerical studies demonstrate the usefulness of the angle-based methods

Flexible Classification Techniques with Biomedical Applications

Classification problems are prevalent in many scientific disciplines, especially in biomedical research. Recently, margin based classifiers have become increasingly popular, partly due to their ability in handling large scale problems with fast computational speed and desirable theoretical properties. Despite the success of margin based classifiers, many challenges remain. For example, in practical problems, it can be desirable to estimate the class conditional probability accurately. For high dimensional classification data, penalized margin based classifiers are commonly used. However, when estimating the class conditional probability, the shrinkage effect from the penalty term in the corresponding optimization is often ignored. This effect can lead to large bias in estimation of the class conditional probability. Another important issue on classification is the comparison between soft and hard classifiers for multicategory problems. Moreover, regular multicategory margin based classifiers can suffer from inefficiency by using too many classification functions. In this dissertation, we propose several new classification techniques to overcome the challenges mentioned above. Comprehensive numerical and theoretical studies are presented to demonstrate the usefulness of our new proposed methodologies.Doctor of Philosoph