Extending twin support vector machine classifier for multi-category classification problems

© 2013 – IOS Press and the authors. All rights reservedTwin support vector machine classifier (TWSVM) was proposed by Jayadeva et al., which was used for binary classification problems. TWSVM not only overcomes the difficulties in handling the problem of exemplar unbalance in binary classification problems, but also it is four times faster in training a classifier than classical support vector machines. This paper proposes one-versus-all twin support vector machine classifiers (OVA-TWSVM) for multi-category classification problems by utilizing the strengths of TWSVM. OVA-TWSVM extends TWSVM to solve k-category classification problems by developing k TWSVM where in the ith TWSVM, we only solve the Quadratic Programming Problems (QPPs) for the ith class, and get the ith nonparallel hyperplane corresponding to the ith class data. OVA-TWSVM uses the well known one-versus-all (OVA) approach to construct a corresponding twin support vector machine classifier. We analyze the efficiency of the OVA-TWSVM theoretically, and perform experiments to test its efficiency on both synthetic data sets and several benchmark data sets from the UCI machine learning repository. Both the theoretical analysis and experimental results demonstrate that OVA-TWSVM can outperform the traditional OVA-SVMs classifier. Further experimental comparisons with other multiclass classifiers demonstrated that comparable performance could be achieved.This work is supported in part by the grant of the Fundamental Research Funds for the Central Universities of GK201102007 in PR China, and is also supported by Natural Science Basis Research Plan in Shaanxi Province of China (Program No.2010JM3004), and is at the same time supported by Chinese Academy of Sciences under the Innovative Group Overseas Partnership Grant as well as Natural Science Foundation of China Major International Joint Research Project (NO.71110107026)

nu-Anomica: A Fast Support Vector Based Novelty Detection Technique

In this paper we propose nu-Anomica, a novel anomaly detection technique that can be trained on huge data sets with much reduced running time compared to the benchmark one-class Support Vector Machines algorithm. In -Anomica, the idea is to train the machine such that it can provide a close approximation to the exact decision plane using fewer training points and without losing much of the generalization performance of the classical approach. We have tested the proposed algorithm on a variety of continuous data sets under different conditions. We show that under all test conditions the developed procedure closely preserves the accuracy of standard one-class Support Vector Machines while reducing both the training time and the test time by 5 - 20 times

Evaluation of machine learning classifiers in keratoconus detection from orbscan II examinations

PURPOSE: To evaluate the performance of support vector machine, multi-layer perceptron and radial basis function neural network as auxiliary tools to identify keratoconus from Orbscan II maps. METHODS: A total of 318 maps were selected and classified into four categories: normal (n = 172), astigmatism (n = 89), keratoconus (n = 46) and photorefractive keratectomy (n = 11). For each map, 11 attributes were obtained or calculated from data provided by the Orbscan II. Ten-fold cross-validation was used to train and test the classifiers. Besides accuracy, sensitivity and specificity, receiver operating characteristic (ROC) curves for each classifier were generated, and the areas under the curves were calculated. RESULTS: The three selected classifiers provided a good performance, and there were no differences between their performances. The area under the ROC curve of the support vector machine, multi-layer perceptron and radial basis function neural network were significantly larger than those for all individual Orbscan II attributes evaluated (p<0.05). CONCLUSION: Overall, the results suggest that using a support vector machine, multi-layer perceptron classifiers and radial basis function neural network, these classifiers, trained on Orbscan II data, could represent useful techniques for keratoconus detection

Support Vector Machines and Radon's Theorem

A support vector machine (SVM) is an algorithm which finds a hyperplane that optimally separates labeled data points in $\mathbb{R}^n$ into positive and negative classes. The data points on the margin of this separating hyperplane are called support vectors. We study the possible configurations of support vectors for points in general position. In particular, we connect the possible configurations to Radon's theorem, which provides guarantees for when a set of points can be divided into two classes (positive and negative) whose convex hulls intersect. If the positive and negative support vectors in a generic SVM configuration are projected to the separating hyperplane, then these projected points will form a Radon configuration. Further, with a particular type of general position, we show there are at most $n+1$ support vectors. This can be used to test the level of machine precision needed in a support vector machine implementation. We also show the projections of the convex hulls of the support vectors intersect in a single Radon point, and under a small enough perturbation, the points labeled as support vectors remain labeled as support vectors. We furthermore consider computations studying the expected number of support vectors for randomly generated data

A Comparative Study of the Effect of Sensor Noise on Activity Recognition Models

To provide a better understanding of the relative strengths of Machine Learning based Activity Recognition methods, in this paper we present a comparative analysis of the robustness of three popular methods with respect to sensor noise. Specifically we evaluate the robustness of Naive Bayes classifier, Support Vector Machine, and Random Forest based activity recognition models in three cases which span sensor errors from dead to poorly calibrated sensors. Test data is partially synthesized from a recently annotated activity recognition corpus which includes both interleaved activities and a range of both temporally long and short activities. Results demonstrate that the relative performance of Support Vector Machine classifiers over Naive Bayes classifiers reduces in noisy sensor conditions, but that overall the Random Forest classifier provides best activity recognition accuracy across all noise conditions synthesized in the corpus. Moreover, we find that activity recognition is equally robust across classification techniques with the relative performance of all models holding up under almost all sensor noise conditions considered

Implementation of Logistic Regression Classification Algorithm and Support Vector Machine for Credit Eligibility Prediction

Credit is a provision of money or bills that can be equated with it, the provision of loans or credit. A good credit analysis is very necessary, because it is one of the most important processes in the form of an investigation regarding the smooth or substandard credit repayments. The stages of identifying and predicting customers properly and correctly can be done before the loan process. This is done by examining the historical data of the customer's loan. At this time this activity is an effort made by the banking industry in dealing with credit risk problems. In this research, researchers will apply several data mining classification methods, including Logistic Regression algorithms and Support Vector Machines to predict creditworthiness. The dataset used 481 record motorized vehicle loan data, both problematic and non-problematic. The input variables in this study consisted of thirteen variables, including marital status, number of dependents, age, residence status, home ownership, occupation, employment status, company status, income, down payment, education, length of stay, and housing conditions. From the results of research and testing, the performance of the Logistic Regression model for predicting creditworthiness provided an accuracy rate of 94.81% with an area under the curve (AUC) value of 0.987. While the performance of the Support Vector Machine model provides an accuracy of 94.19% with an area under the curve (AUC) value of 0.978. Based on the T-Test test, the Logistic Regression method has the same performance compared to the Support Vector Machine

Stunting Classification in Children's Measurement Data Using Machine Learning Models

The study conducted a stunting classification of measurement data for children under 5 years old. The dataset has attributes such as: gender, age, weight (BB), height (TB), weight / height (BBTB), weight / age (BBU), and height / age (TBU). The research uses the CRISP-DM methodology in processing the data. The data were tested on several classification models, namely: logistic regression (LR), linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), k-nearest neighbor (KNN), classification and regression trees (CART), nave bayes (NB), support vector machine - linear kernel (SVM-Linear), support vector machine - rbf kernel (SVM-RBF), random forest classifier (RPC), adaboost (ADA), and neural network (MLPC). These models were tested on the dataset to find out the best model in accuracy. The test results show that SVM-RBF produces an accuracy of 78%. SVM-RBF has consistently been at the highest accuracy in several tests. Testing through k-fold cross validation with k=10

Component Outage Estimation based on Support Vector Machine

Predicting power system component outages in response to an imminent hurricane plays a major role in preevent planning and post-event recovery of the power system. An exact prediction of components states, however, is a challenging task and cannot be easily performed. In this paper, a Support Vector Machine (SVM) based method is proposed to help estimate the components states in response to anticipated path and intensity of an imminent hurricane. Components states are categorized into three classes of damaged, operational, and uncertain. The damaged components along with the components in uncertain class are then considered in multiple contingency scenarios of a proposed Event-driven Security-Constrained Unit Commitment (E-SCUC), which considers the simultaneous outage of multiple components under an N-m-u reliability criterion. Experimental results on the IEEE 118-bus test system show the merits and the effectiveness of the proposed SVM classifier and the E-SCUC model in improving power system resilience in response to extreme events