MaxMin-L2-SVC-NCH: A New Method to Train Support Vector Classifier with the Selection of Model's Parameters

The selection of model's parameters plays an important role in the application of support vector classification (SVC). The commonly used method of selecting model's parameters is the k-fold cross validation with grid search (CV). It is extremely time-consuming because it needs to train a large number of SVC models. In this paper, a new method is proposed to train SVC with the selection of model's parameters. Firstly, training SVC with the selection of model's parameters is modeled as a minimax optimization problem (MaxMin-L2-SVC-NCH), in which the minimization problem is an optimization problem of finding the closest points between two normal convex hulls (L2-SVC-NCH) while the maximization problem is an optimization problem of finding the optimal model's parameters. A lower time complexity can be expected in MaxMin-L2-SVC-NCH because CV is abandoned. A gradient-based algorithm is then proposed to solve MaxMin-L2-SVC-NCH, in which L2-SVC-NCH is solved by a projected gradient algorithm (PGA) while the maximization problem is solved by a gradient ascent algorithm with dynamic learning rate. To demonstrate the advantages of the PGA in solving L2-SVC-NCH, we carry out a comparison of the PGA and the famous sequential minimal optimization (SMO) algorithm after a SMO algorithm and some KKT conditions for L2-SVC-NCH are provided. It is revealed that the SMO algorithm is a special case of the PGA. Thus, the PGA can provide more flexibility. The comparative experiments between MaxMin-L2-SVC-NCH and the classical parameter selection models on public datasets show that MaxMin-L2-SVC-NCH greatly reduces the number of models to be trained and the test accuracy is not lost to the classical models. It indicates that MaxMin-L2-SVC-NCH performs better than the other models. We strongly recommend MaxMin-L2-SVC-NCH as a preferred model for SVC task

SAMN: A Sample Attention Memory Network Combining SVM and NN in One Architecture

Support vector machine (SVM) and neural networks (NN) have strong complementarity. SVM focuses on the inner operation among samples while NN focuses on the operation among the features within samples. Thus, it is promising and attractive to combine SVM and NN, as it may provide a more powerful function than SVM or NN alone. However, current work on combining them lacks true integration. To address this, we propose a sample attention memory network (SAMN) that effectively combines SVM and NN by incorporating sample attention module, class prototypes, and memory block to NN. SVM can be viewed as a sample attention machine. It allows us to add a sample attention module to NN to implement the main function of SVM. Class prototypes are representatives of all classes, which can be viewed as alternatives to support vectors. The memory block is used for the storage and update of class prototypes. Class prototypes and memory block effectively reduce the computational cost of sample attention and make SAMN suitable for multi-classification tasks. Extensive experiments show that SAMN achieves better classification performance than single SVM or single NN with similar parameter sizes, as well as the previous best model for combining SVM and NN. The sample attention mechanism is a flexible module that can be easily deepened and incorporated into neural networks that require it

Rank space diversity: A diversity measure of base kernel matrices

This paper studies the diversity measure of base kernel matrices. First, rank space diversity is proposed as a diversity measure of base kernel matrices. Then, a rule for choosing base kernel matrices is deduced by this diversity measure. Last, our rule's validation is claimed by some experiments on artificial data set and benchmark data set

Accuracy of classifier combining based on majority voting

In this paper, we formulate the accuracy of classifier combining which is based on majority voting, there are only two parameter involved, one is the average accuracy of individual classifiers, the other we call it Lapsed Accuracy (LA) is related with the efficiency of classifier combining, and we discuss the theoretical bounds of majority voting via the formula

A study on piecewise polynomial smooth approximation to the plus function

In smooth support vector machine (SSVM), the plus function must be approximated by some smooth function, and the approximate error will affect the classification ability. This paper studies the smooth approximation to the plus function by piecewise polynomials. First, some standard piecewise polynomial smooth approximation problems are formulated. Then, the existence and uniqueness of solution for these problems are proved and the analytic solutions are achieved. The comparison between the results in this paper and the previous ones shows that the piecewise polynomial functions in this paper achieve better approximation to the plus function

A strategy of maximizing the sum of weighted margins for ranking multi classification problem

This paper discusses the strategies of maximizing the sum of margins for ranking multi classification problem. First, the strategy of maximizing the sum of margins (MSW is extended to maximizing the sum of weighted margins (MSWM). Using MSWM, a mathematical model is established to deal with the ranking multi classification problems where the importance of margins between classes is different, and its dual model is deduced. Then, by introducing the concept of algebraic margin, which is a generalization of geometric margin, the MSWM is further extended to maximizing the sum of weighed algebraic margins (MSWAM). Based on the MSWAM, the deduced mathematical model of the ranking multi classification problem not only has positive generalization ability, but is also a simple linear programming model

Background extraction algorithm based on K-means clustering algorithm and histogram analysis

Conference Name:2012 IEEE International Conference on Computer Science and Automation Engineering, CSAE 2012. Conference Address: Zhangjiajie, China. Time:May 25, 2012 - May 27, 2012.IEEE Beijing Section; Hunan University of Humanities, Science and Technology; Tongji University; Xiamen University; Central South UniversityBackground extraction is a fundamental task in many computer vision applications. This paper presents a novel method based on K-means Clustering Algorithm. First, the intensity values are divided into several groups by clustering method automatically. Then the mean of intensities in the most frequent group is assigned as the background intensity. The experiment results show that the proposed approach achieves more accurate background than traditional method. 漏 2012 IEEE