Solution Path Algorithm for Twin Multi-class Support Vector Machine

The twin support vector machine and its extensions have made great achievements in dealing with binary classification problems, however, which is faced with some difficulties such as model selection and solving multi-classification problems quickly. This paper is devoted to the fast regularization parameter tuning algorithm for the twin multi-class support vector machine. A new sample dataset division method is adopted and the Lagrangian multipliers are proved to be piecewise linear with respect to the regularization parameters by combining the linear equations and block matrix theory. Eight kinds of events are defined to seek for the starting event and then the solution path algorithm is designed, which greatly reduces the computational cost. In addition, only few points are combined to complete the initialization and Lagrangian multipliers are proved to be 1 as the regularization parameter tends to infinity. Simulation results based on UCI datasets show that the proposed method can achieve good classification performance with reducing the computational cost of grid search method from exponential level to the constant level

A mathematical programming approach to SVM-based classification with label noise

The authors of this research acknowledge financial support by the Spanish Ministerio de Ciencia y Tecnologia, Agencia Estatal de Investigacion and Fondos Europeos de Desarrollo Regional (FEDER) via project PID2020114594GB-C21. The authors also acknowledge partial support from projects FEDER-US-1256951, Junta de Andalucía P18-FR-1422, CEI-3-FQM331, NetmeetData: Ayudas Fundación BBVA a equipos de investigación científica 2019. The first author was also supported by projects P18-FR-2369 (Junta de Andalucía) and IMAG-Maria de Maeztu grant CEX2020-001105-M /AEI /10.13039/501100011033. (Spanish Ministerio de Ciencia y Tecnologia).In this paper we propose novel methodologies to optimally construct Support Vector Machine-based classifiers that take into account that label noise occur in the training sample. We propose different alternatives based on solving Mixed Integer Linear and Non Linear models by incorporating decisions on relabeling some of the observations in the training dataset. The first method incorporates relabeling directly in the SVM model while a second family of methods combines clustering with classification at the same time, giving rise to a model that applies simultaneously similarity measures and SVM. Extensive computational experiments are reported based on a battery of standard datasets taken from UCI Machine Learning repository, showing the effectiveness of the proposed approaches.Spanish Ministerio de Ciencia y Tecnologia, Agencia Estatal de Investigacion and Fondos Europeos de Desarrollo Regional (FEDER) via project PID2020114594GB-C21FEDER-US-1256951Junta de Andalucía P18-FR-1422CEI-3-FQM331NetmeetData: Ayudas Fundación BBVA a equipos de investigación científica 2019Project P18-FR-2369 Junta de AndalucíaIMAG-Maria de Maeztu grant CEX2020-001105-M /AEI /10.13039/501100011033. (Spanish Ministerio de Ciencia y Tecnologia

Support matrix machine: A review

Support vector machine (SVM) is one of the most studied paradigms in the realm of machine learning for classification and regression problems. It relies on vectorized input data. However, a significant portion of the real-world data exists in matrix format, which is given as input to SVM by reshaping the matrices into vectors. The process of reshaping disrupts the spatial correlations inherent in the matrix data. Also, converting matrices into vectors results in input data with a high dimensionality, which introduces significant computational complexity. To overcome these issues in classifying matrix input data, support matrix machine (SMM) is proposed. It represents one of the emerging methodologies tailored for handling matrix input data. The SMM method preserves the structural information of the matrix data by using the spectral elastic net property which is a combination of the nuclear norm and Frobenius norm. This article provides the first in-depth analysis of the development of the SMM model, which can be used as a thorough summary by both novices and experts. We discuss numerous SMM variants, such as robust, sparse, class imbalance, and multi-class classification models. We also analyze the applications of the SMM model and conclude the article by outlining potential future research avenues and possibilities that may motivate academics to advance the SMM algorithm

LSTSVR-PI: Least square twin support vector regression with privileged information

In an educational setting, a teacher plays a crucial role in various classroom teaching patterns. Similarly, mirroring this aspect of human learning, the learning using privileged information (LUPI) paradigm introduces additional information to instruct learning models during the training stage. A different approach to train the twin variant of the regression model is provided by the new least square twin support vector regression using privileged information (LSTSVR-PI), which integrates the LUPI paradigm to utilize additional sources of information into the least square twin support vector regression. The proposed LSTSVR-PI solves system of linear equations which adds up to the efficiency of the model. Further, we also establish a generalization error bound based on the Rademacher complexity of the proposed model and incorporate the structural risk minimization principle. The proposed LSTSVR-PI fills the gap between the contemporary paradigm of LUPI and classical LSTSVR. Further, to assess the performance of the proposed model, we conduct numerical experiments along with the baseline models across various artificially generated and real-world datasets. The various experiments and statistical analysis infer the superiority of the proposed model. Moreover, as an application, we conduct experiments on time series datasets, which results in the superiority of the proposed LSTSVR-PI