Support Vector Selection for Regression Machines

In this paper, we propose a method to select support vectors to improve the performance of support vector regression machines. First, the orthogonal least-squares method is adopted to evaluate the support vectors based on their error reduction ratios. By selecting the representative support vectors, we can obtain a simpler model which helps avoid the over-fitting problem. Second, the simplified model is further refined by applying the gradient descent method to tune the parameters of the kernel functions. Learning rules for minimizing the regularized risk functional are derived. Experimental results have shown that our approach can improve effectively the generalization capability of support vector regressors

Nonstationary regression with support vector machines

In this work, we introduce a method for data analysis in nonstationary environments: time-adaptive support vector regression (TA-SVR). The proposed approach extends a previous development that was limited to classification problems. Focusing our study on time series applications, we show that TA-SVR can improve the accuracy of several aspects of nonstationary data analysis, namely the tasks of modelling and prediction, input relevance estimation, and reconstruction of a hidden forcing profile.Fil: Uzal, Lucas César. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Rosario. Centro Internacional Franco Argentino de Ciencias de la Información y Sistemas; ArgentinaFil: Grinblat, Guillermo Luis. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Rosario. Centro Internacional Franco Argentino de Ciencias de la Información y Sistemas; ArgentinaFil: Granitto, Pablo Miguel. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Rosario. Centro Internacional Franco Argentino de Ciencias de la Información y Sistemas; ArgentinaFil: Verdes, Pablo Fabian. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Rosario. Centro Internacional Franco Argentino de Ciencias de la Información y Sistemas; Argentin

On-line support vector machines for function approximation

This paper describes an on-line method for building epsilon-insensitive support vector machines for regression as described in (Vapnik, 1995). The method is an extension of the method developed by (Cauwenberghs & Poggio, 2000) for building incremental support vector machines for classification. Machines obtained by using this approach are equivalent to the ones obtained by applying exact methods like quadratic programming, but they are obtained more quickly and allow the incremental addition of new points, removal of existing points and update of target values for existing data. This development opens the application of SVM regression to areas such as on-line prediction of temporal series or generalization of value functions in reinforcement learning.Postprint (published version

From Regression to Classification in Support Vector Machines

We study the relation between support vector machines (SVMs) for regression (SVMR) and SVM for classification (SVMC). We show that for a given SVMC solution there exists a SVMR solution which is equivalent for a certain choice of the parameters. In particular our result is that for $epsilon$ sufficiently close to one, the optimal hyperplane and threshold for the SVMC problem with regularization parameter C_c are equal to (1-epsilon)^{- 1} times the optimal hyperplane and threshold for SVMR with regularization parameter C_r = (1-epsilon)C_c. A direct consequence of this result is that SVMC can be seen as a special case of SVMR

Using a multi-objective genetic algorithm for SVM construction

Support Vector Machines are kernel machines useful for classification and regression problems. In this paper, they are used for non-linear regression of environmental data. From a structural point of view, Support Vector Machines are particular Artificial Neural Networks and their training paradigm has some positive implications. In fact, the original training approach is useful to overcome the curse of dimensionality and too strict assumptions on statistics of the errors in data. Support Vector Machines and Radial Basis Function Regularised Networks are presented within a common structural framework for non-linear regression in order to emphasise the training strategy for support vector machines and to better explain the multi-objective approach in support vector machines' construction. A support vector machine's performance depends on the kernel parameter, input selection and ε-tube optimal dimension. These will be used as decision variables for the evolutionary strategy based on a Genetic Algorithm, which exhibits the number of support vectors, for the capacity of machine, and the fitness to a validation subset, for the model accuracy in mapping the underlying physical phenomena, as objective functions. The strategy is tested on a case study dealing with groundwater modelling, based on time series (past measured rainfalls and levels) for level predictions at variable time horizons

Support Vector Machines in Classification and Regression Problems

117 σ.Η μηχανική μάθηση έχει ως στόχο τη δημιουργία αλγορίθμων ικανών να βελτιώνουν την απόδοση τους, αξιοποιώντας προγενέστερη γνώση και εμπειρία, με σκοπό την εξαγωγή χρήσιμων συμπερασμάτων και την περιγραφή φαινομένων, μέσω της επεξεργασίας δεδομένων τεράστιου, πολλές φορές, όγκου. Το ζητούμενο στην περίπτωση της επιβλεπόμενης μάθησης είναι η κατασκευή ενός μοντέλου που αναπαριστά τη γνώστη που αποκτήθηκε μέσω της εμπειρίας και το οποίο στη συνέχεια χρησιμοποιείται για την αξιολόγηση νέων παρατηρήσεων. Μία από τις πιο οικείες μεθόδους περιγραφής φαινομένων είναι η ταξινόμηση, η ένταξη δηλαδή κάθε παρατήρησης σε μία ομάδα, από ένα πεπερασμένο πλήθος υποψήφιων ομάδων. Η παρούσα εργασία επικεντρώνεται στην παρουσίαση ενός πολύ διαδεδομένου αλγόριθμου ταξινόμησης, προερχόμενου από τον τομέα της μηχανικής μάθησης, με όνομα «Μηχανή Διανυσματικής Υποστήριξης» (Support Vector Machine - SVM). Η ανάπτυξη του θεωρητικού υπόβαθρου του αλγόριθμου παρουσιάζεται σταδιακά, ώστε να γίνει κατανοητή από τον αναγνώστη όλη η διαδρομή. Πιο συγκεκριμένα, το πρώτο κεφάλαιο αποτελεί μια εισαγωγή στους αλγόριθμους εξόρυξης δεδομένων (Data Mining) και σε σχετικές εφαρμογές αυτών. Στο δεύτερο κεφάλαιο παρουσιάζονται οι θεμελιώδεις έννοιες που απαιτούνται για την κατανόηση των SVMs. Στο τρίτο και στο τέταρτο κεφάλαιο γίνεται μία λεπτομερής αναφορά στις Μηχανές Διανυσματικής Υποστήριξης και στην Παλινδρόμηση με SVM, αντίστοιχα. Στη συνέχεια, στο πέμπτο κεφάλαιο παρουσιάζουμε τις μεθόδους αξιολόγησης του μοντέλου ενώ στο έκτο κεφάλαιο κάνουμε μία μικρή αναφορά στην επιλογή χαρακτηριστικών με SVM. Στο έβδομο και τελευταίο κεφάλαιο παρουσιάζουμε τρεις εφαρμογές καθώς και την ερμηνεία των αντίστοιχων αποτελεσμάτων, με σκοπό να αξιολογήσουμε τη γνώση που αποκτήσαμε.The aim of machine learning is to develop algorithms capable of improving their own performance, exploiting existing data, stored in huge databases, in order to discover knowledge and interpret several phenomena. Supervised learning aims in creating a model that takes into account the knowledge adapted by experience, and then uses it for evaluating new observations. One of the most common methods for describing phenomena is through classification. Where a particular object is classified to one of several available classes of objects. The presentation thesis focuses on one of the most promising classification algorithms in the field of machine learning, the «The Support Vector Machine» (SVM). The presentation of the theoretical foundation advances gradually, starting from the most intuitive classification algorithm and reaching up to the optimized approach of SVM, so that it΄s easier for the reader to follow. More specifically, the first chapter is an introduction to data mining algorithms and some related applications. The second chapter presents the fundamental concepts required for an understanding of SVMs. In the third and fourth chapter, there is a detailed report on Support Vector Machines and Regression with SVM, respectively. Then, the fifth chapter presents the evaluation methods of the model while in the sixth chapter a short reference to the feature selection with SVM in made. In the seventh and final chapter three applications and the interpretation of the corresponding results are presented, thus we are able to evaluate the knowledge gained.Δανάη Π. Γιαννούλ

Pattern recognition system based on support vector machines: HIV-1 integrase inhibitors application

Support Vector Machines (SVM) represent one of the most promising Machine Learning (ML) tools that can be applied to develop a predictive Quantitative Structure-Activity Relationship (QSAR) models using molecular descriptors. The performance and predictive power of support vector machines (SVM) for regression problems in quantitative structure-activity relationship were investigated. The SVM results are superior to those obtained by artificial neural network and multiple linear regression. These results indicate that the SVM model with the kernel radial basis function can be used as an alternative tool for regression problems in quantitative structure-activity relationship. Keywords: Support Vector Machines; Artificial Neural Network; Quantitative Structure-Activity Relationship