Statistical Mechanics of Support Vector Networks

Using methods of Statistical Physics, we investigate the generalization performance of support vector machines (SVMs), which have been recently introduced as a general alternative to neural networks. For nonlinear classification rules, the generalization error saturates on a plateau, when the number of examples is too small to properly estimate the coefficients of the nonlinear part. When trained on simple rules, we find that SVMs overfit only weakly. The performance of SVMs is strongly enhanced, when the distribution of the inputs has a gap in feature space.Comment: REVTeX, 4 pages, 2 figures, accepted by Phys. Rev. Lett (typos corrected

Support vector networks for prediction of floor pressures in shallow cavity flows

During the last decade, Support Vector Machines (SVM) have proved to be very successful tools for classification and regression problems. The representational performance of this type of networks is studied on a cavity flow facility developed to investigate the characteristics of aerodynamic flows at various Mach numbers. Several test conditions have been experimented to collect a set of data, which is in the form of pressure readings from particular points in the test section. The goal is to develop a SVM based model that emulates the one step ahead behavior of the flow measurement at the cavity floor. The SVM based model is built for a very limited amount of training data and the model is tested for an extended set of test conditions. A relative error is defined to measure the reconstruction performance, and the peak value of the FFT magnitude of the error is measured. The results indicate that the SVM based model is capable of matching the experimental data satisfactorily over the conditions that are close to the training data collection conditions, and the performance degrades as the Mach number gets away from the conditions considered during training. ©2006 IEEE

Decoding Sequence Classification Models for Acquiring New Biological Insights

Classifying biological sequences is one of the most important tasks in computational biology. In the last decade, support vector machines (SVMs) in combination with sequence kernels have emerged as a de-facto standard. These methods are theoretically well-founded, reliable, and provide high-accuracy solutions at low computational cost. However, obtaining a highly accurate classifier is rarely the end of the story in many practical situations. Instead, one often aims to acquire biological knowledge about the principles underlying a given classification task. SVMs with traditional sequence kernels do not offer a straightforward way of accessing this knowledge.

In this contribution, we propose a new approach to analyzing biological sequences on the basis of support vector machines with sequence kernels. We first extract explicit pattern weights from a given SVM. When classifying a sequence, we then compute a prediction profile by distributing the weight of each pattern to the sequence positions that match the pattern. The final profile not only allows assessing the importance of a position, but also determining for which class it is indicative. Since it is unfeasible to analyze profiles of all sequences in a given data set, we advocate using affinity propagation (AP) clustering to narrow down the analysis to a small set of typical sequences.

The proposed approach is applicable to a wide range of biological sequences and a wide selection of sequence kernels. To illustrate our framework, we present the prediction of oligomerization tendencies of coiled coil proteins as a case study.
&#xa