Compact Random Feature Maps

Kernel approximation using randomized feature maps has recently gained a lot of interest. In this work, we identify that previous approaches for polynomial kernel approximation create maps that are rank deficient, and therefore do not utilize the capacity of the projected feature space effectively. To address this challenge, we propose compact random feature maps (CRAFTMaps) to approximate polynomial kernels more concisely and accurately. We prove the error bounds of CRAFTMaps demonstrating their superior kernel reconstruction performance compared to the previous approximation schemes. We show how structured random matrices can be used to efficiently generate CRAFTMaps, and present a single-pass algorithm using CRAFTMaps to learn non-linear multi-class classifiers. We present experiments on multiple standard data-sets with performance competitive with state-of-the-art results.Comment: 9 page

A Novel Model of Working Set Selection for SMO Decomposition Methods

In the process of training Support Vector Machines (SVMs) by decomposition methods, working set selection is an important technique, and some exciting schemes were employed into this field. To improve working set selection, we propose a new model for working set selection in sequential minimal optimization (SMO) decomposition methods. In this model, it selects B as working set without reselection. Some properties are given by simple proof, and experiments demonstrate that the proposed method is in general faster than existing methods.Comment: 8 pages, 12 figures, it was submitted to IEEE International conference of Tools on Artificial Intelligenc

Large quadratic programs in training gaussian support vector machines

We consider the numerical solution of the large convex quadratic program arising in training the learning machines named support vector machines. Since the matrix of the quadratic form is dense and generally large, solution approaches based on explicitstorage of this matrix are not practicable. Well known strategies for this quadratic program are based on decomposition techniques that split the problem into a sequence of smaller quadratic programming subproblems. For the solution of these subproblems we present an iterative projection-type method suited for the structure of the constraints and very eective in case of Gaussian support vector machines. We develop an appropriate decomposition technique designed to exploit the high performance of the proposed inner solver on medium or large subproblems. Numerical experiments on large-scale benchmark problems allow to compare this approach with another widelyused decomposition technique. Finally, a parallel extension of the proposed strategy is described

Support Vector Machines in a real time tracking architecture

The standard approach to tracking an object of interest in a video stream is to use an object detector, a classifier and a tracker in sequential order. This work investigates the use of Support Vector Machines (SVM) as classifiers for real-time tracking systems, combining them with Kalman Filter predictors. Support Vector Machines have been proved successful in a variety of classification tasks such as recognizing faces, cars, handwriting and others. However their use has been hampered by the complexity and computational time involved in the training and classification stages. In recent years new methods and techniques for training and classification of Support Vector Machines have been discovered making possible their utilization in real-time applications. These methods have been explored and improved resulting in a framework for fast prototyping and development of real-time tracking systems. New optimal and sub-optimal methods for parallel SVM training based on biased and unbiased versions of the Sequential Minimal Optimization algorithm are presented. They provide a trade-off between time performance and accuracy. Time performance in the classification stage is significantly improved by reducing the number of support vectors with almost no loss in accuracy. New methods to allow the reduction with different kernels are presented. The effectiveness of the approach developed is demonstrated in a face tracking problem where the objective is to track the lips and eyes of a subject in a video stream in real-time

Sparse kernel SVMs via cutting-plane training

Appeal from a decision of the Board of Review, Industrial Commission of Utah on February 7, 1989