14 research outputs found
Training Support Vector Machines Using Frank-Wolfe Optimization Methods
Training a Support Vector Machine (SVM) requires the solution of a quadratic
programming problem (QP) whose computational complexity becomes prohibitively
expensive for large scale datasets. Traditional optimization methods cannot be
directly applied in these cases, mainly due to memory restrictions.
By adopting a slightly different objective function and under mild conditions
on the kernel used within the model, efficient algorithms to train SVMs have
been devised under the name of Core Vector Machines (CVMs). This framework
exploits the equivalence of the resulting learning problem with the task of
building a Minimal Enclosing Ball (MEB) problem in a feature space, where data
is implicitly embedded by a kernel function.
In this paper, we improve on the CVM approach by proposing two novel methods
to build SVMs based on the Frank-Wolfe algorithm, recently revisited as a fast
method to approximate the solution of a MEB problem. In contrast to CVMs, our
algorithms do not require to compute the solutions of a sequence of
increasingly complex QPs and are defined by using only analytic optimization
steps. Experiments on a large collection of datasets show that our methods
scale better than CVMs in most cases, sometimes at the price of a slightly
lower accuracy. As CVMs, the proposed methods can be easily extended to machine
learning problems other than binary classification. However, effective
classifiers are also obtained using kernels which do not satisfy the condition
required by CVMs and can thus be used for a wider set of problems
Generalized Pattern Search methods for a class of nonsmooth optimization problems with structure
A two-stage method for nonlinear inverse problems
AbstractIn this work we are interested in the solution of nonlinear inverse problems of the form F(x)=y. We consider a two-stage method which is third order convergent for well-posed problems. Combining the method with Levenberg–Marquardt regularization of the linearized problems at each stage and using the discrepancy principle as a stopping criterion, we obtain a regularization method for ill-posed problems. Numerical experiments on some parameter identification and inverse acoustic scattering problems are presented to illustrate the performance of the method
A New Algorithm for Training SVMs using Approximate Minimal Enclosing Balls
It has been shown that many kernel methods can be equivalently formulated as minimal enclosing ball (MEB) problems in a certain feature space. Exploiting this reduction, efficient algorithms to scale up Support Vector Machines (SVMs) and other kernel methods have been introduced under the name of Core Vector Machines (CVMs). In this paper, we study a new algorithm to train SVMs based on an instance of the Frank-Wolfe optimization method recently proposed to approximate the solution of the MEB problem. We show that, specialized to SVM training, this algorithm can scale better than CVMs at the price of a slightly lower accuracy
Path following by SVD
Abstract. In this paper, we propose a path-following method for computing a curve of equilibria of a dynamical system, based upon the smooth Singular Value Decomposition (SVD) of the Jacobian matrix. Our method is capable of detecting fold points, and continuing past folds. It is also able to detect branch points and to switch branches at such points. Algorithmic details and examples are given.