713 research outputs found
Modified Frank-Wolfe Algorithm for Enhanced Sparsity in Support Vector Machine Classifiers
This work proposes a new algorithm for training a re-weighted L2 Support
Vector Machine (SVM), inspired on the re-weighted Lasso algorithm of Cand\`es
et al. and on the equivalence between Lasso and SVM shown recently by Jaggi. In
particular, the margin required for each training vector is set independently,
defining a new weighted SVM model. These weights are selected to be binary, and
they are automatically adapted during the training of the model, resulting in a
variation of the Frank-Wolfe optimization algorithm with essentially the same
computational complexity as the original algorithm. As shown experimentally,
this algorithm is computationally cheaper to apply since it requires less
iterations to converge, and it produces models with a sparser representation in
terms of support vectors and which are more stable with respect to the
selection of the regularization hyper-parameter
A PARTAN-Accelerated Frank-Wolfe Algorithm for Large-Scale SVM Classification
Frank-Wolfe algorithms have recently regained the attention of the Machine
Learning community. Their solid theoretical properties and sparsity guarantees
make them a suitable choice for a wide range of problems in this field. In
addition, several variants of the basic procedure exist that improve its
theoretical properties and practical performance. In this paper, we investigate
the application of some of these techniques to Machine Learning, focusing in
particular on a Parallel Tangent (PARTAN) variant of the FW algorithm that has
not been previously suggested or studied for this type of problems. We provide
experiments both in a standard setting and using a stochastic speed-up
technique, showing that the considered algorithms obtain promising results on
several medium and large-scale benchmark datasets for SVM classification
- …