COFFIN: A Computational Framework for Linear SVMs

In a variety of applications, kernel machines such as Support Vector Machines (SVMs) have been used with great success often delivering state-of-the-art results. Using the kernel trick, they work on several domains and even enable heterogeneous data fusion by concatenating feature spaces or multiple kernel learning. Unfortunately, they are not suited for truly large-scale applications since they suffer from the curse of supporting vectors, i.e., the speed of applying SVMs decays linearly with the number of support vectors. In this paper we develop COFFIN — a new training strategy for linear SVMs that effectively allows the use of on demand computed kernel feature spaces and virtual examples in the primal. With linear training and prediction effort this framework leverages SVM applications to truly large-scale problems: As an example, we train SVMs for human splice site recognition involving 50 million examples and sophisticated string kernels. Additionally, we learn an SVM based gender detector on 5 million examples on low-tech hardware and achieve beyond the stateof-the-art accuracies on both tasks. Source code, data sets and scripts are freely available fro

Sequential coordinatewise algorithm for the non-negative least squares problem.

Abstract. This paper contributes to the solution of the non-negative least squares problem (NNLS). The NNLS problem constitutes a substantial part of many computer vision methods and methods in other fields, too. We propose a novel sequential coordinate-wise algorithm which is easy to implement and it is able to cope with large scale problems. We also derive stopping conditions which allow to control the distance of the solution found to the optimal one in terms of the optimized objective function. The proposed algorithm showed promising performance in comparison to the projected Landweber method