2 research outputs found
Approximate Stochastic Subgradient Estimation Training for Support Vector Machines
Subgradient algorithms for training support vector machines have been quite
successful for solving large-scale and online learning problems. However, they
have been restricted to linear kernels and strongly convex formulations. This
paper describes efficient subgradient approaches without such limitations. Our
approaches make use of randomized low-dimensional approximations to nonlinear
kernels, and minimization of a reduced primal formulation using an algorithm
based on robust stochastic approximation, which do not require strong
convexity. Experiments illustrate that our approaches produce solutions of
comparable prediction accuracy with the solutions acquired from existing SVM
solvers, but often in much shorter time. We also suggest efficient prediction
schemes that depend only on the dimension of kernel approximation, not on the
number of support vectors.Comment: An extended version of the ICPRAM 2012 pape
Distributed Inference for Linear Support Vector Machine
The growing size of modern data brings many new challenges to existing
statistical inference methodologies and theories, and calls for the development
of distributed inferential approaches. This paper studies distributed inference
for linear support vector machine (SVM) for the binary classification task.
Despite a vast literature on SVM, much less is known about the inferential
properties of SVM, especially in a distributed setting. In this paper, we
propose a multi-round distributed linear-type (MDL) estimator for conducting
inference for linear SVM. The proposed estimator is computationally efficient.
In particular, it only requires an initial SVM estimator and then successively
refines the estimator by solving simple weighted least squares problem.
Theoretically, we establish the Bahadur representation of the estimator. Based
on the representation, the asymptotic normality is further derived, which shows
that the MDL estimator achieves the optimal statistical efficiency, i.e., the
same efficiency as the classical linear SVM applying to the entire data set in
a single machine setup. Moreover, our asymptotic result avoids the condition on
the number of machines or data batches, which is commonly assumed in
distributed estimation literature, and allows the case of diverging dimension.
We provide simulation studies to demonstrate the performance of the proposed
MDL estimator.Comment: 50 pages, 11 figure