39 research outputs found
Qualitative Robustness of Support Vector Machines
Support vector machines have attracted much attention in theoretical and in
applied statistics. Main topics of recent interest are consistency, learning
rates and robustness. In this article, it is shown that support vector machines
are qualitatively robust. Since support vector machines can be represented by a
functional on the set of all probability measures, qualitative robustness is
proven by showing that this functional is continuous with respect to the
topology generated by weak convergence of probability measures. Combined with
the existence and uniqueness of support vector machines, our results show that
support vector machines are the solutions of a well-posed mathematical problem
in Hadamard's sense
Asymptotic Normality of Support Vector Machine Variants and Other Regularized Kernel Methods
In nonparametric classification and regression problems, regularized kernel
methods, in particular support vector machines, attract much attention in
theoretical and in applied statistics. In an abstract sense, regularized kernel
methods (simply called SVMs here) can be seen as regularized M-estimators for a
parameter in a (typically infinite dimensional) reproducing kernel Hilbert
space. For smooth loss functions, it is shown that the difference between the
estimator, i.e.\ the empirical SVM, and the theoretical SVM is asymptotically
normal with rate . That is, the standardized difference converges
weakly to a Gaussian process in the reproducing kernel Hilbert space. As common
in real applications, the choice of the regularization parameter may depend on
the data. The proof is done by an application of the functional delta-method
and by showing that the SVM-functional is suitably Hadamard-differentiable
Predictability, Stability, and Computability of Locally Learnt SVMs
We will have a look at the principles predictability, stability, and computability in the field of support vector machines. Support vector machines (SVMs), well-known in machine learning, play a successful role in classification and regression in many areas of science. In the past three decades, much research has been conducted on the statistical and computational properties of support vector machines and related kernel methods. On the one hand, consistency (predictability) and robustness (stability) of the method are of interest. On the other hand, from an applied point of view, there is interest in a method that can deal with many observations and many features (computability). Since SVMs require a lot of computing power and storage capacity, various possibilities for processing large data sets have been proposed. One of them is called regionalization. It divides the space of declaring variables into possibly overlapping domains in a data driven way and defines the function to predict the output by the formation of locally learnt support vector machines. Another advantage of regionalization should be mentioned.
If the generating distribution in different regions of the input space has different characteristics, learning only one “global” SVM may lead to an imprecise estimate. Locally trained predictors can overcome this problem. It is possible to show that a locally learnt predictor is consistent and robust under assumptions that can be checked by the user of this method