626 research outputs found
Linear components of quadratic classifiers
This is pre-print of an article published in Advances in Data Analysis and Classification. The final authenticated version is available online at: https://doi.org/10.1007/s11634-018-0321-6We obtain a decomposition of any quadratic classifier in terms of products
of hyperplanes. These hyperplanes can be viewed as relevant linear components
of the quadratic rule (with respect to the underlying classification problem). As an
application, we introduce the associated multidirectional classifier; a piecewise linear
classification rule induced by the approximating products. Such a classifier is useful to
determine linear combinations of the predictor variables with ability to discriminate.
We also show that this classifier can be used as a tool to reduce the dimension of the
data and helps identify the most important variables to classify new elements. Finally,
we illustrate with a real data set the use of these linear components to construct oblique
classification treesThis research was supported by the Spanish MCyT grant MTM2016-78751-
Minimizing the error of linear separators on linearly inseparable data
Given linearly inseparable sets R of red points and B of blue points, we consider several
measures of how far they are from being separable. Intuitively, given a potential separator
(‘‘classifier’’), we measure its quality (‘‘error’’) according to how much work it would take
to move the misclassified points across the classifier to yield separated sets. We consider
several measures of work and provide algorithms to find linear classifiers that minimize
the error under these different measures.Ministerio de Educación y Ciencia MTM2008-05866-C03-0
Sign rank versus VC dimension
This work studies the maximum possible sign rank of sign
matrices with a given VC dimension . For , this maximum is {three}. For
, this maximum is . For , similar but
slightly less accurate statements hold. {The lower bounds improve over previous
ones by Ben-David et al., and the upper bounds are novel.}
The lower bounds are obtained by probabilistic constructions, using a theorem
of Warren in real algebraic topology. The upper bounds are obtained using a
result of Welzl about spanning trees with low stabbing number, and using the
moment curve.
The upper bound technique is also used to: (i) provide estimates on the
number of classes of a given VC dimension, and the number of maximum classes of
a given VC dimension -- answering a question of Frankl from '89, and (ii)
design an efficient algorithm that provides an multiplicative
approximation for the sign rank.
We also observe a general connection between sign rank and spectral gaps
which is based on Forster's argument. Consider the adjacency
matrix of a regular graph with a second eigenvalue of absolute value
and . We show that the sign rank of the signed
version of this matrix is at least . We use this connection to
prove the existence of a maximum class with VC
dimension and sign rank . This answers a question
of Ben-David et al.~regarding the sign rank of large VC classes. We also
describe limitations of this approach, in the spirit of the Alon-Boppana
theorem.
We further describe connections to communication complexity, geometry,
learning theory, and combinatorics.Comment: 33 pages. This is a revised version of the paper "Sign rank versus VC
dimension". Additional results in this version: (i) Estimates on the number
of maximum VC classes (answering a question of Frankl from '89). (ii)
Estimates on the sign rank of large VC classes (answering a question of
Ben-David et al. from '03). (iii) A discussion on the computational
complexity of computing the sign-ran
Fast DD-classification of functional data
A fast nonparametric procedure for classifying functional data is introduced.
It consists of a two-step transformation of the original data plus a classifier
operating on a low-dimensional hypercube. The functional data are first mapped
into a finite-dimensional location-slope space and then transformed by a
multivariate depth function into the -plot, which is a subset of the unit
hypercube. This transformation yields a new notion of depth for functional
data. Three alternative depth functions are employed for this, as well as two
rules for the final classification on . The resulting classifier has
to be cross-validated over a small range of parameters only, which is
restricted by a Vapnik-Cervonenkis bound. The entire methodology does not
involve smoothing techniques, is completely nonparametric and allows to achieve
Bayes optimality under standard distributional settings. It is robust,
efficiently computable, and has been implemented in an R environment.
Applicability of the new approach is demonstrated by simulations as well as a
benchmark study
Accelerating Kernel Classifiers Through Borders Mapping
Support vector machines (SVM) and other kernel techniques represent a family
of powerful statistical classification methods with high accuracy and broad
applicability. Because they use all or a significant portion of the training
data, however, they can be slow, especially for large problems. Piecewise
linear classifiers are similarly versatile, yet have the additional advantages
of simplicity, ease of interpretation and, if the number of component linear
classifiers is not too large, speed. Here we show how a simple, piecewise
linear classifier can be trained from a kernel-based classifier in order to
improve the classification speed. The method works by finding the root of the
difference in conditional probabilities between pairs of opposite classes to
build up a representation of the decision boundary. When tested on 17 different
datasets, it succeeded in improving the classification speed of a SVM for 12 of
them by up to two orders-of-magnitude. Of these, two were less accurate than a
simple, linear classifier. The method is best suited to problems with continuum
features data and smooth probability functions. Because the component linear
classifiers are built up individually from an existing classifier, rather than
through a simultaneous optimization procedure, the classifier is also fast to
train.Comment: This is the final, published version which is quite different from
the first draft. A small but important error has been caught and correcte
- …