7,738 research outputs found
An adaptive nearest neighbor rule for classification
We introduce a variant of the -nearest neighbor classifier in which is
chosen adaptively for each query, rather than supplied as a parameter. The
choice of depends on properties of each neighborhood, and therefore may
significantly vary between different points. (For example, the algorithm will
use larger for predicting the labels of points in noisy regions.)
We provide theory and experiments that demonstrate that the algorithm
performs comparably to, and sometimes better than, -NN with an optimal
choice of . In particular, we derive bounds on the convergence rates of our
classifier that depend on a local quantity we call the `advantage' which is
significantly weaker than the Lipschitz conditions used in previous convergence
rate proofs. These generalization bounds hinge on a variant of the seminal
Uniform Convergence Theorem due to Vapnik and Chervonenkis; this variant
concerns conditional probabilities and may be of independent interest
Active Nearest-Neighbor Learning in Metric Spaces
We propose a pool-based non-parametric active learning algorithm for general
metric spaces, called MArgin Regularized Metric Active Nearest Neighbor
(MARMANN), which outputs a nearest-neighbor classifier. We give prediction
error guarantees that depend on the noisy-margin properties of the input
sample, and are competitive with those obtained by previously proposed passive
learners. We prove that the label complexity of MARMANN is significantly lower
than that of any passive learner with similar error guarantees. MARMANN is
based on a generalized sample compression scheme, and a new label-efficient
active model-selection procedure
Nonparametrically consistent depth-based classifiers
We introduce a class of depth-based classification procedures that are of a
nearest-neighbor nature. Depth, after symmetrization, indeed provides the
center-outward ordering that is necessary and sufficient to define nearest
neighbors. Like all their depth-based competitors, the resulting classifiers
are affine-invariant, hence in particular are insensitive to unit changes.
Unlike the former, however, the latter achieve Bayes consistency under
virtually any absolutely continuous distributions - a concept we call
nonparametric consistency, to stress the difference with the stronger universal
consistency of the standard NN classifiers. We investigate the finite-sample
performances of the proposed classifiers through simulations and show that they
outperform affine-invariant nearest-neighbor classifiers obtained through an
obvious standardization construction. We illustrate the practical value of our
classifiers on two real data examples. Finally, we shortly discuss the possible
uses of our depth-based neighbors in other inference problems.Comment: Published at http://dx.doi.org/10.3150/13-BEJ561 in the Bernoulli
(http://isi.cbs.nl/bernoulli/) by the International Statistical
Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm
- …