32,203 research outputs found
Distributed Adaptive Nearest Neighbor Classifier: Algorithm and Theory
When data is of an extraordinarily large size or physically stored in
different locations, the distributed nearest neighbor (NN) classifier is an
attractive tool for classification. We propose a novel distributed adaptive NN
classifier for which the number of nearest neighbors is a tuning parameter
stochastically chosen by a data-driven criterion. An early stopping rule is
proposed when searching for the optimal tuning parameter, which not only speeds
up the computation but also improves the finite sample performance of the
proposed Algorithm. Convergence rate of excess risk of the distributed adaptive
NN classifier is investigated under various sub-sample size compositions. In
particular, we show that when the sub-sample sizes are sufficiently large, the
proposed classifier achieves the nearly optimal convergence rate. Effectiveness
of the proposed approach is demonstrated through simulation studies as well as
an empirical application to a real-world dataset
An adaptive multiclass nearest neighbor classifier
We consider a problem of multiclass classification, where the training sample
is generated from the model , , and are
unknown -Holder continuous functions.Given a test point , our goal
is to predict its label. A widely used -nearest-neighbors classifier
constructs estimates of and uses a plug-in rule
for the prediction. However, it requires a proper choice of the smoothing
parameter , which may become tricky in some situations. In our
solution, we fix several integers , compute corresponding
-nearest-neighbor estimates for each and each and apply an
aggregation procedure. We study an algorithm, which constructs a convex
combination of these estimates such that the aggregated estimate behaves
approximately as well as an oracle choice. We also provide a non-asymptotic
analysis of the procedure, prove its adaptation to the unknown smoothness
parameter and to the margin and establish rates of convergence under
mild assumptions.Comment: Accepted in ESAIM: Probability & Statistics. The original publication
is available at www.esaim-ps.or
An adaptive nearest neighbor rule for classification
We introduce a variant of the -nearest neighbor classifier in which is
chosen adaptively for each query, rather than supplied as a parameter. The
choice of depends on properties of each neighborhood, and therefore may
significantly vary between different points. (For example, the algorithm will
use larger for predicting the labels of points in noisy regions.)
We provide theory and experiments that demonstrate that the algorithm
performs comparably to, and sometimes better than, -NN with an optimal
choice of . In particular, we derive bounds on the convergence rates of our
classifier that depend on a local quantity we call the `advantage' which is
significantly weaker than the Lipschitz conditions used in previous convergence
rate proofs. These generalization bounds hinge on a variant of the seminal
Uniform Convergence Theorem due to Vapnik and Chervonenkis; this variant
concerns conditional probabilities and may be of independent interest
Classification with the nearest neighbor rule in general finite dimensional spaces: necessary and sufficient conditions
Given an -sample of random vectors whose
joint law is unknown, the long-standing problem of supervised classification
aims to \textit{optimally} predict the label of a given a new observation
. In this context, the nearest neighbor rule is a popular flexible and
intuitive method in non-parametric situations.
Even if this algorithm is commonly used in the machine learning and
statistics communities, less is known about its prediction ability in general
finite dimensional spaces, especially when the support of the density of the
observations is . This paper is devoted to the study of the
statistical properties of the nearest neighbor rule in various situations. In
particular, attention is paid to the marginal law of , as well as the
smoothness and margin properties of the \textit{regression function} . We identify two necessary and sufficient conditions to
obtain uniform consistency rates of classification and to derive sharp
estimates in the case of the nearest neighbor rule. Some numerical experiments
are proposed at the end of the paper to help illustrate the discussion.Comment: 53 Pages, 3 figure
- …