21,970 research outputs found
Support vector machines with a reject option
This paper studies regularization with high-dimensional features for
support vector machines with a built-in reject option (meaning that the
decision of classifying an observation can be withheld at a cost lower than
that of misclassification). The procedure can be conveniently implemented as a
linear program and computed using standard software. We prove that the
minimizer of the penalized population risk favors sparse solutions and show
that the behavior of the empirical risk minimizer mimics that of the population
risk minimizer. We also introduce a notion of classification complexity and
prove that our minimizers adapt to the unknown complexity. Using a novel oracle
inequality for the excess risk, we identify situations where fast rates of
convergence occur.Comment: Published in at http://dx.doi.org/10.3150/10-BEJ320 the Bernoulli
(http://isi.cbs.nl/bernoulli/) by the International Statistical
Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm
Support Vector Machines with a Reject Option
We consider the problem of binary classification where the classifier may abstain instead of classifying each observation. The Bayes decision rule for this setup, known as Chow’s rule, is defined by two thresholds on posterior probabilities. From simple desiderata, namely the consistency and the sparsity of the classifier, we derive the double hinge loss function that focuses on estimating conditional probabilities only in the vicinity of the threshold points of the optimal decision rule. We show that, for suitable kernel machines, our approach is universally consistent. We cast the problem of minimizing the double hinge loss as a quadratic program akin to the standard SVM optimization problem and propose an active set method to solve it efficiently. We finally provide preliminary experimental results illustrating the interest of our constructive approach to devising loss functions
Sparse Reject Option Classifier Using Successive Linear Programming
In this paper, we propose an approach for learning sparse reject option
classifiers using double ramp loss . We use DC programming to find the
risk minimizer. The algorithm solves a sequence of linear programs to learn the
reject option classifier. We show that the loss is Fisher consistent.
We also show that the excess risk of loss is upper bounded by the excess
risk of . We derive the generalization error bounds for the proposed
approach. We show the effectiveness of the proposed approach by experimenting
it on several real world datasets. The proposed approach not only performs
comparable to the state of the art but it also successfully learns sparse
classifiers
On Reject and Refine Options in Multicategory Classification
In many real applications of statistical learning, a decision made from
misclassification can be too costly to afford; in this case, a reject option,
which defers the decision until further investigation is conducted, is often
preferred. In recent years, there has been much development for binary
classification with a reject option. Yet, little progress has been made for the
multicategory case. In this article, we propose margin-based multicategory
classification methods with a reject option. In addition, and more importantly,
we introduce a new and unique refine option for the multicategory problem,
where the class of an observation is predicted to be from a set of class
labels, whose cardinality is not necessarily one. The main advantage of both
options lies in their capacity of identifying error-prone observations.
Moreover, the refine option can provide more constructive information for
classification by effectively ruling out implausible classes. Efficient
implementations have been developed for the proposed methods. On the
theoretical side, we offer a novel statistical learning theory and show a fast
convergence rate of the excess -risk of our methods with emphasis on
diverging dimensionality and number of classes. The results can be further
improved under a low noise assumption. A set of comprehensive simulation and
real data studies has shown the usefulness of the new learning tools compared
to regular multicategory classifiers. Detailed proofs of theorems and extended
numerical results are included in the supplemental materials available online.Comment: A revised version of this paper was accepted for publication in the
Journal of the American Statistical Association Theory and Methods Section.
52 pages, 6 figure
Classifiers With a Reject Option for Early Time-Series Classification
Early classification of time-series data in a dynamic environment is a
challenging problem of great importance in signal processing. This paper
proposes a classifier architecture with a reject option capable of online
decision making without the need to wait for the entire time series signal to
be present. The main idea is to classify an odor/gas signal with an acceptable
accuracy as early as possible. Instead of using posterior probability of a
classifier, the proposed method uses the "agreement" of an ensemble to decide
whether to accept or reject the candidate label. The introduced algorithm is
applied to the bio-chemistry problem of odor classification to build a novel
Electronic-Nose called Forefront-Nose. Experimental results on wind tunnel
test-bed facility confirms the robustness of the forefront-nose compared to the
standard classifiers from both earliness and recognition perspectives
Is Deep Learning Safe for Robot Vision? Adversarial Examples against the iCub Humanoid
Deep neural networks have been widely adopted in recent years, exhibiting
impressive performances in several application domains. It has however been
shown that they can be fooled by adversarial examples, i.e., images altered by
a barely-perceivable adversarial noise, carefully crafted to mislead
classification. In this work, we aim to evaluate the extent to which
robot-vision systems embodying deep-learning algorithms are vulnerable to
adversarial examples, and propose a computationally efficient countermeasure to
mitigate this threat, based on rejecting classification of anomalous inputs. We
then provide a clearer understanding of the safety properties of deep networks
through an intuitive empirical analysis, showing that the mapping learned by
such networks essentially violates the smoothness assumption of learning
algorithms. We finally discuss the main limitations of this work, including the
creation of real-world adversarial examples, and sketch promising research
directions.Comment: Accepted for publication at the ICCV 2017 Workshop on Vision in
Practice on Autonomous Robots (ViPAR
- …