60,174 research outputs found
On the design of robust classifiers for computer vision
The design of robust classifiers, which can contend with the noisy and outlier ridden datasets typical of computer vision, is studied. It is argued that such robustness requires loss functions that penalize both large positive and negative margins. The probability elicitation view of classifier design is adopted, and a set of necessary conditions for the design of such losses is identified. These conditions are used to derive a novel robust Bayes-consistent loss, denoted Tangent loss, and an associated boosting algorithm, denoted TangentBoost. Experiments with data from the computer vision problems of scene classification, object tracking, and multiple instance learning show that TangentBoost consistently outperforms previous boosting algorithms. 1
Stratified Adversarial Robustness with Rejection
Recently, there is an emerging interest in adversarially training a
classifier with a rejection option (also known as a selective classifier) for
boosting adversarial robustness. While rejection can incur a cost in many
applications, existing studies typically associate zero cost with rejecting
perturbed inputs, which can result in the rejection of numerous
slightly-perturbed inputs that could be correctly classified. In this work, we
study adversarially-robust classification with rejection in the stratified
rejection setting, where the rejection cost is modeled by rejection loss
functions monotonically non-increasing in the perturbation magnitude. We
theoretically analyze the stratified rejection setting and propose a novel
defense method -- Adversarial Training with Consistent Prediction-based
Rejection (CPR) -- for building a robust selective classifier. Experiments on
image datasets demonstrate that the proposed method significantly outperforms
existing methods under strong adaptive attacks. For instance, on CIFAR-10, CPR
reduces the total robust loss (for different rejection losses) by at least 7.3%
under both seen and unseen attacks.Comment: Paper published at International Conference on Machine Learning
(ICML'23
Generating Compact Tree Ensembles via Annealing
Tree ensembles are flexible predictive models that can capture relevant
variables and to some extent their interactions in a compact and interpretable
manner. Most algorithms for obtaining tree ensembles are based on versions of
boosting or Random Forest. Previous work showed that boosting algorithms
exhibit a cyclic behavior of selecting the same tree again and again due to the
way the loss is optimized. At the same time, Random Forest is not based on loss
optimization and obtains a more complex and less interpretable model. In this
paper we present a novel method for obtaining compact tree ensembles by growing
a large pool of trees in parallel with many independent boosting threads and
then selecting a small subset and updating their leaf weights by loss
optimization. We allow for the trees in the initial pool to have different
depths which further helps with generalization. Experiments on real datasets
show that the obtained model has usually a smaller loss than boosting, which is
also reflected in a lower misclassification error on the test set.Comment: Comparison with Random Forest included in the results sectio
- …