33,676 research outputs found
C2AE: Class Conditioned Auto-Encoder for Open-set Recognition
Models trained for classification often assume that all testing classes are
known while training. As a result, when presented with an unknown class during
testing, such closed-set assumption forces the model to classify it as one of
the known classes. However, in a real world scenario, classification models are
likely to encounter such examples. Hence, identifying those examples as unknown
becomes critical to model performance. A potential solution to overcome this
problem lies in a class of learning problems known as open-set recognition. It
refers to the problem of identifying the unknown classes during testing, while
maintaining performance on the known classes. In this paper, we propose an
open-set recognition algorithm using class conditioned auto-encoders with novel
training and testing methodology. In contrast to previous methods, training
procedure is divided in two sub-tasks, 1. closed-set classification and, 2.
open-set identification (i.e. identifying a class as known or unknown). Encoder
learns the first task following the closed-set classification training
pipeline, whereas decoder learns the second task by reconstructing conditioned
on class identity. Furthermore, we model reconstruction errors using the
Extreme Value Theory of statistical modeling to find the threshold for
identifying known/unknown class samples. Experiments performed on multiple
image classification datasets show proposed method performs significantly
better than state of the art.Comment: CVPR2019 (Oral
Binary Classifier Calibration using an Ensemble of Near Isotonic Regression Models
Learning accurate probabilistic models from data is crucial in many practical
tasks in data mining. In this paper we present a new non-parametric calibration
method called \textit{ensemble of near isotonic regression} (ENIR). The method
can be considered as an extension of BBQ, a recently proposed calibration
method, as well as the commonly used calibration method based on isotonic
regression. ENIR is designed to address the key limitation of isotonic
regression which is the monotonicity assumption of the predictions. Similar to
BBQ, the method post-processes the output of a binary classifier to obtain
calibrated probabilities. Thus it can be combined with many existing
classification models. We demonstrate the performance of ENIR on synthetic and
real datasets for the commonly used binary classification models. Experimental
results show that the method outperforms several common binary classifier
calibration methods. In particular on the real data, ENIR commonly performs
statistically significantly better than the other methods, and never worse. It
is able to improve the calibration power of classifiers, while retaining their
discrimination power. The method is also computationally tractable for large
scale datasets, as it is time, where is the number of
samples
- …