Machine learning models deployed in the open world may encounter observations
that they were not trained to recognize, and they risk misclassifying such
observations with high confidence. Therefore, it is essential that these models
are able to ascertain what is in-distribution (ID) and out-of-distribution
(OOD), to avoid this misclassification. In recent years, huge strides have been
made in creating models that are robust to this distinction. As a result, the
current state-of-the-art has reached near perfect performance on relatively
coarse-grained OOD detection tasks, such as distinguishing horses from trucks,
while struggling with finer-grained classification, like differentiating models
of commercial aircraft. In this paper, we describe a new theoretical framework
for understanding fine- and coarse-grained OOD detection, we re-conceptualize
fine grained classification into a three part problem, and we propose a new
baseline task for OOD models on two fine-grained hierarchical data sets, two
new evaluation methods to differentiate fine- and coarse-grained OOD
performance, along with a new loss function for models in this task