There is a recently discovered and intriguing phenomenon called Neural
Collapse: at the terminal phase of training a deep neural network for
classification, the within-class penultimate feature means and the associated
classifier vectors of all flat classes collapse to the vertices of a simplex
Equiangular Tight Frame (ETF). Recent work has tried to exploit this phenomenon
by fixing the related classifier weights to a pre-computed ETF to induce neural
collapse and maximize the separation of the learned features when training with
imbalanced data. In this work, we propose to fix the linear classifier of a
deep neural network to a Hierarchy-Aware Frame (HAFrame), instead of an ETF,
and use a cosine similarity-based auxiliary loss to learn hierarchy-aware
penultimate features that collapse to the HAFrame. We demonstrate that our
approach reduces the mistake severity of the model's predictions while
maintaining its top-1 accuracy on several datasets of varying scales with
hierarchies of heights ranging from 3 to 12. We will release our code on GitHub
in the near future