2 research outputs found
Multiclass Alignment of Confidence and Certainty for Network Calibration
Deep neural networks (DNNs) have made great strides in pushing the
state-of-the-art in several challenging domains. Recent studies reveal that
they are prone to making overconfident predictions. This greatly reduces the
overall trust in model predictions, especially in safety-critical applications.
Early work in improving model calibration employs post-processing techniques
which rely on limited parameters and require a hold-out set. Some recent
train-time calibration methods, which involve all model parameters, can
outperform the postprocessing methods. To this end, we propose a new train-time
calibration method, which features a simple, plug-and-play auxiliary loss known
as multi-class alignment of predictive mean confidence and predictive certainty
(MACC). It is based on the observation that a model miscalibration is directly
related to its predictive certainty, so a higher gap between the mean
confidence and certainty amounts to a poor calibration both for in-distribution
and out-of-distribution predictions. Armed with this insight, our proposed loss
explicitly encourages a confident (or underconfident) model to also provide a
low (or high) spread in the presoftmax distribution. Extensive experiments on
ten challenging datasets, covering in-domain, out-domain, non-visual
recognition and medical image classification scenarios, show that our method
achieves state-of-the-art calibration performance for both in-domain and
out-domain predictions. Our code and models will be publicly released.Comment: Accepted at GCPR 202
Towards Interpretable Sleep Stage Classification Using Cross-Modal Transformers
Accurate sleep stage classification is significant for sleep health
assessment. In recent years, several machine-learning based sleep staging
algorithms have been developed, and in particular, deep-learning based
algorithms have achieved performance on par with human annotation. Despite the
improved performance, a limitation of most deep-learning based algorithms is
their black-box behavior, which has limited their use in clinical settings.
Here, we propose a cross-modal transformer, which is a transformer-based method
for sleep stage classification. The proposed cross-modal transformer consists
of a novel cross-modal transformer encoder architecture along with a
multi-scale one-dimensional convolutional neural network for automatic
representation learning. Our method outperforms the state-of-the-art methods
and eliminates the black-box behavior of deep-learning models by utilizing the
interpretability aspect of the attention modules. Furthermore, our method
provides considerable reductions in the number of parameters and training time
compared to the state-of-the-art methods. Our code is available at
https://github.com/Jathurshan0330/Cross-Modal-Transformer.Comment: 11 pages, 7 figures, 6 table