3 research outputs found
A Knowledge Distillation Framework For Enhancing Ear-EEG Based Sleep Staging With Scalp-EEG Data
Sleep plays a crucial role in the well-being of human lives. Traditional
sleep studies using Polysomnography are associated with discomfort and often
lower sleep quality caused by the acquisition setup. Previous works have
focused on developing less obtrusive methods to conduct high-quality sleep
studies, and ear-EEG is among popular alternatives. However, the performance of
sleep staging based on ear-EEG is still inferior to scalp-EEG based sleep
staging. In order to address the performance gap between scalp-EEG and ear-EEG
based sleep staging, we propose a cross-modal knowledge distillation strategy,
which is a domain adaptation approach. Our experiments and analysis validate
the effectiveness of the proposed approach with existing architectures, where
it enhances the accuracy of the ear-EEG based sleep staging by 3.46% and
Cohen's kappa coefficient by a margin of 0.038.Comment: Code available at :
https://github.com/Mithunjha/EarEEG_KnowledgeDistillatio
Towards Interpretable Sleep Stage Classification Using Cross-Modal Transformers
Accurate sleep stage classification is significant for sleep health
assessment. In recent years, several machine-learning based sleep staging
algorithms have been developed, and in particular, deep-learning based
algorithms have achieved performance on par with human annotation. Despite the
improved performance, a limitation of most deep-learning based algorithms is
their black-box behavior, which has limited their use in clinical settings.
Here, we propose a cross-modal transformer, which is a transformer-based method
for sleep stage classification. The proposed cross-modal transformer consists
of a novel cross-modal transformer encoder architecture along with a
multi-scale one-dimensional convolutional neural network for automatic
representation learning. Our method outperforms the state-of-the-art methods
and eliminates the black-box behavior of deep-learning models by utilizing the
interpretability aspect of the attention modules. Furthermore, our method
provides considerable reductions in the number of parameters and training time
compared to the state-of-the-art methods. Our code is available at
https://github.com/Jathurshan0330/Cross-Modal-Transformer.Comment: 11 pages, 7 figures, 6 table
Contrastive Deep Encoding Enables Uncertainty-aware Machine-learning-assisted Histopathology
Deep neural network models can learn clinically relevant features from
millions of histopathology images. However generating high-quality annotations
to train such models for each hospital, each cancer type, and each diagnostic
task is prohibitively laborious. On the other hand, terabytes of training data
-- while lacking reliable annotations -- are readily available in the public
domain in some cases. In this work, we explore how these large datasets can be
consciously utilized to pre-train deep networks to encode informative
representations. We then fine-tune our pre-trained models on a fraction of
annotated training data to perform specific downstream tasks. We show that our
approach can reach the state-of-the-art (SOTA) for patch-level classification
with only 1-10% randomly selected annotations compared to other SOTA
approaches. Moreover, we propose an uncertainty-aware loss function, to
quantify the model confidence during inference. Quantified uncertainty helps
experts select the best instances to label for further training. Our
uncertainty-aware labeling reaches the SOTA with significantly fewer
annotations compared to random labeling. Last, we demonstrate how our
pre-trained encoders can surpass current SOTA for whole-slide image
classification with weak supervision. Our work lays the foundation for data and
task-agnostic pre-trained deep networks with quantified uncertainty.Comment: 18 pages, 8 figure