3 research outputs found
Enhancing Low-resolution Face Recognition with Feature Similarity Knowledge Distillation
In this study, we introduce a feature knowledge distillation framework to
improve low-resolution (LR) face recognition performance using knowledge
obtained from high-resolution (HR) images. The proposed framework transfers
informative features from an HR-trained network to an LR-trained network by
reducing the distance between them. A cosine similarity measure was employed as
a distance metric to effectively align the HR and LR features. This approach
differs from conventional knowledge distillation frameworks, which use the L_p
distance metrics and offer the advantage of converging well when reducing the
distance between features of different resolutions. Our framework achieved a 3%
improvement over the previous state-of-the-art method on the AgeDB-30 benchmark
without bells and whistles, while maintaining a strong performance on HR
images. The effectiveness of cosine similarity as a distance metric was
validated through statistical analysis, making our approach a promising
solution for real-world applications in which LR images are frequently
encountered. The code and pretrained models are publicly available on
https://github.com/gist-ailab/feature-similarity-KD
SleePyCo: Automatic Sleep Scoring with Feature Pyramid and Contrastive Learning
Automatic sleep scoring is essential for the diagnosis and treatment of sleep
disorders and enables longitudinal sleep tracking in home environments.
Conventionally, learning-based automatic sleep scoring on single-channel
electroencephalogram (EEG) is actively studied because obtaining multi-channel
signals during sleep is difficult. However, learning representation from raw
EEG signals is challenging owing to the following issues: 1) sleep-related EEG
patterns occur on different temporal and frequency scales and 2) sleep stages
share similar EEG patterns. To address these issues, we propose a deep learning
framework named SleePyCo that incorporates 1) a feature pyramid and 2)
supervised contrastive learning for automatic sleep scoring. For the feature
pyramid, we propose a backbone network named SleePyCo-backbone to consider
multiple feature sequences on different temporal and frequency scales.
Supervised contrastive learning allows the network to extract class
discriminative features by minimizing the distance between intra-class features
and simultaneously maximizing that between inter-class features. Comparative
analyses on four public datasets demonstrate that SleePyCo consistently
outperforms existing frameworks based on single-channel EEG. Extensive ablation
experiments show that SleePyCo exhibits enhanced overall performance, with
significant improvements in discrimination between the N1 and rapid eye
movement (REM) stages.Comment: 14 pages, 3 figures, 8 table
Block Selection Method for Using Feature Norm in Out-of-distribution Detection
Detecting out-of-distribution (OOD) inputs during the inference stage is
crucial for deploying neural networks in the real world. Previous methods
commonly relied on the output of a network derived from the highly activated
feature map. In this study, we first revealed that a norm of the feature map
obtained from the other block than the last block can be a better indicator of
OOD detection. Motivated by this, we propose a simple framework consisting of
FeatureNorm: a norm of the feature map and NormRatio: a ratio of FeatureNorm
for ID and OOD to measure the OOD detection performance of each block. In
particular, to select the block that provides the largest difference between
FeatureNorm of ID and FeatureNorm of OOD, we create Jigsaw puzzle images as
pseudo OOD from ID training samples and calculate NormRatio, and the block with
the largest value is selected. After the suitable block is selected, OOD
detection with the FeatureNorm outperforms other OOD detection methods by
reducing FPR95 by up to 52.77% on CIFAR10 benchmark and by up to 48.53% on
ImageNet benchmark. We demonstrate that our framework can generalize to various
architectures and the importance of block selection, which can improve previous
OOD detection methods as well.Comment: 11 pages including reference. 5 figures and 5 table