5 research outputs found
Instance-specific and Model-adaptive Supervision for Semi-supervised Semantic Segmentation
Recently, semi-supervised semantic segmentation has achieved promising
performance with a small fraction of labeled data. However, most existing
studies treat all unlabeled data equally and barely consider the differences
and training difficulties among unlabeled instances. Differentiating unlabeled
instances can promote instance-specific supervision to adapt to the model's
evolution dynamically. In this paper, we emphasize the cruciality of instance
differences and propose an instance-specific and model-adaptive supervision for
semi-supervised semantic segmentation, named iMAS. Relying on the model's
performance, iMAS employs a class-weighted symmetric intersection-over-union to
evaluate quantitative hardness of each unlabeled instance and supervises the
training on unlabeled data in a model-adaptive manner. Specifically, iMAS
learns from unlabeled instances progressively by weighing their corresponding
consistency losses based on the evaluated hardness. Besides, iMAS dynamically
adjusts the augmentation for each instance such that the distortion degree of
augmented instances is adapted to the model's generalization capability across
the training course. Not integrating additional losses and training procedures,
iMAS can obtain remarkable performance gains against current state-of-the-art
approaches on segmentation benchmarks under different semi-supervised partition
protocols
Beyond Attentive Tokens: Incorporating Token Importance and Diversity for Efficient Vision Transformers
Vision transformers have achieved significant improvements on various vision
tasks but their quadratic interactions between tokens significantly reduce
computational efficiency. Many pruning methods have been proposed to remove
redundant tokens for efficient vision transformers recently. However, existing
studies mainly focus on the token importance to preserve local attentive tokens
but completely ignore the global token diversity. In this paper, we emphasize
the cruciality of diverse global semantics and propose an efficient token
decoupling and merging method that can jointly consider the token importance
and diversity for token pruning. According to the class token attention, we
decouple the attentive and inattentive tokens. In addition to preserving the
most discriminative local tokens, we merge similar inattentive tokens and match
homogeneous attentive tokens to maximize the token diversity. Despite its
simplicity, our method obtains a promising trade-off between model complexity
and classification accuracy. On DeiT-S, our method reduces the FLOPs by 35%
with only a 0.2% accuracy drop. Notably, benefiting from maintaining the token
diversity, our method can even improve the accuracy of DeiT-T by 0.1% after
reducing its FLOPs by 40%
Augmentation Matters: A Simple-yet-Effective Approach to Semi-supervised Semantic Segmentation
Recent studies on semi-supervised semantic segmentation (SSS) have seen fast
progress. Despite their promising performance, current state-of-the-art methods
tend to increasingly complex designs at the cost of introducing more network
components and additional training procedures. Differently, in this work, we
follow a standard teacher-student framework and propose AugSeg, a simple and
clean approach that focuses mainly on data perturbations to boost the SSS
performance. We argue that various data augmentations should be adjusted to
better adapt to the semi-supervised scenarios instead of directly applying
these techniques from supervised learning. Specifically, we adopt a simplified
intensity-based augmentation that selects a random number of data
transformations with uniformly sampling distortion strengths from a continuous
space. Based on the estimated confidence of the model on different unlabeled
samples, we also randomly inject labelled information to augment the unlabeled
samples in an adaptive manner. Without bells and whistles, our simple AugSeg
can readily achieve new state-of-the-art performance on SSS benchmarks under
different partition protocols.Comment: 10 pages, 8 table