163 research outputs found
Semi-supervised semantic segmentation needs strong, varied perturbations
Consistency regularization describes a class of approaches that have yielded
ground breaking results in semi-supervised classification problems. Prior work
has established the cluster assumption - under which the data distribution
consists of uniform class clusters of samples separated by low density regions
- as important to its success. We analyze the problem of semantic segmentation
and find that its' distribution does not exhibit low density regions separating
classes and offer this as an explanation for why semi-supervised segmentation
is a challenging problem, with only a few reports of success. We then identify
choice of augmentation as key to obtaining reliable performance without such
low-density regions. We find that adapted variants of the recently proposed
CutOut and CutMix augmentation techniques yield state-of-the-art
semi-supervised semantic segmentation results in standard datasets.
Furthermore, given its challenging nature we propose that semantic segmentation
acts as an effective acid test for evaluating semi-supervised regularizers.
Implementation at: https://github.com/Britefury/cutmix-semisup-seg.Comment: 21 pages, 7 figures, accepted to BMVC 202
ClassMix: Segmentation-Based Data Augmentation for Semi-Supervised Learning
The state of the art in semantic segmentation is steadily increasing in
performance, resulting in more precise and reliable segmentations in many
different applications. However, progress is limited by the cost of generating
labels for training, which sometimes requires hours of manual labor for a
single image. Because of this, semi-supervised methods have been applied to
this task, with varying degrees of success. A key challenge is that common
augmentations used in semi-supervised classification are less effective for
semantic segmentation. We propose a novel data augmentation mechanism called
ClassMix, which generates augmentations by mixing unlabelled samples, by
leveraging on the network's predictions for respecting object boundaries. We
evaluate this augmentation technique on two common semi-supervised semantic
segmentation benchmarks, showing that it attains state-of-the-art results.
Lastly, we also provide extensive ablation studies comparing different design
decisions and training regimes.Comment: This paper has been accepted to WACV202
Semi-supervised Semantic Segmentation with Prototype-based Consistency Regularization
Semi-supervised semantic segmentation requires the model to effectively
propagate the label information from limited annotated images to unlabeled
ones. A challenge for such a per-pixel prediction task is the large intra-class
variation, i.e., regions belonging to the same class may exhibit a very
different appearance even in the same picture. This diversity will make the
label propagation hard from pixels to pixels. To address this problem, we
propose a novel approach to regularize the distribution of within-class
features to ease label propagation difficulty. Specifically, our approach
encourages the consistency between the prediction from a linear predictor and
the output from a prototype-based predictor, which implicitly encourages
features from the same pseudo-class to be close to at least one within-class
prototype while staying far from the other between-class prototypes. By further
incorporating CutMix operations and a carefully-designed prototype maintenance
strategy, we create a semi-supervised semantic segmentation algorithm that
demonstrates superior performance over the state-of-the-art methods from
extensive experimental evaluation on both Pascal VOC and Cityscapes benchmarks.Comment: Accepted to NeurIPS 202
Switching Temporary Teachers for Semi-Supervised Semantic Segmentation
The teacher-student framework, prevalent in semi-supervised semantic
segmentation, mainly employs the exponential moving average (EMA) to update a
single teacher's weights based on the student's. However, EMA updates raise a
problem in that the weights of the teacher and student are getting coupled,
causing a potential performance bottleneck. Furthermore, this problem may
become more severe when training with more complicated labels such as
segmentation masks but with few annotated data. This paper introduces Dual
Teacher, a simple yet effective approach that employs dual temporary teachers
aiming to alleviate the coupling problem for the student. The temporary
teachers work in shifts and are progressively improved, so consistently prevent
the teacher and student from becoming excessively close. Specifically, the
temporary teachers periodically take turns generating pseudo-labels to train a
student model and maintain the distinct characteristics of the student model
for each epoch. Consequently, Dual Teacher achieves competitive performance on
the PASCAL VOC, Cityscapes, and ADE20K benchmarks with remarkably shorter
training times than state-of-the-art methods. Moreover, we demonstrate that our
approach is model-agnostic and compatible with both CNN- and Transformer-based
models. Code is available at \url{https://github.com/naver-ai/dual-teacher}.Comment: NeurIPS-202
Augmentation Matters: A Simple-yet-Effective Approach to Semi-supervised Semantic Segmentation
Recent studies on semi-supervised semantic segmentation (SSS) have seen fast
progress. Despite their promising performance, current state-of-the-art methods
tend to increasingly complex designs at the cost of introducing more network
components and additional training procedures. Differently, in this work, we
follow a standard teacher-student framework and propose AugSeg, a simple and
clean approach that focuses mainly on data perturbations to boost the SSS
performance. We argue that various data augmentations should be adjusted to
better adapt to the semi-supervised scenarios instead of directly applying
these techniques from supervised learning. Specifically, we adopt a simplified
intensity-based augmentation that selects a random number of data
transformations with uniformly sampling distortion strengths from a continuous
space. Based on the estimated confidence of the model on different unlabeled
samples, we also randomly inject labelled information to augment the unlabeled
samples in an adaptive manner. Without bells and whistles, our simple AugSeg
can readily achieve new state-of-the-art performance on SSS benchmarks under
different partition protocols.Comment: 10 pages, 8 table
SMars: Semi-Supervised Learning for Mars Semantic Segmentation
Deep learning has become a powerful tool for Mars exploration. Mars terrain
semantic segmentation is an important Martian vision task, which is the base of
rover autonomous planning and safe driving. However, there is a lack of
sufficient detailed and high-confidence data annotations, which are exactly
required by most deep learning methods to obtain a good model. To address this
problem, we propose our solution from the perspective of joint data and method
design. We first present a newdataset S5Mars for Semi-SuperviSed learning on
Mars Semantic Segmentation, which contains 6K high-resolution images and is
sparsely annotated based on confidence, ensuring the high quality of labels.
Then to learn from this sparse data, we propose a semi-supervised learning
(SSL) framework for Mars image semantic segmentation, to learn representations
from limited labeled data. Different from the existing SSL methods which are
mostly targeted at the Earth image data, our method takes into account Mars
data characteristics. Specifically, we first investigate the impact of current
widely used natural image augmentations on Mars images. Based on the analysis,
we then proposed two novel and effective augmentations for SSL of Mars
segmentation, AugIN and SAM-Mix, which serve as strong augmentations to boost
the model performance. Meanwhile, to fully leverage the unlabeled data, we
introduce a soft-to-hard consistency learning strategy, learning from different
targets based on prediction confidence. Experimental results show that our
method can outperform state-of-the-art SSL approaches remarkably. Our proposed
dataset is available at https://jhang2020.github.io/S5Mars.github.io/
Co-training with High-Confidence Pseudo Labels for Semi-supervised Medical Image Segmentation
Consistency regularization and pseudo labeling-based semi-supervised methods
perform co-training using the pseudo labels from multi-view inputs. However,
such co-training models tend to converge early to a consensus, degenerating to
the self-training ones, and produce low-confidence pseudo labels from the
perturbed inputs during training. To address these issues, we propose an
Uncertainty-guided Collaborative Mean-Teacher (UCMT) for semi-supervised
semantic segmentation with the high-confidence pseudo labels. Concretely, UCMT
consists of two main components: 1) collaborative mean-teacher (CMT) for
encouraging model disagreement and performing co-training between the
sub-networks, and 2) uncertainty-guided region mix (UMIX) for manipulating the
input images according to the uncertainty maps of CMT and facilitating CMT to
produce high-confidence pseudo labels. Combining the strengths of UMIX with
CMT, UCMT can retain model disagreement and enhance the quality of pseudo
labels for the co-training segmentation. Extensive experiments on four public
medical image datasets including 2D and 3D modalities demonstrate the
superiority of UCMT over the state-of-the-art. Code is available at:
https://github.com/Senyh/UCMT
Instance-specific and Model-adaptive Supervision for Semi-supervised Semantic Segmentation
Recently, semi-supervised semantic segmentation has achieved promising
performance with a small fraction of labeled data. However, most existing
studies treat all unlabeled data equally and barely consider the differences
and training difficulties among unlabeled instances. Differentiating unlabeled
instances can promote instance-specific supervision to adapt to the model's
evolution dynamically. In this paper, we emphasize the cruciality of instance
differences and propose an instance-specific and model-adaptive supervision for
semi-supervised semantic segmentation, named iMAS. Relying on the model's
performance, iMAS employs a class-weighted symmetric intersection-over-union to
evaluate quantitative hardness of each unlabeled instance and supervises the
training on unlabeled data in a model-adaptive manner. Specifically, iMAS
learns from unlabeled instances progressively by weighing their corresponding
consistency losses based on the evaluated hardness. Besides, iMAS dynamically
adjusts the augmentation for each instance such that the distortion degree of
augmented instances is adapted to the model's generalization capability across
the training course. Not integrating additional losses and training procedures,
iMAS can obtain remarkable performance gains against current state-of-the-art
approaches on segmentation benchmarks under different semi-supervised partition
protocols
- …