2,459 research outputs found
Uncertainty-Aware Consistency Regularization for Cross-Domain Semantic Segmentation
Unsupervised domain adaptation (UDA) aims to adapt existing models of the
source domain to a new target domain with only unlabeled data. Many
adversarial-based UDA methods involve high-instability training and have to
carefully tune the optimization procedure. Some non-adversarial UDA methods
employ a consistency regularization on the target predictions of a student
model and a teacher model under different perturbations, where the teacher
shares the same architecture with the student and is updated by the exponential
moving average of the student. However, these methods suffer from noticeable
negative transfer resulting from either the error-prone discriminator network
or the unreasonable teacher model. In this paper, we propose an
uncertainty-aware consistency regularization method for cross-domain semantic
segmentation. By exploiting the latent uncertainty information of the target
samples, more meaningful and reliable knowledge from the teacher model can be
transferred to the student model. In addition, we further reveal the reason why
the current consistency regularization is often unstable in minimizing the
distribution discrepancy. We also show that our method can effectively ease
this issue by mining the most reliable and meaningful samples with a dynamic
weighting scheme of consistency loss. Experiments demonstrate that the proposed
method outperforms the state-of-the-art methods on two domain adaptation
benchmarks, GTAV Cityscapes and SYNTHIA
Cityscapes
Context-Aware Mixup for Domain Adaptive Semantic Segmentation
Unsupervised domain adaptation (UDA) aims to adapt a model of the labeled
source domain to an unlabeled target domain. Existing UDA-based semantic
segmentation approaches always reduce the domain shifts in pixel level, feature
level, and output level. However, almost all of them largely neglect the
contextual dependency, which is generally shared across different domains,
leading to less-desired performance. In this paper, we propose a novel
Context-Aware Mixup (CAMix) framework for domain adaptive semantic
segmentation, which exploits this important clue of context-dependency as
explicit prior knowledge in a fully end-to-end trainable manner for enhancing
the adaptability toward the target domain. Firstly, we present a contextual
mask generation strategy by leveraging the accumulated spatial distributions
and prior contextual relationships. The generated contextual mask is critical
in this work and will guide the context-aware domain mixup on three different
levels. Besides, provided the context knowledge, we introduce a
significance-reweighted consistency loss to penalize the inconsistency between
the mixed student prediction and the mixed teacher prediction, which alleviates
the negative transfer of the adaptation, e.g., early performance degradation.
Extensive experiments and analysis demonstrate the effectiveness of our method
against the state-of-the-art approaches on widely-used UDA benchmarks.Comment: Accepted to IEEE Transactions on Circuits and Systems for Video
Technology (TCSVT
Hard-aware Instance Adaptive Self-training for Unsupervised Cross-domain Semantic Segmentation
The divergence between labeled training data and unlabeled testing data is a
significant challenge for recent deep learning models. Unsupervised domain
adaptation (UDA) attempts to solve such problem. Recent works show that
self-training is a powerful approach to UDA. However, existing methods have
difficulty in balancing the scalability and performance. In this paper, we
propose a hard-aware instance adaptive self-training framework for UDA on the
task of semantic segmentation. To effectively improve the quality and diversity
of pseudo-labels, we develop a novel pseudo-label generation strategy with an
instance adaptive selector. We further enrich the hard class pseudo-labels with
inter-image information through a skillfully designed hard-aware pseudo-label
augmentation. Besides, we propose the region-adaptive regularization to smooth
the pseudo-label region and sharpen the non-pseudo-label region. For the
non-pseudo-label region, consistency constraint is also constructed to
introduce stronger supervision signals during model optimization. Our method is
so concise and efficient that it is easy to be generalized to other UDA
methods. Experiments on GTA5 to Cityscapes, SYNTHIA to Cityscapes, and
Cityscapes to Oxford RobotCar demonstrate the superior performance of our
approach compared with the state-of-the-art methods.Comment: arXiv admin note: text overlap with arXiv:2008.1219
DACS: Domain Adaptation via Cross-domain Mixed Sampling
Semantic segmentation models based on convolutional neural networks have
recently displayed remarkable performance for a multitude of applications.
However, these models typically do not generalize well when applied on new
domains, especially when going from synthetic to real data. In this paper we
address the problem of unsupervised domain adaptation (UDA), which attempts to
train on labelled data from one domain (source domain), and simultaneously
learn from unlabelled data in the domain of interest (target domain). Existing
methods have seen success by training on pseudo-labels for these unlabelled
images. Multiple techniques have been proposed to mitigate low-quality
pseudo-labels arising from the domain shift, with varying degrees of success.
We propose DACS: Domain Adaptation via Cross-domain mixed Sampling, which mixes
images from the two domains along with the corresponding labels and
pseudo-labels. These mixed samples are then trained on, in addition to the
labelled data itself. We demonstrate the effectiveness of our solution by
achieving state-of-the-art results for GTA5 to Cityscapes, a common
synthetic-to-real semantic segmentation benchmark for UDA.Comment: This paper has been accepted to WACV202
Learning with Limited Annotations: A Survey on Deep Semi-Supervised Learning for Medical Image Segmentation
Medical image segmentation is a fundamental and critical step in many
image-guided clinical approaches. Recent success of deep learning-based
segmentation methods usually relies on a large amount of labeled data, which is
particularly difficult and costly to obtain especially in the medical imaging
domain where only experts can provide reliable and accurate annotations.
Semi-supervised learning has emerged as an appealing strategy and been widely
applied to medical image segmentation tasks to train deep models with limited
annotations. In this paper, we present a comprehensive review of recently
proposed semi-supervised learning methods for medical image segmentation and
summarized both the technical novelties and empirical results. Furthermore, we
analyze and discuss the limitations and several unsolved problems of existing
approaches. We hope this review could inspire the research community to explore
solutions for this challenge and further promote the developments in medical
image segmentation field
Exploring Feature Representation Learning for Semi-supervised Medical Image Segmentation
This paper presents a simple yet effective two-stage framework for
semi-supervised medical image segmentation. Our key insight is to explore the
feature representation learning with labeled and unlabeled (i.e., pseudo
labeled) images to enhance the segmentation performance. In the first stage, we
present an aleatoric uncertainty-aware method, namely AUA, to improve the
segmentation performance for generating high-quality pseudo labels. Considering
the inherent ambiguity of medical images, AUA adaptively regularizes the
consistency on images with low ambiguity. To enhance the representation
learning, we propose a stage-adaptive contrastive learning method, including a
boundary-aware contrastive loss to regularize the labeled images in the first
stage and a prototype-aware contrastive loss to optimize both labeled and
pseudo labeled images in the second stage. The boundary-aware contrastive loss
only optimizes pixels around the segmentation boundaries to reduce the
computational cost. The prototype-aware contrastive loss fully leverages both
labeled images and pseudo labeled images by building a centroid for each class
to reduce computational cost for pair-wise comparison. Our method achieves the
best results on two public medical image segmentation benchmarks. Notably, our
method outperforms the prior state-of-the-art by 5.7% on Dice for colon tumor
segmentation relying on just 5% labeled images.Comment: On submission to TM
- …