22 research outputs found
Towards Adaptive Semantic Segmentation by Progressive Feature Refinement
As one of the fundamental tasks in computer vision, semantic segmentation
plays an important role in real world applications. Although numerous deep
learning models have made notable progress on several mainstream datasets with
the rapid development of convolutional networks, they still encounter various
challenges in practical scenarios. Unsupervised adaptive semantic segmentation
aims to obtain a robust classifier trained with source domain data, which is
able to maintain stable performance when deployed to a target domain with
different data distribution. In this paper, we propose an innovative
progressive feature refinement framework, along with domain adversarial
learning to boost the transferability of segmentation networks. Specifically,
we firstly align the multi-stage intermediate feature maps of source and target
domain images, and then a domain classifier is adopted to discriminate the
segmentation output. As a result, the segmentation models trained with source
domain images can be transferred to a target domain without significant
performance degradation. Experimental results verify the efficiency of our
proposed method compared with state-of-the-art methods
Uncertainty-Aware Consistency Regularization for Cross-Domain Semantic Segmentation
Unsupervised domain adaptation (UDA) aims to adapt existing models of the
source domain to a new target domain with only unlabeled data. Many
adversarial-based UDA methods involve high-instability training and have to
carefully tune the optimization procedure. Some non-adversarial UDA methods
employ a consistency regularization on the target predictions of a student
model and a teacher model under different perturbations, where the teacher
shares the same architecture with the student and is updated by the exponential
moving average of the student. However, these methods suffer from noticeable
negative transfer resulting from either the error-prone discriminator network
or the unreasonable teacher model. In this paper, we propose an
uncertainty-aware consistency regularization method for cross-domain semantic
segmentation. By exploiting the latent uncertainty information of the target
samples, more meaningful and reliable knowledge from the teacher model can be
transferred to the student model. In addition, we further reveal the reason why
the current consistency regularization is often unstable in minimizing the
distribution discrepancy. We also show that our method can effectively ease
this issue by mining the most reliable and meaningful samples with a dynamic
weighting scheme of consistency loss. Experiments demonstrate that the proposed
method outperforms the state-of-the-art methods on two domain adaptation
benchmarks, GTAV Cityscapes and SYNTHIA
Cityscapes
Unsupervised Domain Adaptation with Multiple Domain Discriminators and Adaptive Self-Training
Unsupervised Domain Adaptation (UDA) aims at improving the generalization
capability of a model trained on a source domain to perform well on a target
domain for which no labeled data is available. In this paper, we consider the
semantic segmentation of urban scenes and we propose an approach to adapt a
deep neural network trained on synthetic data to real scenes addressing the
domain shift between the two different data distributions. We introduce a novel
UDA framework where a standard supervised loss on labeled synthetic data is
supported by an adversarial module and a self-training strategy aiming at
aligning the two domain distributions. The adversarial module is driven by a
couple of fully convolutional discriminators dealing with different domains:
the first discriminates between ground truth and generated maps, while the
second between segmentation maps coming from synthetic or real world data. The
self-training module exploits the confidence estimated by the discriminators on
unlabeled data to select the regions used to reinforce the learning process.
Furthermore, the confidence is thresholded with an adaptive mechanism based on
the per-class overall confidence. Experimental results prove the effectiveness
of the proposed strategy in adapting a segmentation network trained on
synthetic datasets like GTA5 and SYNTHIA, to real world datasets like
Cityscapes and Mapillary.Comment: 8 pages, 3 figures, 2 table