24 research outputs found
Re-weighted Adversarial Adaptation Network for Unsupervised Domain Adaptation
Unsupervised Domain Adaptation (UDA) aims to trans- fer domain knowledge from existing well-defined tasks to new ones where labels are unavailable. In the real-world applications, as the domain (task) discrepancies are usu- ally uncontrollable, it is significantly motivated to match the feature distributions even if the domain discrepancies are disparate. Additionally, as no label is available in the target domain, how to successfully adapt the classifier from the source to the target domain still remains an open ques- tion. In this paper, we propose the Re-weighted Adversarial Adaptation Network (RAAN) to reduce the feature distribu- tion divergence and adapt the classifier when domain dis- crepancies are disparate. Specifically, to alleviate the need of common supports in matching the feature distribution, we choose to minimize optimal transport (OT) based Earth- Mover (EM) distance and reformulate it to a minimax ob- jective function. Utilizing this, RAAN can be trained in an end-to-end and adversarial manner. To further adapt the classifier, we propose to match the label distribution and embed it into the adversarial training. Finally, after ex- tensive evaluation of our method using UDA datasets of varying difficulty, RAAN achieved the state-of-the-art re- sults and outperformed other methods by a large margin when the domain shifts are disparate
Re-Weighted Adversarial Adaptation Network for Unsupervised Domain Adaptation
Unsupervised Domain Adaptation (UDA) aims to transfer domain knowledge from existing well-defined tasks to new ones where labels are unavailable. In the real-world applications, as the domain (task) discrepancies are usually uncontrollable, it is significantly motivated to match the feature distributions even if the domain discrepancies are disparate. Additionally, as no label is available in the target domain, how to successfully adapt the classifier from the source to the target domain still remains an open question. In this paper, we propose the Re-weighted Adversarial Adaptation Network (RAAN) to reduce the feature distribution divergence and adapt the classifier when domain discrepancies are disparate. Specifically, to alleviate the need of common supports in matching the feature distribution, we choose to minimize optimal transport (OT) based Earth-Mover (EM) distance and reformulate it to a minimax objective function. Utilizing this, RAAN can be trained in an end-to-end and adversarial manner. To further adapt the classifier, we propose to match the label distribution and embed it into the adversarial training. Finally, after extensive evaluation of our method using UDA datasets of varying difficulty, RAAN achieved the state-of-the-art results and outperformed other methods by a large margin when the domain shifts are disparate
Improved Techniques for Adversarial Discriminative Domain Adaptation
Adversarial discriminative domain adaptation (ADDA) is an efficient framework
for unsupervised domain adaptation in image classification, where the source
and target domains are assumed to have the same classes, but no labels are
available for the target domain. We investigate whether we can improve
performance of ADDA with a new framework and new loss formulations. Following
the framework of semi-supervised GANs, we first extend the discriminator output
over the source classes, in order to model the joint distribution over domain
and task. We thus leverage on the distribution over the source encoder
posteriors (which is fixed during adversarial training) and propose maximum
mean discrepancy (MMD) and reconstruction-based loss functions for aligning the
target encoder distribution to the source domain. We compare and provide a
comprehensive analysis of how our framework and loss formulations extend over
simple multi-class extensions of ADDA and other discriminative variants of
semi-supervised GANs. In addition, we introduce various forms of regularization
for stabilizing training, including treating the discriminator as a denoising
autoencoder and regularizing the target encoder with source examples to reduce
overfitting under a contraction mapping (i.e., when the target per-class
distributions are contracting during alignment with the source). Finally, we
validate our framework on standard domain adaptation datasets, such as SVHN and
MNIST. We also examine how our framework benefits recognition problems based on
modalities that lack training data, by introducing and evaluating on a
neuromorphic vision sensing (NVS) sign language recognition dataset, where the
source and target domains constitute emulated and real neuromorphic spike
events respectively. Our results on all datasets show that our proposal
competes or outperforms the state-of-the-art in unsupervised domain adaptation.Comment: To appear in IEEE Transactions on Image Processin
Curriculum based dropout discriminator for domain adaptation
Domain adaptation is essential to enable wide usage of deep learning based
networks trained using large labeled datasets. Adversarial learning based
techniques have shown their utility towards solving this problem using a
discriminator that ensures source and target distributions are close. However,
here we suggest that rather than using a point estimate, it would be useful if
a distribution based discriminator could be used to bridge this gap. This could
be achieved using multiple classifiers or using traditional ensemble methods.
In contrast, we suggest that a Monte Carlo dropout based ensemble discriminator
could suffice to obtain the distribution based discriminator. Specifically, we
propose a curriculum based dropout discriminator that gradually increases the
variance of the sample based distribution and the corresponding reverse
gradients are used to align the source and target feature representations. The
detailed results and thorough ablation analysis show that our model outperforms
state-of-the-art results.Comment: BMVC 2019 Accepted, Project Page:
https://delta-lab-iitk.github.io/CD3A
Mining Label Distribution Drift in Unsupervised Domain Adaptation
Unsupervised domain adaptation targets to transfer task knowledge from
labeled source domain to related yet unlabeled target domain, and is catching
extensive interests from academic and industrial areas. Although tremendous
efforts along this direction have been made to minimize the domain divergence,
unfortunately, most of existing methods only manage part of the picture by
aligning feature representations from different domains. Beyond the discrepancy
in feature space, the gap between known source label and unknown target label
distribution, recognized as label distribution drift, is another crucial factor
raising domain divergence, and has not been paid enough attention and well
explored. From this point, in this paper, we first experimentally reveal how
label distribution drift brings negative effects on current domain adaptation
methods. Next, we propose Label distribution Matching Domain Adversarial
Network (LMDAN) to handle data distribution shift and label distribution drift
jointly. In LMDAN, label distribution drift problem is addressed by the
proposed source samples weighting strategy, which select samples to contribute
to positive adaptation and avoid negative effects brought by the mismatched in
label distribution. Finally, different from general domain adaptation
experiments, we modify domain adaptation datasets to create the considerable
label distribution drift between source and target domain. Numerical results
and empirical model analysis show that LMDAN delivers superior performance
compared to other state-of-the-art domain adaptation methods under such
scenarios