21 research outputs found
Multi-Adversarial Domain Adaptation
Recent advances in deep domain adaptation reveal that adversarial learning
can be embedded into deep networks to learn transferable features that reduce
distribution discrepancy between the source and target domains. Existing domain
adversarial adaptation methods based on single domain discriminator only align
the source and target data distributions without exploiting the complex
multimode structures. In this paper, we present a multi-adversarial domain
adaptation (MADA) approach, which captures multimode structures to enable
fine-grained alignment of different data distributions based on multiple domain
discriminators. The adaptation can be achieved by stochastic gradient descent
with the gradients computed by back-propagation in linear-time. Empirical
evidence demonstrates that the proposed model outperforms state of the art
methods on standard domain adaptation datasets.Comment: AAAI 2018 Oral. arXiv admin note: substantial text overlap with
arXiv:1705.10667, arXiv:1707.0790
Theoretic Analysis and Extremely Easy Algorithms for Domain Adaptive Feature Learning
Domain adaptation problems arise in a variety of applications, where a
training dataset from the \textit{source} domain and a test dataset from the
\textit{target} domain typically follow different distributions. The primary
difficulty in designing effective learning models to solve such problems lies
in how to bridge the gap between the source and target distributions. In this
paper, we provide comprehensive analysis of feature learning algorithms used in
conjunction with linear classifiers for domain adaptation. Our analysis shows
that in order to achieve good adaptation performance, the second moments of the
source domain distribution and target domain distribution should be similar.
Based on our new analysis, a novel extremely easy feature learning algorithm
for domain adaptation is proposed. Furthermore, our algorithm is extended by
leveraging multiple layers, leading to a deep linear model. We evaluate the
effectiveness of the proposed algorithms in terms of domain adaptation tasks on
the Amazon review dataset and the spam dataset from the ECML/PKDD 2006
discovery challenge.Comment: ijca
Domain adaptation of weighted majority votes via perturbed variation-based self-labeling
In machine learning, the domain adaptation problem arrives when the test
(target) and the train (source) data are generated from different
distributions. A key applied issue is thus the design of algorithms able to
generalize on a new distribution, for which we have no label information. We
focus on learning classification models defined as a weighted majority vote
over a set of real-val ued functions. In this context, Germain et al. (2013)
have shown that a measure of disagreement between these functions is crucial to
control. The core of this measure is a theoretical bound--the C-bound (Lacasse
et al., 2007)--which involves the disagreement and leads to a well performing
majority vote learning algorithm in usual non-adaptative supervised setting:
MinCq. In this work, we propose a framework to extend MinCq to a domain
adaptation scenario. This procedure takes advantage of the recent perturbed
variation divergence between distributions proposed by Harel and Mannor (2012).
Justified by a theoretical bound on the target risk of the vote, we provide to
MinCq a target sample labeled thanks to a perturbed variation-based
self-labeling focused on the regions where the source and target marginals
appear similar. We also study the influence of our self-labeling, from which we
deduce an original process for tuning the hyperparameters. Finally, our
framework called PV-MinCq shows very promising results on a rotation and
translation synthetic problem
Unsupervised Domain Adaptation with Similarity Learning
The objective of unsupervised domain adaptation is to leverage features from
a labeled source domain and learn a classifier for an unlabeled target domain,
with a similar but different data distribution. Most deep learning approaches
to domain adaptation consist of two steps: (i) learn features that preserve a
low risk on labeled samples (source domain) and (ii) make the features from
both domains to be as indistinguishable as possible, so that a classifier
trained on the source can also be applied on the target domain. In general, the
classifiers in step (i) consist of fully-connected layers applied directly on
the indistinguishable features learned in (ii). In this paper, we propose a
different way to do the classification, using similarity learning. The proposed
method learns a pairwise similarity function in which classification can be
performed by computing similarity between prototype representations of each
category. The domain-invariant features and the categorical prototype
representations are learned jointly and in an end-to-end fashion. At inference
time, images from the target domain are compared to the prototypes and the
label associated with the one that best matches the image is outputed. The
approach is simple, scalable and effective. We show that our model achieves
state-of-the-art performance in different unsupervised domain adaptation
scenarios
A Fine-Grain Error Map Prediction and Segmentation Quality Assessment Framework for Whole-Heart Segmentation
When introducing advanced image computing algorithms, e.g., whole-heart
segmentation, into clinical practice, a common suspicion is how reliable the
automatically computed results are. In fact, it is important to find out the
failure cases and identify the misclassified pixels so that they can be
excluded or corrected for the subsequent analysis or diagnosis. However, it is
not a trivial problem to predict the errors in a segmentation mask when ground
truth (usually annotated by experts) is absent. In this work, we attempt to
address the pixel-wise error map prediction problem and the per-case mask
quality assessment problem using a unified deep learning (DL) framework.
Specifically, we first formalize an error map prediction problem, then we
convert it to a segmentation problem and build a DL network to tackle it. We
also derive a quality indicator (QI) from a predicted error map to measure the
overall quality of a segmentation mask. To evaluate the proposed framework, we
perform extensive experiments on a public whole-heart segmentation dataset,
i.e., MICCAI 2017 MMWHS. By 5-fold cross validation, we obtain an overall Dice
score of 0.626 for the error map prediction task, and observe a high Pearson
correlation coefficient (PCC) of 0.972 between QI and the actual segmentation
accuracy (Acc), as well as a low mean absolute error (MAE) of 0.0048 between
them, which evidences the efficacy of our method in both error map prediction
and quality assessment.Comment: 9 pages, accepted by MICCAI'1
Joint Distribution Optimal Transportation for Domain Adaptation
This paper deals with the unsupervised domain adaptation problem, where one
wants to estimate a prediction function in a given target domain without
any labeled sample by exploiting the knowledge available from a source domain
where labels are known. Our work makes the following assumption: there exists a
non-linear transformation between the joint feature/label space distributions
of the two domain and . We propose a solution of
this problem with optimal transport, that allows to recover an estimated target
by optimizing simultaneously the optimal coupling
and . We show that our method corresponds to the minimization of a bound on
the target error, and provide an efficient algorithmic solution, for which
convergence is proved. The versatility of our approach, both in terms of class
of hypothesis or loss functions is demonstrated with real world classification
and regression problems, for which we reach or surpass state-of-the-art
results.Comment: Accepted for publication at NIPS 201
A New PAC-Bayesian Perspective on Domain Adaptation
We study the issue of PAC-Bayesian domain adaptation: We want to learn, from
a source domain, a majority vote model dedicated to a target one. Our
theoretical contribution brings a new perspective by deriving an upper-bound on
the target risk where the distributions' divergence---expressed as a
ratio---controls the trade-off between a source error measure and the target
voters' disagreement. Our bound suggests that one has to focus on regions where
the source data is informative.From this result, we derive a PAC-Bayesian
generalization bound, and specialize it to linear classifiers. Then, we infer a
learning algorithmand perform experiments on real data.Comment: Published at ICML 201