9,496 research outputs found
A review of domain adaptation without target labels
Domain adaptation has become a prominent problem setting in machine learning
and related fields. This review asks the question: how can a classifier learn
from a source domain and generalize to a target domain? We present a
categorization of approaches, divided into, what we refer to as, sample-based,
feature-based and inference-based methods. Sample-based methods focus on
weighting individual observations during training based on their importance to
the target domain. Feature-based methods revolve around on mapping, projecting
and representing features such that a source classifier performs well on the
target domain and inference-based methods incorporate adaptation into the
parameter estimation procedure, for instance through constraints on the
optimization procedure. Additionally, we review a number of conditions that
allow for formulating bounds on the cross-domain generalization error. Our
categorization highlights recurring ideas and raises questions important to
further research.Comment: 20 pages, 5 figure
Optimal Transport for Domain Adaptation
Domain adaptation from one data space (or domain) to another is one of the
most challenging tasks of modern data analytics. If the adaptation is done
correctly, models built on a specific data space become more robust when
confronted to data depicting the same semantic concepts (the classes), but
observed by another observation system with its own specificities. Among the
many strategies proposed to adapt a domain to another, finding a common
representation has shown excellent properties: by finding a common
representation for both domains, a single classifier can be effective in both
and use labelled samples from the source domain to predict the unlabelled
samples of the target domain. In this paper, we propose a regularized
unsupervised optimal transportation model to perform the alignment of the
representations in the source and target domains. We learn a transportation
plan matching both PDFs, which constrains labelled samples in the source domain
to remain close during transport. This way, we exploit at the same time the few
labeled information in the source and the unlabelled distributions observed in
both domains. Experiments in toy and challenging real visual adaptation
examples show the interest of the method, that consistently outperforms state
of the art approaches
Joint Distribution Optimal Transportation for Domain Adaptation
This paper deals with the unsupervised domain adaptation problem, where one
wants to estimate a prediction function in a given target domain without
any labeled sample by exploiting the knowledge available from a source domain
where labels are known. Our work makes the following assumption: there exists a
non-linear transformation between the joint feature/label space distributions
of the two domain and . We propose a solution of
this problem with optimal transport, that allows to recover an estimated target
by optimizing simultaneously the optimal coupling
and . We show that our method corresponds to the minimization of a bound on
the target error, and provide an efficient algorithmic solution, for which
convergence is proved. The versatility of our approach, both in terms of class
of hypothesis or loss functions is demonstrated with real world classification
and regression problems, for which we reach or surpass state-of-the-art
results.Comment: Accepted for publication at NIPS 201
Wasserstein Distance Guided Representation Learning for Domain Adaptation
Domain adaptation aims at generalizing a high-performance learner on a target
domain via utilizing the knowledge distilled from a source domain which has a
different but related data distribution. One solution to domain adaptation is
to learn domain invariant feature representations while the learned
representations should also be discriminative in prediction. To learn such
representations, domain adaptation frameworks usually include a domain
invariant representation learning approach to measure and reduce the domain
discrepancy, as well as a discriminator for classification. Inspired by
Wasserstein GAN, in this paper we propose a novel approach to learn domain
invariant feature representations, namely Wasserstein Distance Guided
Representation Learning (WDGRL). WDGRL utilizes a neural network, denoted by
the domain critic, to estimate empirical Wasserstein distance between the
source and target samples and optimizes the feature extractor network to
minimize the estimated Wasserstein distance in an adversarial manner. The
theoretical advantages of Wasserstein distance for domain adaptation lie in its
gradient property and promising generalization bound. Empirical studies on
common sentiment and image classification adaptation datasets demonstrate that
our proposed WDGRL outperforms the state-of-the-art domain invariant
representation learning approaches.Comment: The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI
2018
- …