119,636 research outputs found
Optimal Transport for Domain Adaptation
Domain adaptation from one data space (or domain) to another is one of the
most challenging tasks of modern data analytics. If the adaptation is done
correctly, models built on a specific data space become more robust when
confronted to data depicting the same semantic concepts (the classes), but
observed by another observation system with its own specificities. Among the
many strategies proposed to adapt a domain to another, finding a common
representation has shown excellent properties: by finding a common
representation for both domains, a single classifier can be effective in both
and use labelled samples from the source domain to predict the unlabelled
samples of the target domain. In this paper, we propose a regularized
unsupervised optimal transportation model to perform the alignment of the
representations in the source and target domains. We learn a transportation
plan matching both PDFs, which constrains labelled samples in the source domain
to remain close during transport. This way, we exploit at the same time the few
labeled information in the source and the unlabelled distributions observed in
both domains. Experiments in toy and challenging real visual adaptation
examples show the interest of the method, that consistently outperforms state
of the art approaches
Optimal Transport for Domain Adaptation
International audienceDomain adaptation is one of the most chal- lenging tasks of modern data analytics. If the adapta- tion is done correctly, models built on a specific data representation become more robust when confronted to data depicting the same classes, but described by another observation system. Among the many strategies proposed, finding domain-invariant representations has shown excel- lent properties, in particular since it allows to train a unique classifier effective in all domains. In this paper, we propose a regularized unsupervised optimal transportation model to perform the alignment of the representations in the source and target domains. We learn a transportation plan matching both PDFs, which constrains labeled samples of the same class in the source domain to remain close during transport. This way, we exploit at the same time the labeled samples in the source and the distributions observed in both domains. Experiments on toy and challenging real visual adaptation examples show the interest of the method, that consistently outperforms state of the art approaches. In addition, numerical experiments show that our approach leads to better performances on domain invariant deep learning features and can be easily adapted to the semi- supervised case where few labeled samples are available in the target domain
Hierarchical Optimal Transport for Unsupervised Domain Adaptation
In this paper, we propose a novel approach for unsupervised domain
adaptation, that relates notions of optimal transport, learning probability
measures and unsupervised learning. The proposed approach, HOT-DA, is based on
a hierarchical formulation of optimal transport, that leverages beyond the
geometrical information captured by the ground metric, richer structural
information in the source and target domains. The additional information in the
labeled source domain is formed instinctively by grouping samples into
structures according to their class labels. While exploring hidden structures
in the unlabeled target domain is reduced to the problem of learning
probability measures through Wasserstein barycenter, which we prove to be
equivalent to spectral clustering. Experiments on a toy dataset with
controllable complexity and two challenging visual adaptation datasets show the
superiority of the proposed approach over the state-of-the-art
Unsupervised Domain Adaptation via Deep Hierarchical Optimal Transport
Unsupervised domain adaptation is a challenging task that aims to estimate a
transferable model for unlabeled target domain by exploiting source labeled
data. Optimal Transport (OT) based methods recently have been proven to be a
promising direction for domain adaptation due to their competitive performance.
However, most of these methods coarsely aligned source and target
distributions, leading to the over-aligned problem where the
category-discriminative information is mixed up although domain-invariant
representations can be learned. In this paper, we propose a Deep Hierarchical
Optimal Transport method (DeepHOT) for unsupervised domain adaptation. The main
idea is to use hierarchical optimal transport to learn both domain-invariant
and category-discriminative representations by mining the rich structural
correlations among domain data. The DeepHOT framework consists of a
domain-level OT and an image-level OT, where the latter is used as the ground
distance metric for the former. The image-level OT captures structural
associations of local image regions that are beneficial to image
classification, while the domain-level OT learns domain-invariant
representations by leveraging the underlying geometry of domains. However, due
to the high computational complexity, the optimal transport based models are
limited in some scenarios. To this end, we propose a robust and efficient
implementation of the DeepHOT framework by approximating origin OT with sliced
Wasserstein distance in image-level OT and using a mini-batch unbalanced
optimal transport for domain-level OT. Extensive experiments show that DeepHOT
surpasses the state-of-the-art methods in four benchmark datasets. Code will be
released on GitHub.Comment: 9 pages, 3 figure
Semi-supervised Learning of Pushforwards For Domain Translation & Adaptation
Given two probability densities on related data spaces, we seek a map pushing
one density to the other while satisfying application-dependent constraints.
For maps to have utility in a broad application space (including domain
translation, domain adaptation, and generative modeling), the map must be
available to apply on out-of-sample data points and should correspond to a
probabilistic model over the two spaces. Unfortunately, existing approaches,
which are primarily based on optimal transport, do not address these needs. In
this paper, we introduce a novel pushforward map learning algorithm that
utilizes normalizing flows to parameterize the map. We first re-formulate the
classical optimal transport problem to be map-focused and propose a learning
algorithm to select from all possible maps under the constraint that the map
minimizes a probability distance and application-specific regularizers; thus,
our method can be seen as solving a modified optimal transport problem. Once
the map is learned, it can be used to map samples from a source domain to a
target domain. In addition, because the map is parameterized as a composition
of normalizing flows, it models the empirical distributions over the two data
spaces and allows both sampling and likelihood evaluation for both data sets.
We compare our method (parOT) to related optimal transport approaches in the
context of domain adaptation and domain translation on benchmark data sets.
Finally, to illustrate the impact of our work on applied problems, we apply
parOT to a real scientific application: spectral calibration for
high-dimensional measurements from two vastly different environmentsComment: 19 pages, 7 figure
Joint Distribution Optimal Transportation for Domain Adaptation
This paper deals with the unsupervised domain adaptation problem, where one
wants to estimate a prediction function in a given target domain without
any labeled sample by exploiting the knowledge available from a source domain
where labels are known. Our work makes the following assumption: there exists a
non-linear transformation between the joint feature/label space distributions
of the two domain and . We propose a solution of
this problem with optimal transport, that allows to recover an estimated target
by optimizing simultaneously the optimal coupling
and . We show that our method corresponds to the minimization of a bound on
the target error, and provide an efficient algorithmic solution, for which
convergence is proved. The versatility of our approach, both in terms of class
of hypothesis or loss functions is demonstrated with real world classification
and regression problems, for which we reach or surpass state-of-the-art
results.Comment: Accepted for publication at NIPS 201
Functional optimal transport: map estimation and domain adaptation for functional data
We introduce a formulation of optimal transport problem for distributions on
function spaces, where the stochastic map between functional domains can be
partially represented in terms of an (infinite-dimensional) Hilbert-Schmidt
operator mapping a Hilbert space of functions to another. For numerous machine
learning tasks, data can be naturally viewed as samples drawn from spaces of
functions, such as curves and surfaces, in high dimensions. Optimal transport
for functional data analysis provides a useful framework of treatment for such
domains. In this work, we develop an efficient algorithm for finding the
stochastic transport map between functional domains and provide theoretical
guarantees on the existence, uniqueness, and consistency of our estimate for
the Hilbert-Schmidt operator. We validate our method on synthetic datasets and
study the geometric properties of the transport map. Experiments on real-world
datasets of robot arm trajectories further demonstrate the effectiveness of our
method on applications in domain adaptation.Comment: 23 pages, 6 figures, 2 table
- …