86 research outputs found
Empirical Optimal Transport between Conditional Distributions
Given samples from two joint distributions, we consider the problem of
Optimal Transportation (OT) between the corresponding distributions conditioned
on a common variable. The objective of this work is to estimate the associated
transport cost (Wasserstein distance) as well as the transport plan between the
conditionals as a function of the conditioned value. Since matching conditional
distributions is at the core of supervised training of discriminative models
and (implicit) conditional-generative models, OT between conditionals has the
potential to be employed in diverse machine learning applications. However,
since the conditionals involved in OT are implicitly specified via the joint
samples, it is challenging to formulate this problem, especially when (i) the
variable conditioned on is continuous and (ii) the marginal of this variable in
the two distributions is different. We overcome these challenges by employing a
specific kernel MMD (Maximum Mean Discrepancy) based regularizer that ensures
the marginals of our conditional transport plan are close to the conditionals
specified via the given joint samples. Under mild conditions, we prove that our
estimator for this regularized transport cost is statistically consistent and
derive finite-sample bounds on the estimation error. Application-specific
details for parameterizing our conditional transport plan are also presented.
Furthermore, we empirically evaluate our methodology on benchmark datasets in
applications like classification, prompt learning for few-shot classification,
and conditional-generation in the context of predicting cell responses to
cancer treatment
Co-regularized Alignment for Unsupervised Domain Adaptation
Deep neural networks, trained with large amount of labeled data, can fail to
generalize well when tested with examples from a \emph{target domain} whose
distribution differs from the training data distribution, referred as the
\emph{source domain}. It can be expensive or even infeasible to obtain required
amount of labeled data in all possible domains. Unsupervised domain adaptation
sets out to address this problem, aiming to learn a good predictive model for
the target domain using labeled examples from the source domain but only
unlabeled examples from the target domain. Domain alignment approaches this
problem by matching the source and target feature distributions, and has been
used as a key component in many state-of-the-art domain adaptation methods.
However, matching the marginal feature distributions does not guarantee that
the corresponding class conditional distributions will be aligned across the
two domains. We propose co-regularized domain alignment for unsupervised domain
adaptation, which constructs multiple diverse feature spaces and aligns source
and target distributions in each of them individually, while encouraging that
alignments agree with each other with regard to the class predictions on the
unlabeled target examples. The proposed method is generic and can be used to
improve any domain adaptation method which uses domain alignment. We
instantiate it in the context of a recent state-of-the-art method and observe
that it provides significant performance improvements on several domain
adaptation benchmarks.Comment: NIPS 2018 accepted versio
- …