3 research outputs found
Unsupervised feature learning with discriminative encoder
In recent years, deep discriminative models have achieved extraordinary
performance on supervised learning tasks, significantly outperforming their
generative counterparts. However, their success relies on the presence of a
large amount of labeled data. How can one use the same discriminative models
for learning useful features in the absence of labels? We address this question
in this paper, by jointly modeling the distribution of data and latent features
in a manner that explicitly assigns zero probability to unobserved data. Rather
than maximizing the marginal probability of observed data, we maximize the
joint probability of the data and the latent features using a two step EM-like
procedure. To prevent the model from overfitting to our initial selection of
latent features, we use adversarial regularization. Depending on the task, we
allow the latent features to be one-hot or real-valued vectors and define a
suitable prior on the features. For instance, one-hot features correspond to
class labels and are directly used for the unsupervised and semi-supervised
classification task, whereas real-valued feature vectors are fed as input to
simple classifiers for auxiliary supervised discrimination tasks. The proposed
model, which we dub discriminative encoder (or DisCoder), is flexible in the
type of latent features that it can capture. The proposed model achieves
state-of-the-art performance on several challenging tasks.Comment: 10 pages, 4 figures, International Conference on Data Mining, 201
CUDA: Contradistinguisher for Unsupervised Domain Adaptation
In this paper, we propose a simple model referred as Contradistinguisher
(CTDR) for unsupervised domain adaptation whose objective is to jointly learn
to contradistinguish on unlabeled target domain in a fully unsupervised manner
along with prior knowledge acquired by supervised learning on an entirely
different domain. Most recent works in domain adaptation rely on an indirect
way of first aligning the source and target domain distributions and then learn
a classifier on a labeled source domain to classify target domain. This
approach of an indirect way of addressing the real task of unlabeled target
domain classification has three main drawbacks. (i) The sub-task of obtaining a
perfect alignment of the domain in itself might be impossible due to large
domain shift (e.g., language domains). (ii) The use of multiple classifiers to
align the distributions unnecessarily increases the complexity of the neural
networks leading to over-fitting in many cases. (iii) Due to distribution
alignment, the domain-specific information is lost as the domains get morphed.
In this work, we propose a simple and direct approach that does not require
domain alignment. We jointly learn CTDR on both source and target distribution
for unsupervised domain adaptation task using contradistinguish loss for the
unlabeled target domain in conjunction with a supervised loss for labeled
source domain. Our experiments show that avoiding domain alignment by directly
addressing the task of unlabeled target domain classification using CTDR
achieves state-of-the-art results on eight visual and four language benchmark
domain adaptation datasets.Comment: International Conference on Data Mining, ICDM 201