571 research outputs found
Improving Generalization for Abstract Reasoning Tasks Using Disentangled Feature Representations
In this work we explore the generalization characteristics of unsupervised
representation learning by leveraging disentangled VAE's to learn a useful
latent space on a set of relational reasoning problems derived from Raven
Progressive Matrices. We show that the latent representations, learned by
unsupervised training using the right objective function, significantly
outperform the same architectures trained with purely supervised learning,
especially when it comes to generalization
Disentangling Factors of Variation with Cycle-Consistent Variational Auto-Encoders
Generative models that learn disentangled representations for different
factors of variation in an image can be very useful for targeted data
augmentation. By sampling from the disentangled latent subspace of interest, we
can efficiently generate new data necessary for a particular task. Learning
disentangled representations is a challenging problem, especially when certain
factors of variation are difficult to label. In this paper, we introduce a
novel architecture that disentangles the latent space into two complementary
subspaces by using only weak supervision in form of pairwise similarity labels.
Inspired by the recent success of cycle-consistent adversarial architectures,
we use cycle-consistency in a variational auto-encoder framework. Our
non-adversarial approach is in contrast with the recent works that combine
adversarial training with auto-encoders to disentangle representations. We show
compelling results of disentangled latent subspaces on three datasets and
compare with recent works that leverage adversarial training
Disentanglement of Correlated Factors via Hausdorff Factorized Support
A grand goal in deep learning research is to learn representations capable of
generalizing across distribution shifts. Disentanglement is one promising
direction aimed at aligning a models representations with the underlying
factors generating the data (e.g. color or background). Existing
disentanglement methods, however, rely on an often unrealistic assumption: that
factors are statistically independent. In reality, factors (like object color
and shape) are correlated. To address this limitation, we propose a relaxed
disentanglement criterion - the Hausdorff Factorized Support (HFS) criterion -
that encourages a factorized support, rather than a factorial distribution, by
minimizing a Hausdorff distance. This allows for arbitrary distributions of the
factors over their support, including correlations between them. We show that
the use of HFS consistently facilitates disentanglement and recovery of
ground-truth factors across a variety of correlation settings and benchmarks,
even under severe training correlations and correlation shifts, with in parts
over +60% in relative improvement over existing disentanglement methods. In
addition, we find that leveraging HFS for representation learning can even
facilitate transfer to downstream tasks such as classification under
distribution shifts. We hope our original approach and positive empirical
results inspire further progress on the open problem of robust generalization
- …