1,105 research outputs found
Disentangling and Learning Robust Representations with Natural Clustering
Learning representations that disentangle the underlying factors of
variability in data is an intuitive way to achieve generalization in deep
models. In this work, we address the scenario where generative factors present
a multimodal distribution due to the existence of class distinction in the
data. We propose N-VAE, a model which is capable of separating factors of
variation which are exclusive to certain classes from factors that are shared
among classes. This model implements an explicitly compositional latent
variable structure by defining a class-conditioned latent space and a shared
latent space. We show its usefulness for detecting and disentangling
class-dependent generative factors as well as its capacity to generate
artificial samples which contain characteristics unseen in the training data.Comment: Accepted at ICMLA 201
Disentangled Representation Learning
Disentangled Representation Learning (DRL) aims to learn a model capable of
identifying and disentangling the underlying factors hidden in the observable
data in representation form. The process of separating underlying factors of
variation into variables with semantic meaning benefits in learning explainable
representations of data, which imitates the meaningful understanding process of
humans when observing an object or relation. As a general learning strategy,
DRL has demonstrated its power in improving the model explainability,
controlability, robustness, as well as generalization capacity in a wide range
of scenarios such as computer vision, natural language processing, data mining
etc. In this article, we comprehensively review DRL from various aspects
including motivations, definitions, methodologies, evaluations, applications
and model designs. We discuss works on DRL based on two well-recognized
definitions, i.e., Intuitive Definition and Group Theory Definition. We further
categorize the methodologies for DRL into four groups, i.e., Traditional
Statistical Approaches, Variational Auto-encoder Based Approaches, Generative
Adversarial Networks Based Approaches, Hierarchical Approaches and Other
Approaches. We also analyze principles to design different DRL models that may
benefit different tasks in practical applications. Finally, we point out
challenges in DRL as well as potential research directions deserving future
investigations. We believe this work may provide insights for promoting the DRL
research in the community.Comment: 22 pages,9 figure
- …