400 research outputs found
Learning Bijective Feature Maps for Linear ICA
Separating high-dimensional data like images into independent latent factors,
i.e independent component analysis (ICA), remains an open research problem. As
we show, existing probabilistic deep generative models (DGMs), which are
tailor-made for image data, underperform on non-linear ICA tasks. To address
this, we propose a DGM which combines bijective feature maps with a linear ICA
model to learn interpretable latent structures for high-dimensional data. Given
the complexities of jointly training such a hybrid model, we introduce novel
theory that constrains linear ICA to lie close to the manifold of orthogonal
rectangular matrices, the Stiefel manifold. By doing so we create models that
converge quickly, are easy to train, and achieve better unsupervised latent
factor discovery than flow-based models, linear ICA, and Variational
Autoencoders on images.Comment: 8 page
Indeterminacy and Strong Identifiability in Generative Models
Most modern probabilistic generative models, such as the variational
autoencoder (VAE), have certain indeterminacies that are unresolvable even with
an infinite amount of data. Different tasks tolerate different indeterminacies,
however recent applications have indicated the need for strongly identifiable
models, in which an observation corresponds to a unique latent code. Progress
has been made towards reducing model indeterminacies while maintaining
flexibility, and recent work excludes many--but not all--indeterminacies. In
this work, we motivate model-identifiability in terms of task-identifiability,
then construct a theoretical framework for analyzing the indeterminacies of
latent variable models, which enables their precise characterization in terms
of the generator function and prior distribution spaces. We reveal that strong
identifiability is possible even with highly flexible nonlinear generators, and
give two such examples. One is a straightforward modification of iVAE
(arXiv:1907.04809 [stat.ML]); the other uses triangular monotonic maps, leading
to novel connections between optimal transport and identifiability
- …