2,745 research outputs found
Adaptive Density Estimation for Generative Models
Unsupervised learning of generative models has seen tremendous progress over
recent years, in particular due to generative adversarial networks (GANs),
variational autoencoders, and flow-based models. GANs have dramatically
improved sample quality, but suffer from two drawbacks: (i) they mode-drop,
i.e., do not cover the full support of the train data, and (ii) they do not
allow for likelihood evaluations on held-out data. In contrast,
likelihood-based training encourages models to cover the full support of the
train data, but yields poorer samples. These mutual shortcomings can in
principle be addressed by training generative latent variable models in a
hybrid adversarial-likelihood manner. However, we show that commonly made
parametric assumptions create a conflict between them, making successful hybrid
models non trivial. As a solution, we propose to use deep invertible
transformations in the latent variable decoder. This approach allows for
likelihood computations in image space, is more efficient than fully invertible
models, and can take full advantage of adversarial training. We show that our
model significantly improves over existing hybrid models: offering GAN-like
samples, IS and FID scores that are competitive with fully adversarial models,
and improved likelihood scores
BRUNO: A Deep Recurrent Model for Exchangeable Data
We present a novel model architecture which leverages deep learning tools to
perform exact Bayesian inference on sets of high dimensional, complex
observations. Our model is provably exchangeable, meaning that the joint
distribution over observations is invariant under permutation: this property
lies at the heart of Bayesian inference. The model does not require variational
approximations to train, and new samples can be generated conditional on
previous samples, with cost linear in the size of the conditioning set. The
advantages of our architecture are demonstrated on learning tasks that require
generalisation from short observed sequences while modelling sequence
variability, such as conditional image generation, few-shot learning, and
anomaly detection.Comment: NIPS 201
Flow-GAN: Combining Maximum Likelihood and Adversarial Learning in Generative Models
Adversarial learning of probabilistic models has recently emerged as a
promising alternative to maximum likelihood. Implicit models such as generative
adversarial networks (GAN) often generate better samples compared to explicit
models trained by maximum likelihood. Yet, GANs sidestep the characterization
of an explicit density which makes quantitative evaluations challenging. To
bridge this gap, we propose Flow-GANs, a generative adversarial network for
which we can perform exact likelihood evaluation, thus supporting both
adversarial and maximum likelihood training. When trained adversarially,
Flow-GANs generate high-quality samples but attain extremely poor
log-likelihood scores, inferior even to a mixture model memorizing the training
data; the opposite is true when trained by maximum likelihood. Results on MNIST
and CIFAR-10 demonstrate that hybrid training can attain high held-out
likelihoods while retaining visual fidelity in the generated samples.Comment: AAAI 201
- …