43 research outputs found
Revisiting Reweighted Wake-Sleep for Models with Stochastic Control Flow
Stochastic control-flow models (SCFMs) are a class of generative models that
involve branching on choices from discrete random variables. Amortized
gradient-based learning of SCFMs is challenging as most approaches targeting
discrete variables rely on their continuous relaxations---which can be
intractable in SCFMs, as branching on relaxations requires evaluating all
(exponentially many) branching paths. Tractable alternatives mainly combine
REINFORCE with complex control-variate schemes to improve the variance of naive
estimators. Here, we revisit the reweighted wake-sleep (RWS) (Bornschein and
Bengio, 2015) algorithm, and through extensive evaluations, show that it
outperforms current state-of-the-art methods in learning SCFMs. Further, in
contrast to the importance weighted autoencoder, we observe that RWS learns
better models and inference networks with increasing numbers of particles. Our
results suggest that RWS is a competitive, often preferable, alternative for
learning SCFMs.Comment: Tuan Anh Le and Adam R. Kosiorek contributed equally; accepted to
Uncertainty in Artificial Intelligence 201
Natural Evolution Strategies as a Black Box Estimator for Stochastic Variational Inference
Stochastic variational inference and its derivatives in the form of
variational autoencoders enjoy the ability to perform Bayesian inference on
large datasets in an efficient manner. However, performing inference with a VAE
requires a certain design choice (i.e. reparameterization trick) to allow
unbiased and low variance gradient estimation, restricting the types of models
that can be created. To overcome this challenge, an alternative estimator based
on natural evolution strategies is proposed. This estimator does not make
assumptions about the kind of distributions used, allowing for the creation of
models that would otherwise not have been possible under the VAE framework
Optimal Variance Control of the Score Function Gradient Estimator for Importance Weighted Bounds
This paper introduces novel results for the score function gradient estimator
of the importance weighted variational bound (IWAE). We prove that in the limit
of large (number of importance samples) one can choose the control variate
such that the Signal-to-Noise ratio (SNR) of the estimator grows as .
This is in contrast to the standard pathwise gradient estimator where the SNR
decreases as . Based on our theoretical findings we develop a novel
control variate that extends on VIMCO. Empirically, for the training of both
continuous and discrete generative models, the proposed method yields superior
variance reduction, resulting in an SNR for IWAE that increases with
without relying on the reparameterization trick. The novel estimator is
competitive with state-of-the-art reparameterization-free gradient estimators
such as Reweighted Wake-Sleep (RWS) and the thermodynamic variational objective
(TVO) when training generative models
Amortised learning by wake-sleep
Models that employ latent variables to capture structure in observed data lie at the heart of many current unsupervised learning algorithms, but exact maximum-likelihood learning for powerful and flexible latent-variable models is almost always intractable. Thus, state-of-the-art approaches either abandon the maximum-likelihood framework entirely, or else rely on a variety of variational approximations to the posterior distribution over the latents. Here, we propose an alternative approach that we call amortised learning. Rather than computing an approximation to the posterior over latents, we use a wake-sleep Monte-Carlo strategy to learn a function that directly estimates the maximum-likelihood parameter updates. Amortised learning is possible whenever samples of latents and observations can be simulated from the generative model, treating the model as a “black box”. We demonstrate its effectiveness on a wide range of complex models, including those with latents that are discrete or supported on non-Euclidean spaces