2 research outputs found
Learning Model Reparametrizations: Implicit Variational Inference by Fitting MCMC distributions
We introduce a new algorithm for approximate inference that combines
reparametrization, Markov chain Monte Carlo and variational methods. We
construct a very flexible implicit variational distribution synthesized by an
arbitrary Markov chain Monte Carlo operation and a deterministic transformation
that can be optimized using the reparametrization trick. Unlike current methods
for implicit variational inference, our method avoids the computation of log
density ratios and therefore it is easily applicable to arbitrary continuous
and differentiable models. We demonstrate the proposed algorithm for fitting
banana-shaped distributions and for training variational autoencoders.Comment: 16 pages, 6 figure
Fast and Accurate Variational Inference for Models with Many Latent Variables
Models with a large number of latent variables are often used to fully
utilize the information in big or complex data. However, they can be difficult
to estimate using standard approaches, and variational inference methods are a
popular alternative. Key to the success of these is the selection of an
approximation to the target density that is accurate, tractable and fast to
calibrate using optimization methods. Most existing choices can be inaccurate
or slow to calibrate when there are many latent variables. Instead, we propose
a family of tractable variational approximations that are more accurate and
faster to calibrate for this case. It combines a parsimonious parametric
approximation for the parameter posterior, with the exact conditional posterior
of the latent variables. We derive a simplified expression for the
re-parameterization gradient of the variational lower bound, which is the main
ingredient of efficient optimization algorithms used to implement variational
estimation. To do so only requires the ability to generate exactly or
approximately from the conditional posterior of the latent variables, rather
than to compute its density. We illustrate using two complex contemporary
econometric examples. The first is a nonlinear multivariate state space model
for U.S. macroeconomic variables. The second is a random coefficients tobit
model applied to two million sales by 20,000 individuals from a large marketing
panel. In both cases, we show that our approximating family is considerably
more accurate than mean field or structured Gaussian approximations, and faster
than Markov chain Monte Carlo. Last, we show how to implement data sub-sampling
in variational inference for our approximation, which can lead to a further
reduction in computation time.Comment: Macroeconomic example was replaced by the bigger and more challenging
time varying parameter vector autoregression model with stochastic
volatility. Microeconomic example was extended to 20,000 individuals and
variational subsampling is also implemented for this exampl