13,346 research outputs found
Automatic Differentiation Variational Inference
Probabilistic modeling is iterative. A scientist posits a simple model, fits
it to her data, refines it according to her analysis, and repeats. However,
fitting complex models to large data is a bottleneck in this process. Deriving
algorithms for new models can be both mathematically and computationally
challenging, which makes it difficult to efficiently cycle through the steps.
To this end, we develop automatic differentiation variational inference (ADVI).
Using our method, the scientist only provides a probabilistic model and a
dataset, nothing else. ADVI automatically derives an efficient variational
inference algorithm, freeing the scientist to refine and explore many models.
ADVI supports a broad class of models-no conjugacy assumptions are required. We
study ADVI across ten different models and apply it to a dataset with millions
of observations. ADVI is integrated into Stan, a probabilistic programming
system; it is available for immediate use
Moment-Based Variational Inference for Markov Jump Processes
We propose moment-based variational inference as a flexible framework for
approximate smoothing of latent Markov jump processes. The main ingredient of
our approach is to partition the set of all transitions of the latent process
into classes. This allows to express the Kullback-Leibler divergence between
the approximate and the exact posterior process in terms of a set of moment
functions that arise naturally from the chosen partition. To illustrate
possible choices of the partition, we consider special classes of jump
processes that frequently occur in applications. We then extend the results to
parameter inference and demonstrate the method on several examples.Comment: Accepted by the 36th International Conference on Machine Learning
(ICML 2019
A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning
This paper takes a step towards temporal reasoning in a dynamically changing
video, not in the pixel space that constitutes its frames, but in a latent
space that describes the non-linear dynamics of the objects in its world. We
introduce the Kalman variational auto-encoder, a framework for unsupervised
learning of sequential data that disentangles two latent representations: an
object's representation, coming from a recognition model, and a latent state
describing its dynamics. As a result, the evolution of the world can be
imagined and missing data imputed, both without the need to generate high
dimensional frames at each time step. The model is trained end-to-end on videos
of a variety of simulated physical systems, and outperforms competing methods
in generative and missing data imputation tasks.Comment: NIPS 201
- …