16,607 research outputs found
Black Box Variational Inference
Variational inference has become a widely used method to approximate
posteriors in complex latent variables models. However, deriving a variational
inference algorithm generally requires significant model-specific analysis, and
these efforts can hinder and deter us from quickly developing and exploring a
variety of models for a problem at hand. In this paper, we present a "black
box" variational inference algorithm, one that can be quickly applied to many
models with little additional derivation. Our method is based on a stochastic
optimization of the variational objective where the noisy gradient is computed
from Monte Carlo samples from the variational distribution. We develop a number
of methods to reduce the variance of the gradient, always maintaining the
criterion that we want to avoid difficult model-based derivations. We evaluate
our method against the corresponding black box sampling based methods. We find
that our method reaches better predictive likelihoods much faster than sampling
methods. Finally, we demonstrate that Black Box Variational Inference lets us
easily explore a wide space of models by quickly constructing and evaluating
several models of longitudinal healthcare data
Query-based Deep Improvisation
In this paper we explore techniques for generating new music using a
Variational Autoencoder (VAE) neural network that was trained on a corpus of
specific style. Instead of randomly sampling the latent states of the network
to produce free improvisation, we generate new music by querying the network
with musical input in a style different from the training corpus. This allows
us to produce new musical output with longer-term structure that blends aspects
of the query to the style of the network. In order to control the level of this
blending we add a noisy channel between the VAE encoder and decoder using
bit-allocation algorithm from communication rate-distortion theory. Our
experiments provide new insight into relations between the representational and
structural information of latent states and the query signal, suggesting their
possible use for composition purposes
- …