36 research outputs found
Flexible and accurate inference and learning for deep generative models
We introduce a new approach to learning in hierarchical latent-variable
generative models called the "distributed distributional code Helmholtz
machine", which emphasises flexibility and accuracy in the inferential process.
In common with the original Helmholtz machine and later variational autoencoder
algorithms (but unlike adverserial methods) our approach learns an explicit
inference or "recognition" model to approximate the posterior distribution over
the latent variables. Unlike in these earlier methods, the posterior
representation is not limited to a narrow tractable parameterised form (nor is
it represented by samples). To train the generative and recognition models we
develop an extended wake-sleep algorithm inspired by the original Helmholtz
Machine. This makes it possible to learn hierarchical latent models with both
discrete and continuous variables, where an accurate posterior representation
is essential. We demonstrate that the new algorithm outperforms current
state-of-the-art methods on synthetic, natural image patch and the MNIST data
sets
Synthetic Document Generator for Annotation-free Layout Recognition
Analyzing the layout of a document to identify headers, sections, tables,
figures etc. is critical to understanding its content. Deep learning based
approaches for detecting the layout structure of document images have been
promising. However, these methods require a large number of annotated examples
during training, which are both expensive and time consuming to obtain. We
describe here a synthetic document generator that automatically produces
realistic documents with labels for spatial positions, extents and categories
of the layout elements. The proposed generative process treats every physical
component of a document as a random variable and models their intrinsic
dependencies using a Bayesian Network graph. Our hierarchical formulation using
stochastic templates allow parameter sharing between documents for retaining
broad themes and yet the distributional characteristics produces visually
unique samples, thereby capturing complex and diverse layouts. We empirically
illustrate that a deep layout detection model trained purely on the synthetic
documents can match the performance of a model that uses real documents
A Novel Variational Lower Bound for Inverse Reinforcement Learning
Inverse reinforcement learning (IRL) seeks to learn the reward function from
expert trajectories, to understand the task for imitation or collaboration
thereby removing the need for manual reward engineering. However, IRL in the
context of large, high-dimensional problems with unknown dynamics has been
particularly challenging. In this paper, we present a new Variational Lower
Bound for IRL (VLB-IRL), which is derived under the framework of a
probabilistic graphical model with an optimality node. Our method
simultaneously learns the reward function and policy under the learned reward
function by maximizing the lower bound, which is equivalent to minimizing the
reverse Kullback-Leibler divergence between an approximated distribution of
optimality given the reward function and the true distribution of optimality
given trajectories. This leads to a new IRL method that learns a valid reward
function such that the policy under the learned reward achieves expert-level
performance on several known domains. Importantly, the method outperforms the
existing state-of-the-art IRL algorithms on these domains by demonstrating
better reward from the learned policy