333 research outputs found
Neural adaptive sequential Monte Carlo
Sequential Monte Carlo (SMC), or particle filtering, is a popular class of
methods for sampling from an intractable target distribution using a sequence
of simpler intermediate distributions. Like other importance sampling-based
methods, performance is critically dependent on the proposal distribution: a
bad proposal can lead to arbitrarily inaccurate estimates of the target
distribution. This paper presents a new method for automatically adapting the
proposal using an approximation of the Kullback-Leibler divergence between the
true posterior and the proposal distribution. The method is very flexible,
applicable to any parameterized proposal distribution and it supports online
and batch variants. We use the new framework to adapt powerful proposal
distributions with rich parameterizations based upon neural networks leading to
Neural Adaptive Sequential Monte Carlo (NASMC). Experiments indicate that NASMC
significantly improves inference in a non-linear state space model
outperforming adaptive proposal methods including the Extended Kalman and
Unscented Particle Filters. Experiments also indicate that improved inference
translates into improved parameter learning when NASMC is used as a subroutine
of Particle Marginal Metropolis Hastings. Finally we show that NASMC is able to
train a latent variable recurrent neural network (LV-RNN) achieving results
that compete with the state-of-the-art for polymorphic music modelling. NASMC
can be seen as bridging the gap between adaptive SMC methods and the recent
work in scalable, black-box variational inference
Recommended from our members
On Building Generalizable Learning Agents
It has been a long-standing goal in Artificial Intelligence (AI) to build machines that can solve tasks that humans can. Thanks to the recent rapid progress in data-driven methods, which train agents to solve tasks by learning from massive training data, there have been many successes in applying such learning approaches to handle and even solve a number of extremely challenging tasks, including image classification, language generation, robotics control, and several multi-player games. The key factor for all these data-driven successes is that the trained agents can generalize to test scenarios that are unseen during training. This generalization capability is the foundation for building any practical AI system. This thesis studies generalization, the fundamental challenge in AI, and proposes solutions to improve the generalization performances of learning agents in a variety of problems. We start by providing a formal formulation of the generalization problem in the context of reinforcement learning and proposing 4 principles within this formulation to guide the design of training techniques for improved generalization. We validate the effectiveness of our proposed principles by considering 4 different domains, from simple to complex, and developing domain-specific techniques following these principles. Particularly, we begin with the simplest domain, i.e., path-finding on graphs (Part I), and then consider visual navigation in a 3D world (Part II) and competition in complex multi-agent games (Part III), and lastly tackle some natural language processing tasks (Part IV). Empirical evidences demonstrate that the proposed principles can generally lead to much improved generalization performances in a wide range of problems
- …