370 research outputs found
Bayesian Policy Gradients via Alpha Divergence Dropout Inference
Policy gradient methods have had great success in solving continuous control
tasks, yet the stochastic nature of such problems makes deterministic value
estimation difficult. We propose an approach which instead estimates a
distribution by fitting the value function with a Bayesian Neural Network. We
optimize an -divergence objective with Bayesian dropout approximation
to learn and estimate this distribution. We show that using the Monte Carlo
posterior mean of the Bayesian value function distribution, rather than a
deterministic network, improves stability and performance of policy gradient
methods in continuous control MuJoCo simulations.Comment: Accepted to Bayesian Deep Learning Workshop at NIPS 201
Variational implicit processes
We introduce the implicit processes (IPs), a stochastic process that places
implicitly defined multivariate distributions over any finite collections of
random variables. IPs are therefore highly flexible implicit priors over
functions, with examples including data simulators, Bayesian neural networks
and non-linear transformations of stochastic processes. A novel and efficient
approximate inference algorithm for IPs, namely the variational implicit
processes (VIPs), is derived using generalised wake-sleep updates. This method
returns simple update equations and allows scalable hyper-parameter learning
with stochastic optimization. Experiments show that VIPs return better
uncertainty estimates and lower errors over existing inference methods for
challenging models such as Bayesian neural networks, and Gaussian processes
- …