285 research outputs found
Accelerated Information Gradient flow
We present a framework for Nesterov's accelerated gradient flows in
probability space. Here four examples of information metrics are considered,
including Fisher-Rao metric, Wasserstein-2 metric, Kalman-Wasserstein metric
and Stein metric. For both Fisher-Rao and Wasserstein-2 metrics, we prove
convergence properties of accelerated gradient flows. In implementations, we
propose a sampling-efficient discrete-time algorithm for Wasserstein-2,
Kalman-Wasserstein and Stein accelerated gradient flows with a restart
technique. We also formulate a kernel bandwidth selection method, which learns
the gradient of logarithm of density from Brownian-motion samples. Numerical
experiments, including Bayesian logistic regression and Bayesian neural
network, show the strength of the proposed methods compared with
state-of-the-art algorithms.Comment: 33 page
- …