637 research outputs found
Federated Generalized Bayesian Learning via Distributed Stein Variational Gradient Descent
This paper introduces Distributed Stein Variational Gradient Descent (DSVGD),
a non-parametric generalized Bayesian inference framework for federated
learning. DSVGD maintains a number of non-random and interacting particles at a
central server to represent the current iterate of the model global posterior.
The particles are iteratively downloaded and updated by one of the agents with
the end goal of minimizing the global free energy. By varying the number of
particles, DSVGD enables a flexible trade-off between per-iteration
communication load and number of communication rounds. DSVGD is shown to
compare favorably to benchmark frequentist and Bayesian federated learning
strategies, also scheduling a single device per iteration, in terms of accuracy
and scalability with respect to the number of agents, while also providing
well-calibrated, and hence trustworthy, predictions
Variational Gaussian Process Diffusion Processes
Diffusion processes are a class of stochastic differential equations (SDEs)
providing a rich family of expressive models that arise naturally in dynamic
modelling tasks. Probabilistic inference and learning under generative models
with latent processes endowed with a non-linear diffusion process prior are
intractable problems. We build upon work within variational inference
approximating the posterior process as a linear diffusion process, point out
pathologies in the approach, and propose an alternative parameterization of the
Gaussian variational process using a continuous exponential family description.
This allows us to trade a slow inference algorithm with fixed-point iterations
for a fast algorithm for convex optimization akin to natural gradient descent,
which also provides a better objective for the learning of model parameters.Comment: 26 pages, 11 figure
- …