2 research outputs found
Sequential Gaussian Processes for Online Learning of Nonstationary Functions
Many machine learning problems can be framed in the context of estimating
functions, and often these are time-dependent functions that are estimated in
real-time as observations arrive. Gaussian processes (GPs) are an attractive
choice for modeling real-valued nonlinear functions due to their flexibility
and uncertainty quantification. However, the typical GP regression model
suffers from several drawbacks: i) Conventional GP inference scales
with respect to the number of observations; ii) updating a GP model
sequentially is not trivial; and iii) covariance kernels often enforce
stationarity constraints on the function, while GPs with non-stationary
covariance kernels are often intractable to use in practice. To overcome these
issues, we propose an online sequential Monte Carlo algorithm to fit mixtures
of GPs that capture non-stationary behavior while allowing for fast,
distributed inference. By formulating hyperparameter optimization as a
multi-armed bandit problem, we accelerate mixing for real time inference. Our
approach empirically improves performance over state-of-the-art methods for
online GP estimation in the context of prediction for simulated non-stationary
data and hospital time series data
Continual Multi-task Gaussian Processes
We address the problem of continual learning in multi-task Gaussian process
(GP) models for handling sequential input-output observations. Our approach
extends the existing prior-posterior recursion of online Bayesian inference,
i.e.\ past posterior discoveries become future prior beliefs, to the infinite
functional space setting of GP. For a reason of scalability, we introduce
variational inference together with an sparse approximation based on inducing
inputs. As a consequence, we obtain tractable continual lower-bounds where two
novel Kullback-Leibler (KL) divergences intervene in a natural way. The key
technical property of our method is the recursive reconstruction of conditional
GP priors conditioned on the variational parameters learned so far. To achieve
this goal, we introduce a novel factorization of past variational
distributions, where the predictive GP equation propagates the posterior
uncertainty forward. We then demonstrate that it is possible to derive GP
models over many types of sequential observations, either discrete or
continuous and amenable to stochastic optimization. The continual inference
approach is also applicable to scenarios where potential multi-channel or
heterogeneous observations might appear. Extensive experiments demonstrate that
the method is fully scalable, shows a reliable performance and is robust to
uncertainty error propagation over a plenty of synthetic and real-world
datasets