9 research outputs found
Recurrent Neural Filters: Learning Independent Bayesian Filtering Steps for Time Series Prediction
Despite the recent popularity of deep generative state space models, few
comparisons have been made between network architectures and the inference
steps of the Bayesian filtering framework -- with most models simultaneously
approximating both state transition and update steps with a single recurrent
neural network (RNN). In this paper, we introduce the Recurrent Neural Filter
(RNF), a novel recurrent autoencoder architecture that learns distinct
representations for each Bayesian filtering step, captured by a series of
encoders and decoders. Testing this on three real-world time series datasets,
we demonstrate that the decoupled representations learnt not only improve the
accuracy of one-step-ahead forecasts while providing realistic uncertainty
estimates, but also facilitate multistep prediction through the separation of
encoder stages
Linear-time inference for Gaussian Processes on one dimension
Gaussian Processes (GPs) provide powerful probabilistic frameworks for
interpolation, forecasting, and smoothing, but have been hampered by
computational scaling issues. Here we investigate data sampled on one dimension
(e.g., a scalar or vector time series sampled at arbitrarily-spaced intervals),
for which state-space models are popular due to their linearly-scaling
computational costs. It has long been conjectured that state-space models are
general, able to approximate any one-dimensional GP. We provide the first
general proof of this conjecture, showing that any stationary GP on one
dimension with vector-valued observations governed by a Lebesgue-integrable
continuous kernel can be approximated to any desired precision using a
specifically-chosen state-space model: the Latent Exponentially Generated (LEG)
family. This new family offers several advantages compared to the general
state-space model: it is always stable (no unbounded growth), the covariance
can be computed in closed form, and its parameter space is unconstrained
(allowing straightforward estimation via gradient descent). The theorem's proof
also draws connections to Spectral Mixture Kernels, providing insight about
this popular family of kernels. We develop parallelized algorithms for
performing inference and learning in the LEG model, test the algorithm on real
and synthetic data, and demonstrate scaling to datasets with billions of
samples.Comment: Accepted to JML
State space Gaussian processes with non-Gaussian likelihood
We provide a comprehensive overview and tooling for GP modelling with non-Gaussian likelihoods using state space methods. The state space formulation allows for solving one-dimensonal GP models in O(n) time and memory complexity. While existing literature has focused on the connection between GP regression and state space methods, the computational primitives allowing for inference using general likelihoods in combination with the Laplace approximation (LA), variational Bayes (VB), and assumed density filtering (ADF) / expectation propagation (EP) schemes has been largely overlooked. We present means of combining the efficient O(n) state space methodology with existing inference methods. We also furher extend existing methods, and provide unifying code implementing all approaches.Peer reviewe