49,154 research outputs found
Towards Flexibility and Interpretability of Gaussian Process State-Space Model
Gaussian process state-space model (GPSSM) has attracted much attention over
the past decade. However, the model representation power of GPSSM is far from
satisfactory. Most GPSSM works rely on the standard Gaussian process (GP) with
a preliminary kernel, such as squared exponential (SE) kernel and Mat\'{e}rn
kernel, which limit the model representation power and its application in
complex scenarios. To address this issue, this paper proposes a novel class of
probabilistic state-space model named TGPSSM that enriches the GP priors in the
standard GPSSM through parametric normalizing flow, making the state-space
model more flexible and expressive. In addition, by inheriting the advantages
of sparse representation of GP models, we propose a scalable and interpretable
variational learning algorithm to learn the TGPSSM and infer the latent
dynamics simultaneously. By integrating a constrained optimization framework
and explicitly constructing a non-Gaussian state variational distribution, the
proposed learning algorithm enables the TGPSSM to significantly improve the
capabilities of state space representation and model inference. Experimental
results based on various synthetic and real datasets corroborate that the
proposed TGPSSM yields superior learning and inference performance compared to
several state-of-the-art methods. The accompanying source code is available at
https://github.com/zhidilin/TGPSSM
Markovian Gaussian Process Variational Autoencoders
Deep generative models are widely used for modelling high-dimensional time
series, such as video animations, audio and climate data. Sequential
variational autoencoders have been successfully considered for many
applications, with many variant models relying on discrete-time methods and
recurrent neural networks (RNNs). On the other hand, continuous-time methods
have recently gained attraction, especially in the context of
irregularly-sampled time series, where they can better handle the data than
discrete-time methods. One such class are Gaussian process variational
autoencoders (GPVAEs), where the VAE prior is set as a Gaussian process (GPs),
allowing inductive biases to be explicitly encoded via the kernel function and
interpretability of the latent space. However, a major limitation of GPVAEs is
that it inherits the same cubic computational cost as GPs. In this work, we
leverage the equivalent discrete state space representation of Markovian GPs to
enable a linear-time GP solver via Kalman filtering and smoothing. We show via
corrupt and missing frames tasks that our method performs favourably,
especially on the latter where it outperforms RNN-based models.Comment: Non-archival paper presented at Workshop on Continuous Time Methods
for Machine Learning. The 39th International Conference on Machine Learning,
Baltimor
- …