Learning on sequential data with evolution equations
- Publication date
- Publisher
Abstract
Data which have a sequential structure are ubiquitous in many scientific domains such as physical sciences or mathematical finance. This motivates an important research effort in developing statistical and machine learning models for sequential data. Recently, the signature map, rooted in the theory of controlled differential equations, has emerged as a principled and systematic way to encode sequences into finite-dimensional vector representations. The signature kernel provides an interface with kernel methods which are recognized as a powerful class of algorithms for learning on structured data. Furthermore, the signature underpins the theory of neural controlled differential equations, neural networks which can handle sequential inputs, and more specifically the case of irregularly sampled time-series.
This thesis is at the intersection of these three research areas and addresses key modelling and computational challenges for learning on sequential data. We make use of the well-established theory of reproducing kernels and the rich mathematical properties of the signature to derive an approximate inference scheme for Gaussian processes, Bayesian kernel methods, for learning with large datasets of multi-channel sequential data. Then, we construct new basis functions and kernel functions for regression problems where the inputs are sets of sequences instead of a single sequence. Finally, we use the modelling paradigm of stochastic partial differential equations to design a neural network architecture for learning functional relationships between spatio-temporal signals.
The role of differential equations of evolutionary type is central in this thesis as they are used to model the relationship between independent and dependent signals, and provide tractable algorithms for kernel methods on sequential data