16 research outputs found
On power sum kernels on symmetric groups
In this note, we introduce a family of "power sum" kernels and the
corresponding Gaussian processes on symmetric groups . Such
processes are bi-invariant: the action of on itself from both
sides does not change their finite-dimensional distributions. We show that the
values of power sum kernels can be efficiently calculated, and we also propose
a method enabling approximate sampling of the corresponding Gaussian processes
with polynomial computational complexity. By doing this we provide the tools
that are required to use the introduced family of kernels and the respective
processes for statistical modeling and machine learning
Hodge-Compositional Edge Gaussian Processes
We propose principled Gaussian processes (GPs) for modeling functions defined
over the edge set of a simplicial 2-complex, a structure similar to a graph in
which edges may form triangular faces. This approach is intended for learning
flow-type data on networks where edge flows can be characterized by the
discrete divergence and curl. Drawing upon the Hodge decomposition, we first
develop classes of divergence-free and curl-free edge GPs, suitable for various
applications. We then combine them to create \emph{Hodge-compositional edge
GPs} that are expressive enough to represent any edge function. These GPs
facilitate direct and independent learning for the different Hodge components
of edge functions, enabling us to capture their relevance during hyperparameter
optimization. To highlight their practical potential, we apply them for flow
data inference in currency exchange, ocean flows and water supply networks,
comparing them to alternative models
Intrinsic Gaussian Vector Fields on Manifolds
Various applications ranging from robotics to climate science require
modeling signals on non-Euclidean domains, such as the sphere. Gaussian process
models on manifolds have recently been proposed for such tasks, in particular
when uncertainty quantification is needed. In the manifold setting,
vector-valued signals can behave very differently from scalar-valued ones, with
much of the progress so far focused on modeling the latter. The former,
however, are crucial for many applications, such as modeling wind speeds or
force fields of unknown dynamical systems. In this paper, we propose novel
Gaussian process models for vector-valued signals on manifolds that are
intrinsically defined and account for the geometry of the space in
consideration. We provide computational primitives needed to deploy the
resulting Hodge-Mat\'ern Gaussian vector fields on the two-dimensional sphere
and the hypertori. Further, we highlight two generalization directions:
discrete two-dimensional meshes and "ideal" manifolds like hyperspheres, Lie
groups, and homogeneous spaces. Finally, we show that our Gaussian vector
fields constitute considerably more refined inductive biases than the extrinsic
fields proposed before
Stationary Kernels and Gaussian Processes on Lie Groups and their Homogeneous Spaces II: non-compact symmetric spaces
Gaussian processes are arguably the most important class of spatiotemporal
models within machine learning. They encode prior information about the modeled
function and can be used for exact or approximate Bayesian learning. In many
applications, particularly in physical sciences and engineering, but also in
areas such as geostatistics and neuroscience, invariance to symmetries is one
of the most fundamental forms of prior information one can consider. The
invariance of a Gaussian process' covariance to such symmetries gives rise to
the most natural generalization of the concept of stationarity to such spaces.
In this work, we develop constructive and practical techniques for building
stationary Gaussian processes on a very large class of non-Euclidean spaces
arising in the context of symmetries. Our techniques make it possible to (i)
calculate covariance kernels and (ii) sample from prior and posterior Gaussian
processes defined on such spaces, both in a practical manner. This work is
split into two parts, each involving different technical considerations: part I
studies compact spaces, while part II studies non-compact spaces possessing
certain structure. Our contributions make the non-Euclidean Gaussian process
models we study compatible with well-understood computational techniques
available in standard Gaussian process software packages, thereby making them
accessible to practitioners
Posterior Contraction Rates for Mat\'ern Gaussian Processes on Riemannian Manifolds
Gaussian processes are used in many machine learning applications that rely
on uncertainty quantification. Recently, computational tools for working with
these models in geometric settings, such as when inputs lie on a Riemannian
manifold, have been developed. This raises the question: can these intrinsic
models be shown theoretically to lead to better performance, compared to simply
embedding all relevant quantities into and using the restriction
of an ordinary Euclidean Gaussian process? To study this, we prove optimal
contraction rates for intrinsic Mat\'ern Gaussian processes defined on compact
Riemannian manifolds. We also prove analogous rates for extrinsic processes
using trace and extension theorems between manifold and ambient Sobolev spaces:
somewhat surprisingly, the rates obtained turn out to coincide with those of
the intrinsic processes, provided that their smoothness parameters are matched
appropriately. We illustrate these rates empirically on a number of examples,
which, mirroring prior work, show that intrinsic processes can achieve better
performance in practice. Therefore, our work shows that finer-grained analyses
are needed to distinguish between different levels of data-efficiency of
geometric Gaussian processes, particularly in settings which involve small data
set sizes and non-asymptotic behavior
Implicit Manifold Gaussian Process Regression
Gaussian process regression is widely used because of its ability to provide
well-calibrated uncertainty estimates and handle small or sparse datasets.
However, it struggles with high-dimensional data. One possible way to scale
this technique to higher dimensions is to leverage the implicit low-dimensional
manifold upon which the data actually lies, as postulated by the manifold
hypothesis. Prior work ordinarily requires the manifold structure to be
explicitly provided though, i.e. given by a mesh or be known to be one of the
well-known manifolds like the sphere. In contrast, in this paper we propose a
Gaussian process regression technique capable of inferring implicit structure
directly from data (labeled and unlabeled) in a fully differentiable way. For
the resulting model, we discuss its convergence to the Mat\'ern Gaussian
process on the assumed manifold. Our technique scales up to hundreds of
thousands of data points, and may improve the predictive performance and
calibration of the standard Gaussian process regression in
high-dimensional~settings
Matern Gaussian processes on Riemannian manifolds
Gaussian processes are an effective model class for learning unknown
functions, particularly in settings where accurately representing predictive
uncertainty is of key importance. Motivated by applications in the physical
sciences, the widely-used Mat\'{e}rn class of Gaussian processes has recently
been generalized to model functions whose domains are Riemannian manifolds, by
re-expressing said processes as solutions of stochastic partial differential
equations. In this work, we propose techniques for computing the kernels of
these processes via spectral theory of the Laplace--Beltrami operator in a
fully constructive manner, thereby allowing them to be trained via standard
scalable techniques such as inducing point methods. We also extend the
generalization from the Mat\'{e}rn to the widely-used squared exponential
Gaussian process. By allowing Riemannian Mat\'{e}rn Gaussian processes to be
trained using well-understood techniques, our work enables their use in
mini-batch, online, and non-conjugate settings, and makes them more accessible
to machine learning practitioners
Isotropic Gaussian Processes on Finite Spaces of Graphs
We propose a principled way to define Gaussian process priors on various sets
of unweighted graphs: directed or undirected, with or without loops. We endow
each of these sets with a geometric structure, inducing the notions of
closeness and symmetries, by turning them into a vertex set of an appropriate
metagraph. Building on this, we describe the class of priors that respect this
structure and are analogous to the Euclidean isotropic processes, like squared
exponential or Mat\'ern. We propose an efficient computational technique for
the ostensibly intractable problem of evaluating these priors' kernels, making
such Gaussian processes usable within the usual toolboxes and downstream
applications. We go further to consider sets of equivalence classes of
unweighted graphs and define the appropriate versions of priors thereon. We
prove a hardness result, showing that in this case, exact kernel computation
cannot be performed efficiently. However, we propose a simple Monte Carlo
approximation for handling moderately sized cases. Inspired by applications in
chemistry, we illustrate the proposed techniques on a real molecular property
prediction task in the small data regime