37 research outputs found
A Noninformative Prior on a Space of Distribution Functions
In a given problem, the Bayesian statistical paradigm requires the
specification of a prior distribution that quantifies relevant information
about the unknowns of main interest external to the data. In cases where little
such information is available, the problem under study may possess an
invariance under a transformation group that encodes a lack of information,
leading to a unique prior---this idea was explored at length by E.T. Jaynes.
Previous successful examples have included location-scale invariance under
linear transformation, multiplicative invariance of the rate at which events in
a counting process are observed, and the derivation of the Haldane prior for a
Bernoulli success probability. In this paper we show that this method can be
extended, by generalizing Jaynes, in two ways: (1) to yield families of
approximately invariant priors, and (2) to the infinite-dimensional setting,
yielding families of priors on spaces of distribution functions. Our results
can be used to describe conditions under which a particular Dirichlet Process
posterior arises from an optimal Bayesian analysis, in the sense that
invariances in the prior and likelihood lead to one and only one posterior
distribution
Stationary Kernels and Gaussian Processes on Lie Groups and their Homogeneous Spaces II: non-compact symmetric spaces
Gaussian processes are arguably the most important class of spatiotemporal
models within machine learning. They encode prior information about the modeled
function and can be used for exact or approximate Bayesian learning. In many
applications, particularly in physical sciences and engineering, but also in
areas such as geostatistics and neuroscience, invariance to symmetries is one
of the most fundamental forms of prior information one can consider. The
invariance of a Gaussian process' covariance to such symmetries gives rise to
the most natural generalization of the concept of stationarity to such spaces.
In this work, we develop constructive and practical techniques for building
stationary Gaussian processes on a very large class of non-Euclidean spaces
arising in the context of symmetries. Our techniques make it possible to (i)
calculate covariance kernels and (ii) sample from prior and posterior Gaussian
processes defined on such spaces, both in a practical manner. This work is
split into two parts, each involving different technical considerations: part I
studies compact spaces, while part II studies non-compact spaces possessing
certain structure. Our contributions make the non-Euclidean Gaussian process
models we study compatible with well-understood computational techniques
available in standard Gaussian process software packages, thereby making them
accessible to practitioners
Posterior Contraction Rates for Mat\'ern Gaussian Processes on Riemannian Manifolds
Gaussian processes are used in many machine learning applications that rely
on uncertainty quantification. Recently, computational tools for working with
these models in geometric settings, such as when inputs lie on a Riemannian
manifold, have been developed. This raises the question: can these intrinsic
models be shown theoretically to lead to better performance, compared to simply
embedding all relevant quantities into and using the restriction
of an ordinary Euclidean Gaussian process? To study this, we prove optimal
contraction rates for intrinsic Mat\'ern Gaussian processes defined on compact
Riemannian manifolds. We also prove analogous rates for extrinsic processes
using trace and extension theorems between manifold and ambient Sobolev spaces:
somewhat surprisingly, the rates obtained turn out to coincide with those of
the intrinsic processes, provided that their smoothness parameters are matched
appropriately. We illustrate these rates empirically on a number of examples,
which, mirroring prior work, show that intrinsic processes can achieve better
performance in practice. Therefore, our work shows that finer-grained analyses
are needed to distinguish between different levels of data-efficiency of
geometric Gaussian processes, particularly in settings which involve small data
set sizes and non-asymptotic behavior
Matern Gaussian processes on Riemannian manifolds
Gaussian processes are an effective model class for learning unknown
functions, particularly in settings where accurately representing predictive
uncertainty is of key importance. Motivated by applications in the physical
sciences, the widely-used Mat\'{e}rn class of Gaussian processes has recently
been generalized to model functions whose domains are Riemannian manifolds, by
re-expressing said processes as solutions of stochastic partial differential
equations. In this work, we propose techniques for computing the kernels of
these processes via spectral theory of the Laplace--Beltrami operator in a
fully constructive manner, thereby allowing them to be trained via standard
scalable techniques such as inducing point methods. We also extend the
generalization from the Mat\'{e}rn to the widely-used squared exponential
Gaussian process. By allowing Riemannian Mat\'{e}rn Gaussian processes to be
trained using well-understood techniques, our work enables their use in
mini-batch, online, and non-conjugate settings, and makes them more accessible
to machine learning practitioners