1,464 research outputs found
How priors of initial hyperparameters affect Gaussian process regression models
The hyperparameters in Gaussian process regression (GPR) model with a
specified kernel are often estimated from the data via the maximum marginal
likelihood. Due to the non-convexity of marginal likelihood with respect to the
hyperparameters, the optimization may not converge to the global maxima. A
common approach to tackle this issue is to use multiple starting points
randomly selected from a specific prior distribution. As a result the choice of
prior distribution may play a vital role in the predictability of this
approach. However, there exists little research in the literature to study the
impact of the prior distributions on the hyperparameter estimation and the
performance of GPR. In this paper, we provide the first empirical study on this
problem using simulated and real data experiments. We consider different types
of priors for the initial values of hyperparameters for some commonly used
kernels and investigate the influence of the priors on the predictability of
GPR models. The results reveal that, once a kernel is chosen, different priors
for the initial hyperparameters have no significant impact on the performance
of GPR prediction, despite that the estimates of the hyperparameters are very
different to the true values in some cases
Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)
We introduce a new structured kernel interpolation (SKI) framework, which
generalises and unifies inducing point methods for scalable Gaussian processes
(GPs). SKI methods produce kernel approximations for fast computations through
kernel interpolation. The SKI framework clarifies how the quality of an
inducing point approach depends on the number of inducing (aka interpolation)
points, interpolation strategy, and GP covariance kernel. SKI also provides a
mechanism to create new scalable kernel methods, through choosing different
kernel interpolation strategies. Using SKI, with local cubic kernel
interpolation, we introduce KISS-GP, which is 1) more scalable than inducing
point alternatives, 2) naturally enables Kronecker and Toeplitz algebra for
substantial additional gains in scalability, without requiring any grid data,
and 3) can be used for fast and expressive kernel learning. KISS-GP costs O(n)
time and storage for GP inference. We evaluate KISS-GP for kernel matrix
approximation, kernel learning, and natural sound modelling.Comment: 19 pages, 4 figure
Gaussian Process Kernels for Pattern Discovery and Extrapolation
Gaussian processes are rich distributions over functions, which provide a Bayesian nonparametric approach to smoothing and interpolation. We introduce simple closed form kernels that can be used with Gaussian processes to discover patterns and enable extrapolation. These kernels are derived by modeling a spectral density β the Fourier transform of a kernel β with a Gaussian mixture. The proposed kernels support a broad class of stationary covariances, but Gaussian process inference remains simple and analytic. We demonstrate the proposed kernels by discovering patterns and performing long range extrapolation on synthetic examples, as well as atmospheric CO2 trends and airline passenger data. We also show that it is possible to reconstruct several popular standard covariances within our framework.Engineering and Applied Science
- β¦