53 research outputs found
Additive Kernels for Gaussian Process Modeling
Gaussian Process (GP) models are often used as mathematical approximations of
computationally expensive experiments. Provided that its kernel is suitably
chosen and that enough data is available to obtain a reasonable fit of the
simulator, a GP model can beneficially be used for tasks such as prediction,
optimization, or Monte-Carlo-based quantification of uncertainty. However, the
former conditions become unrealistic when using classical GPs as the dimension
of input increases. One popular alternative is then to turn to Generalized
Additive Models (GAMs), relying on the assumption that the simulator's response
can approximately be decomposed as a sum of univariate functions. If such an
approach has been successfully applied in approximation, it is nevertheless not
completely compatible with the GP framework and its versatile applications. The
ambition of the present work is to give an insight into the use of GPs for
additive models by integrating additivity within the kernel, and proposing a
parsimonious numerical method for data-driven parameter estimation. The first
part of this article deals with the kernels naturally associated to additive
processes and the properties of the GP models based on such kernels. The second
part is dedicated to a numerical procedure based on relaxation for additive
kernel parameter estimation. Finally, the efficiency of the proposed method is
illustrated and compared to other approaches on Sobol's g-function
Invariances of random fields paths, with applications in Gaussian Process Regression
We study pathwise invariances of centred random fields that can be controlled
through the covariance. A result involving composition operators is obtained in
second-order settings, and we show that various path properties including
additivity boil down to invariances of the covariance kernel. These results are
extended to a broader class of operators in the Gaussian case, via the Lo\`eve
isometry. Several covariance-driven pathwise invariances are illustrated,
including fields with symmetric paths, centred paths, harmonic paths, or sparse
paths. The proposed approach delivers a number of promising results and
perspectives in Gaussian process regression
On ANOVA decompositions of kernels and Gaussian random field paths
The FANOVA (or "Sobol'-Hoeffding") decomposition of multivariate functions
has been used for high-dimensional model representation and global sensitivity
analysis. When the objective function f has no simple analytic form and is
costly to evaluate, a practical limitation is that computing FANOVA terms may
be unaffordable due to numerical integration costs. Several approximate
approaches relying on random field models have been proposed to alleviate these
costs, where f is substituted by a (kriging) predictor or by conditional
simulations. In the present work, we focus on FANOVA decompositions of Gaussian
random field sample paths, and we notably introduce an associated kernel
decomposition (into 2^{2d} terms) called KANOVA. An interpretation in terms of
tensor product projections is obtained, and it is shown that projected kernels
control both the sparsity of Gaussian random field sample paths and the
dependence structure between FANOVA effects. Applications on simulated data
show the relevance of the approach for designing new classes of covariance
kernels dedicated to high-dimensional kriging
Gaussian process models for periodicity detection
We consider the problem of detecting and quantifying the periodic component
of a function given noise-corrupted observations of a limited number of
input/output tuples. Our approach is based on Gaussian process regression which
provides a flexible non-parametric framework for modelling periodic data. We
introduce a novel decomposition of the covariance function as the sum of
periodic and aperiodic kernels. This decomposition allows for the creation of
sub-models which capture the periodic nature of the signal and its complement.
To quantify the periodicity of the signal, we derive a periodicity ratio which
reflects the uncertainty in the fitted sub-models. Although the method can be
applied to many kernels, we give a special emphasis to the Mat\'ern family,
from the expression of the reproducing kernel Hilbert space inner product to
the implementation of the associated periodic kernels in a Gaussian process
toolkit. The proposed method is illustrated by considering the detection of
periodically expressed genes in the arabidopsis genome.Comment: in PeerJ Computer Science, 201
An analytic comparison of regularization methods for Gaussian Processes
Gaussian Processes (GPs) are a popular approach to predict the output of a
parameterized experiment. They have many applications in the field of Computer
Experiments, in particular to perform sensitivity analysis, adaptive design of
experiments and global optimization. Nearly all of the applications of GPs
require the inversion of a covariance matrix that, in practice, is often
ill-conditioned. Regularization methodologies are then employed with
consequences on the GPs that need to be better understood.The two principal
methods to deal with ill-conditioned covariance matrices are i) pseudoinverse
and ii) adding a positive constant to the diagonal (the so-called nugget
regularization).The first part of this paper provides an algebraic comparison
of PI and nugget regularizations. Redundant points, responsible for covariance
matrix singularity, are defined. It is proven that pseudoinverse
regularization, contrarily to nugget regularization, averages the output values
and makes the variance zero at redundant points. However, pseudoinverse and
nugget regularizations become equivalent as the nugget value vanishes. A
measure for data-model discrepancy is proposed which serves for choosing a
regularization technique.In the second part of the paper, a distribution-wise
GP is introduced that interpolates Gaussian distributions instead of data
points. Distribution-wise GP can be seen as an improved regularization method
for GPs
Bayesian Quantile and Expectile Optimisation
Bayesian optimisation is widely used to optimise stochastic black box
functions. While most strategies are focused on optimising conditional
expectations, a large variety of applications require risk-averse decisions and
alternative criteria accounting for the distribution tails need to be
considered. In this paper, we propose new variational models for Bayesian
quantile and expectile regression that are well-suited for heteroscedastic
settings. Our models consist of two latent Gaussian processes accounting
respectively for the conditional quantile (or expectile) and variance that are
chained through asymmetric likelihood functions. Furthermore, we propose two
Bayesian optimisation strategies, either derived from a GP-UCB or Thompson
sampling, that are tailored to such models and that can accommodate large
batches of points. As illustrated in the experimental section, the proposed
approach clearly outperforms the state of the art
- …