174 research outputs found
On ANOVA decompositions of kernels and Gaussian random field paths
The FANOVA (or "Sobol'-Hoeffding") decomposition of multivariate functions
has been used for high-dimensional model representation and global sensitivity
analysis. When the objective function f has no simple analytic form and is
costly to evaluate, a practical limitation is that computing FANOVA terms may
be unaffordable due to numerical integration costs. Several approximate
approaches relying on random field models have been proposed to alleviate these
costs, where f is substituted by a (kriging) predictor or by conditional
simulations. In the present work, we focus on FANOVA decompositions of Gaussian
random field sample paths, and we notably introduce an associated kernel
decomposition (into 2^{2d} terms) called KANOVA. An interpretation in terms of
tensor product projections is obtained, and it is shown that projected kernels
control both the sparsity of Gaussian random field sample paths and the
dependence structure between FANOVA effects. Applications on simulated data
show the relevance of the approach for designing new classes of covariance
kernels dedicated to high-dimensional kriging
Invariances of random fields paths, with applications in Gaussian Process Regression
We study pathwise invariances of centred random fields that can be controlled
through the covariance. A result involving composition operators is obtained in
second-order settings, and we show that various path properties including
additivity boil down to invariances of the covariance kernel. These results are
extended to a broader class of operators in the Gaussian case, via the Lo\`eve
isometry. Several covariance-driven pathwise invariances are illustrated,
including fields with symmetric paths, centred paths, harmonic paths, or sparse
paths. The proposed approach delivers a number of promising results and
perspectives in Gaussian process regression
Task-Adaptive Robot Learning from Demonstration with Gaussian Process Models under Replication
Learning from Demonstration (LfD) is a paradigm that allows robots to learn
complex manipulation tasks that can not be easily scripted, but can be
demonstrated by a human teacher. One of the challenges of LfD is to enable
robots to acquire skills that can be adapted to different scenarios. In this
paper, we propose to achieve this by exploiting the variations in the
demonstrations to retrieve an adaptive and robust policy, using Gaussian
Process (GP) models. Adaptability is enhanced by incorporating task parameters
into the model, which encode different specifications within the same task.
With our formulation, these parameters can be either real, integer, or
categorical. Furthermore, we propose a GP design that exploits the structure of
replications, i.e., repeated demonstrations with identical conditions within
data. Our method significantly reduces the computational cost of model fitting
in complex tasks, where replications are essential to obtain a robust model. We
illustrate our approach through several experiments on a handwritten letter
demonstration dataset.Comment: 8 pages, 9 figure
Recent advances in directional statistics
Mainstream statistical methodology is generally applicable to data observed
in Euclidean space. There are, however, numerous contexts of considerable
scientific interest in which the natural supports for the data under
consideration are Riemannian manifolds like the unit circle, torus, sphere and
their extensions. Typically, such data can be represented using one or more
directions, and directional statistics is the branch of statistics that deals
with their analysis. In this paper we provide a review of the many recent
developments in the field since the publication of Mardia and Jupp (1999),
still the most comprehensive text on directional statistics. Many of those
developments have been stimulated by interesting applications in fields as
diverse as astronomy, medicine, genetics, neurology, aeronautics, acoustics,
image analysis, text mining, environmetrics, and machine learning. We begin by
considering developments for the exploratory analysis of directional data
before progressing to distributional models, general approaches to inference,
hypothesis testing, regression, nonparametric curve estimation, methods for
dimension reduction, classification and clustering, and the modelling of time
series, spatial and spatio-temporal data. An overview of currently available
software for analysing directional data is also provided, and potential future
developments discussed.Comment: 61 page
Kernels for sequentially ordered data
We present a novel framework for learning with sequential data of any kind, such as multivariate time series, strings, or sequences of graphs. The main result is a ”sequentialization”
that transforms any kernel on a given domain into a kernel for sequences in that domain.
This procedure preserves properties such as positive definiteness, the associated kernel feature map is an ordered variant of sample (cross-)moments, and this sequentialized kernel
is consistent in the sense that it converges to a kernel for paths if sequences converge to
paths (by discretization). Further, classical kernels for sequences arise as special cases of
this method. We use dynamic programming and low-rank techniques for tensors to provide
efficient algorithms to compute this sequentialized kernel
High-Dimensional Non-Linear Variable Selection through Hierarchical Kernel Learning
We consider the problem of high-dimensional non-linear variable selection for supervised learning. Our approach is based on performing linear selection among exponentially many appropriately defined positive definite kernels that characterize non-linear interactions between the original variables. To select efficiently from these many kernels, we use the natural hierarchical structure of the problem to extend the multiple kernel learning framework to kernels that can be embedded in a directed acyclic graph; we show that it is then possible to perform kernel selection through a graph-adapted sparsity-inducing norm, in polynomial time in the number of selected kernels. Moreover, we study the consistency of variable selection in high-dimensional settings, showing that under certain assumptions, our regularization framework allows a number of irrelevant variables which is exponential in the number of observations. Our simulations on synthetic datasets and datasets from the UCI repository show state-of-the-art predictive performance for non-linear regression problems
A survey on high-dimensional Gaussian process modeling with application to Bayesian optimization
International audienceBayesian Optimization, the application of Bayesian function approximation to finding optima of expensive functions, has exploded in popularity in recent years. In particular, much attention has been paid to improving its efficiency on problems with many parameters to optimize. This attention has trickled down to the workhorse of high dimensional BO, high dimensional Gaussian process regression, which is also of independent interest. The great flexibility that the Gaussian process prior implies is a boon when modeling complicated, low dimensional surfaces but simply says too little when dimension grows too large. A variety of structural model assumptions have been tested to tame high dimensions, from variable selection and additive decomposition to low dimensional embeddings and beyond. Most of these approaches in turn require modifications of the acquisition function optimization strategy as well. Here we review the defining structural model assumptions and discuss the benefits and drawbacks of these approaches in practice
- …