18 research outputs found
Replication or exploration? Sequential design for stochastic simulation experiments
We investigate the merits of replication, and provide methods for optimal
design (including replicates), with the goal of obtaining globally accurate
emulation of noisy computer simulation experiments. We first show that
replication can be beneficial from both design and computational perspectives,
in the context of Gaussian process surrogate modeling. We then develop a
lookahead based sequential design scheme that can determine if a new run should
be at an existing input location (i.e., replicate) or at a new one (explore).
When paired with a newly developed heteroskedastic Gaussian process model, our
dynamic design scheme facilitates learning of signal and noise relationships
which can vary throughout the input space. We show that it does so efficiently,
on both computational and statistical grounds. In addition to illustrative
synthetic examples, we demonstrate performance on two challenging real-data
simulation experiments, from inventory management and epidemiology.Comment: 34 pages, 9 figure
Predicting the Output From a Stochastic Computer Model When a Deterministic Approximation is Available
The analysis of computer models can be aided by the construction of surrogate
models, or emulators, that statistically model the numerical computer model.
Increasingly, computer models are becoming stochastic, yielding different
outputs each time they are run, even if the same input values are used.
Stochastic computer models are more difficult to analyse and more difficult to
emulate - often requiring substantially more computer model runs to fit. We
present a method of using deterministic approximations of the computer model to
better construct an emulator. The method is applied to numerous toy examples,
as well as an idealistic epidemiology model, and a model from the building
performance field
Goal-oriented adaptive sampling under random field modelling of response probability distributions
In the study of natural and artificial complex systems, responses that are
not completely determined by the considered decision variables are commonly
modelled probabilistically, resulting in response distributions varying across
decision space. We consider cases where the spatial variation of these response
distributions does not only concern their mean and/or variance but also other
features including for instance shape or uni-modality versus multi-modality.
Our contributions build upon a non-parametric Bayesian approach to modelling
the thereby induced fields of probability distributions, and in particular to a
spatial extension of the logistic Gaussian model.
The considered models deliver probabilistic predictions of response
distributions at candidate points, allowing for instance to perform
(approximate) posterior simulations of probability density functions, to
jointly predict multiple moments and other functionals of target distributions,
as well as to quantify the impact of collecting new samples on the state of
knowledge of the distribution field of interest. In particular, we introduce
adaptive sampling strategies leveraging the potential of the considered random
distribution field models to guide system evaluations in a goal-oriented way,
with a view towards parsimoniously addressing calibration and related problems
from non-linear (stochastic) inversion and global optimisation
Kernels over Sets of Finite Sets using RKHS Embeddings, with Application to Bayesian (Combinatorial) Optimization
We focus on kernel methods for set-valued inputs and their application to
Bayesian set optimization, notably combinatorial optimization. We investigate
two classes of set kernels that both rely on Reproducing Kernel Hilbert Space
embeddings, namely the ``Double Sum'' (DS) kernels recently considered in
Bayesian set optimization, and a class introduced here called ``Deep
Embedding'' (DE) kernels that essentially consists in applying a radial kernel
on Hilbert space on top of the canonical distance induced by another kernel
such as a DS kernel. We establish in particular that while DS kernels typically
suffer from a lack of strict positive definiteness, vast subclasses of DE
kernels built upon DS kernels do possess this property, enabling in turn
combinatorial optimization without requiring to introduce a jitter parameter.
Proofs of theoretical results about considered kernels are complemented by a
few practicalities regarding hyperparameter fitting. We furthermore demonstrate
the applicability of our approach in prediction and optimization tasks, relying
both on toy examples and on two test cases from mechanical engineering and
hydrogeology, respectively. Experimental results highlight the applicability
and compared merits of the considered approaches while opening new perspectives
in prediction and sequential design with set inputs
A portfolio approach to massively parallel Bayesian optimization
One way to reduce the time of conducting optimization studies is to evaluate
designs in parallel rather than just one-at-a-time. For expensive-to-evaluate
black-boxes, batch versions of Bayesian optimization have been proposed. They
work by building a surrogate model of the black-box that can be used to select
the designs to evaluate efficiently via an infill criterion. Still, with higher
levels of parallelization becoming available, the strategies that work for a
few tens of parallel evaluations become limiting, in particular due to the
complexity of selecting more evaluations. It is even more crucial when the
black-box is noisy, necessitating more evaluations as well as repeating
experiments. Here we propose a scalable strategy that can keep up with massive
batching natively, focused on the exploration/exploitation trade-off and a
portfolio allocation. We compare the approach with related methods on
deterministic and noisy functions, for mono and multiobjective optimization
tasks. These experiments show similar or better performance than existing
methods, while being orders of magnitude faster
Non-stationary Gaussian Process Surrogates
We provide a survey of non-stationary surrogate models which utilize Gaussian
processes (GPs) or variations thereof, including non-stationary kernel
adaptations, partition and local GPs, and spatial warpings through deep
Gaussian processes. We also overview publicly available software
implementations and conclude with a bake-off involving an 8-dimensional
satellite drag computer experiment. Code for this example is provided in a
public git repository.Comment: 13 pages, 5 figure
The Kalai-Smorodinski solution for many-objective Bayesian optimization
An ongoing aim of research in multiobjective Bayesian optimization is to
extend its applicability to a large number of objectives. While coping with a
limited budget of evaluations, recovering the set of optimal compromise
solutions generally requires numerous observations and is less interpretable
since this set tends to grow larger with the number of objectives. We thus
propose to focus on a specific solution originating from game theory, the
Kalai-Smorodinsky solution, which possesses attractive properties. In
particular, it ensures equal marginal gains over all objectives. We further
make it insensitive to a monotonic transformation of the objectives by
considering the objectives in the copula space. A novel tailored algorithm is
proposed to search for the solution, in the form of a Bayesian optimization
algorithm: sequential sampling decisions are made based on acquisition
functions that derive from an instrumental Gaussian process prior. Our approach
is tested on four problems with respectively four, six, eight, and nine
objectives. The method is available in the Rpackage GPGame available on CRAN at
https://cran.r-project.org/package=GPGame
Active Learning of Piecewise Gaussian Process Surrogates
Active learning of Gaussian process (GP) surrogates has been useful for
optimizing experimental designs for physical/computer simulation experiments,
and for steering data acquisition schemes in machine learning. In this paper,
we develop a method for active learning of piecewise, Jump GP surrogates. Jump
GPs are continuous within, but discontinuous across, regions of a design space,
as required for applications spanning autonomous materials design,
configuration of smart factory systems, and many others. Although our active
learning heuristics are appropriated from strategies originally designed for
ordinary GPs, we demonstrate that additionally accounting for model bias, as
opposed to the usual model uncertainty, is essential in the Jump GP context.
Toward that end, we develop an estimator for bias and variance of Jump GP
models. Illustrations, and evidence of the advantage of our proposed methods,
are provided on a suite of synthetic benchmarks, and real-simulation
experiments of varying complexity.Comment: The main algorithm of this work is protected by a provisional patent
pending with application number 63/386,82