722 research outputs found
GPflowOpt: A Bayesian Optimization Library using TensorFlow
A novel Python framework for Bayesian optimization known as GPflowOpt is
introduced. The package is based on the popular GPflow library for Gaussian
processes, leveraging the benefits of TensorFlow including automatic
differentiation, parallelization and GPU computations for Bayesian
optimization. Design goals focus on a framework that is easy to extend with
custom acquisition functions and models. The framework is thoroughly tested and
well documented, and provides scalability. The current released version of
GPflowOpt includes some standard single-objective acquisition functions, the
state-of-the-art max-value entropy search, as well as a Bayesian
multi-objective approach. Finally, it permits easy use of custom modeling
strategies implemented in GPflow
A warped kernel improving robustness in Bayesian optimization via random embeddings
This works extends the Random Embedding Bayesian Optimization approach by
integrating a warping of the high dimensional subspace within the covariance
kernel. The proposed warping, that relies on elementary geometric
considerations, allows mitigating the drawbacks of the high extrinsic
dimensionality while avoiding the algorithm to evaluate points giving redundant
information. It also alleviates constraints on bound selection for the embedded
domain, thus improving the robustness, as illustrated with a test case with 25
variables and intrinsic dimension 6
Data and uncertainty in extreme risks - a nonlinear expectations approach
Estimation of tail quantities, such as expected shortfall or Value at Risk,
is a difficult problem. We show how the theory of nonlinear expectations, in
particular the Data-robust expectation introduced in [5], can assist in the
quantification of statistical uncertainty for these problems. However, when we
are in a heavy-tailed context (in particular when our data are described by a
Pareto distribution, as is common in much of extreme value theory), the theory
of [5] is insufficient, and requires an additional regularization step which we
introduce. By asking whether this regularization is possible, we obtain a
qualitative requirement for reliable estimation of tail quantities and risk
measures, in a Pareto setting
Differentiating the multipoint Expected Improvement for optimal batch design
This work deals with parallel optimization of expensive objective functions
which are modeled as sample realizations of Gaussian processes. The study is
formalized as a Bayesian optimization problem, or continuous multi-armed bandit
problem, where a batch of q > 0 arms is pulled in parallel at each iteration.
Several algorithms have been developed for choosing batches by trading off
exploitation and exploration. As of today, the maximum Expected Improvement
(EI) and Upper Confidence Bound (UCB) selection rules appear as the most
prominent approaches for batch selection. Here, we build upon recent work on
the multipoint Expected Improvement criterion, for which an analytic expansion
relying on Tallis' formula was recently established. The computational burden
of this selection rule being still an issue in application, we derive a
closed-form expression for the gradient of the multipoint Expected Improvement,
which aims at facilitating its maximization using gradient-based ascent
algorithms. Substantial computational savings are shown in application. In
addition, our algorithms are tested numerically and compared to
state-of-the-art UCB-based batch-sequential algorithms. Combining starting
designs relying on UCB with gradient-based EI local optimization finally
appears as a sound option for batch design in distributed Gaussian Process
optimization
- …