5,291 research outputs found
SOBER: Highly Parallel Bayesian Optimization and Bayesian Quadrature over Discrete and Mixed Spaces
Batch Bayesian optimisation and Bayesian quadrature have been shown to be
sample-efficient methods of performing optimisation and quadrature where
expensive-to-evaluate objective functions can be queried in parallel. However,
current methods do not scale to large batch sizes -- a frequent desideratum in
practice (e.g. drug discovery or simulation-based inference). We present a
novel algorithm, SOBER, which permits scalable and diversified batch global
optimisation and quadrature with arbitrary acquisition functions and kernels
over discrete and mixed spaces. The key to our approach is to reformulate batch
selection for global optimisation as a quadrature problem, which relaxes
acquisition function maximisation (non-convex) to kernel recombination
(convex). Bridging global optimisation and quadrature can efficiently solve
both tasks by balancing the merits of exploitative Bayesian optimisation and
explorative Bayesian quadrature. We show that SOBER outperforms 11 competitive
baselines on 12 synthetic and diverse real-world tasks.Comment: 34 pages, 12 figure
An Entropy Search Portfolio for Bayesian Optimization
Bayesian optimization is a sample-efficient method for black-box global
optimization. How- ever, the performance of a Bayesian optimization method very
much depends on its exploration strategy, i.e. the choice of acquisition
function, and it is not clear a priori which choice will result in superior
performance. While portfolio methods provide an effective, principled way of
combining a collection of acquisition functions, they are often based on
measures of past performance which can be misleading. To address this issue, we
introduce the Entropy Search Portfolio (ESP): a novel approach to portfolio
construction which is motivated by information theoretic considerations. We
show that ESP outperforms existing portfolio methods on several real and
synthetic problems, including geostatistical datasets and simulated control
tasks. We not only show that ESP is able to offer performance as good as the
best, but unknown, acquisition function, but surprisingly it often gives better
performance. Finally, over a wide range of conditions we find that ESP is
robust to the inclusion of poor acquisition functions.Comment: 10 pages, 5 figure
Domain-Agnostic Batch Bayesian Optimization with Diverse Constraints via Bayesian Quadrature
Real-world optimisation problems often feature complex combinations of (1)
diverse constraints, (2) discrete and mixed spaces, and are (3) highly
parallelisable. (4) There are also cases where the objective function cannot be
queried if unknown constraints are not satisfied, e.g. in drug discovery,
safety on animal experiments (unknown constraints) must be established before
human clinical trials (querying objective function) may proceed. However, most
existing works target each of the above three problems in isolation and do not
consider (4) unknown constraints with query rejection. For problems with
diverse constraints and/or unconventional input spaces, it is difficult to
apply these techniques as they are often mutually incompatible. We propose
cSOBER, a domain-agnostic prudent parallel active sampler for Bayesian
optimisation, based on SOBER of Adachi et al. (2023). We consider infeasibility
under unknown constraints as a type of integration error that we can estimate.
We propose a theoretically-driven approach that propagates such error as a
tolerance in the quadrature precision that automatically balances exploitation
and exploration with the expected rejection rate. Moreover, our method flexibly
accommodates diverse constraints and/or discrete and mixed spaces via adaptive
tolerance, including conventional zero-risk cases. We show that cSOBER
outperforms competitive baselines on diverse real-world blackbox-constrained
problems, including safety-constrained drug discovery, and
human-relationship-aware team optimisation over graph-structured space.Comment: 24 pages, 5 figure
- …