548 research outputs found
BOCK : Bayesian Optimization with Cylindrical Kernels
A major challenge in Bayesian Optimization is the boundary issue (Swersky,
2017) where an algorithm spends too many evaluations near the boundary of its
search space. In this paper, we propose BOCK, Bayesian Optimization with
Cylindrical Kernels, whose basic idea is to transform the ball geometry of the
search space using a cylindrical transformation. Because of the transformed
geometry, the Gaussian Process-based surrogate model spends less budget
searching near the boundary, while concentrating its efforts relatively more
near the center of the search region, where we expect the solution to be
located. We evaluate BOCK extensively, showing that it is not only more
accurate and efficient, but it also scales successfully to problems with a
dimensionality as high as 500. We show that the better accuracy and scalability
of BOCK even allows optimizing modestly sized neural network layers, as well as
neural network hyperparameters.Comment: 10 pages, 5 figures, 5 tables, 1 algorith
Linear Embedding-based High-dimensional Batch Bayesian Optimization without Reconstruction Mappings
The optimization of high-dimensional black-box functions is a challenging
problem. When a low-dimensional linear embedding structure can be assumed,
existing Bayesian optimization (BO) methods often transform the original
problem into optimization in a low-dimensional space. They exploit the
low-dimensional structure and reduce the computational burden. However, we
reveal that this approach could be limited or inefficient in exploring the
high-dimensional space mainly due to the biased reconstruction of the
high-dimensional queries from the low-dimensional queries. In this paper, we
investigate a simple alternative approach: tackling the problem in the original
high-dimensional space using the information from the learned low-dimensional
structure. We provide a theoretical analysis of the exploration ability.
Furthermore, we show that our method is applicable to batch optimization
problems with thousands of dimensions without any computational difficulty. We
demonstrate the effectiveness of our method on high-dimensional benchmarks and
a real-world function
- …