225 research outputs found
Comparing Results of 31 Algorithms from the Black-Box Optimization Benchmarking BBOB-2009
pp. 1689-1696This paper presents results of the BBOB-2009 benchmark- ing of 31 search algorithms on 24 noiseless functions in a black-box optimization scenario in continuous domain. The runtime of the algorithms, measured in number of function evaluations, is investigated and a connection between a sin- gle convergence graph and the runtime distribution is uncov- ered. Performance is investigated for different dimensions up to 40-D, for different target precision values, and in dif- ferent subgroups of functions. Searching in larger dimension and multi-modal functions appears to be more difficult. The choice of the best algorithm also depends remarkably on the available budget of function evaluations
COCO: A Platform for Comparing Continuous Optimizers in a Black-Box Setting
We introduce COCO, an open source platform for Comparing Continuous
Optimizers in a black-box setting. COCO aims at automatizing the tedious and
repetitive task of benchmarking numerical optimization algorithms to the
greatest possible extent. The platform and the underlying methodology allow to
benchmark in the same framework deterministic and stochastic solvers for both
single and multiobjective optimization. We present the rationales behind the
(decade-long) development of the platform as a general proposition for
guidelines towards better benchmarking. We detail underlying fundamental
concepts of COCO such as the definition of a problem as a function instance,
the underlying idea of instances, the use of target values, and runtime defined
by the number of function calls as the central performance measure. Finally, we
give a quick overview of the basic code structure and the currently available
test suites.Comment: Optimization Methods and Software, Taylor & Francis, In press,
pp.1-3
KL-based Control of the Learning Schedule for Surrogate Black-Box Optimization
This paper investigates the control of an ML component within the Covariance
Matrix Adaptation Evolution Strategy (CMA-ES) devoted to black-box
optimization. The known CMA-ES weakness is its sample complexity, the number of
evaluations of the objective function needed to approximate the global optimum.
This weakness is commonly addressed through surrogate optimization, learning an
estimate of the objective function a.k.a. surrogate model, and replacing most
evaluations of the true objective function with the (inexpensive) evaluation of
the surrogate model. This paper presents a principled control of the learning
schedule (when to relearn the surrogate model), based on the Kullback-Leibler
divergence of the current search distribution and the training distribution of
the former surrogate model. The experimental validation of the proposed
approach shows significant performance gains on a comprehensive set of
ill-conditioned benchmark problems, compared to the best state of the art
including the quasi-Newton high-precision BFGS method
COCO: Performance Assessment
We present an any-time performance assessment for benchmarking numerical
optimization algorithms in a black-box scenario, applied within the COCO
benchmarking platform. The performance assessment is based on runtimes measured
in number of objective function evaluations to reach one or several quality
indicator target values. We argue that runtime is the only available measure
with a generic, meaningful, and quantitative interpretation. We discuss the
choice of the target values, runlength-based targets, and the aggregation of
results by using simulated restarts, averages, and empirical distribution
functions
Self-Adaptive Surrogate-Assisted Covariance Matrix Adaptation Evolution Strategy
This paper presents a novel mechanism to adapt surrogate-assisted
population-based algorithms. This mechanism is applied to ACM-ES, a recently
proposed surrogate-assisted variant of CMA-ES. The resulting algorithm,
saACM-ES, adjusts online the lifelength of the current surrogate model (the
number of CMA-ES generations before learning a new surrogate) and the surrogate
hyper-parameters. Both heuristics significantly improve the quality of the
surrogate model, yielding a significant speed-up of saACM-ES compared to the
ACM-ES and CMA-ES baselines. The empirical validation of saACM-ES on the
BBOB-2012 noiseless testbed demonstrates the efficiency and the scalability
w.r.t the problem dimension and the population size of the proposed approach,
that reaches new best results on some of the benchmark problems.Comment: Genetic and Evolutionary Computation Conference (GECCO 2012) (2012
Benchmarking Evolutionary Algorithms For Single Objective Real-valued Constrained Optimization - A Critical Review
Benchmarking plays an important role in the development of novel search
algorithms as well as for the assessment and comparison of contemporary
algorithmic ideas. This paper presents common principles that need to be taken
into account when considering benchmarking problems for constrained
optimization. Current benchmark environments for testing Evolutionary
Algorithms are reviewed in the light of these principles. Along with this line,
the reader is provided with an overview of the available problem domains in the
field of constrained benchmarking. Hence, the review supports algorithms
developers with information about the merits and demerits of the available
frameworks.Comment: This manuscript is a preprint version of an article published in
Swarm and Evolutionary Computation, Elsevier, 2018. Number of pages: 4
Challenges of ELA-guided Function Evolution using Genetic Programming
Within the optimization community, the question of how to generate new
optimization problems has been gaining traction in recent years. Within topics
such as instance space analysis (ISA), the generation of new problems can
provide new benchmarks which are not yet explored in existing research. Beyond
that, this function generation can also be exploited for solving complex
real-world optimization problems. By generating functions with similar
properties to the target problem, we can create a robust test set for algorithm
selection and configuration.
However, the generation of functions with specific target properties remains
challenging. While features exist to capture low-level landscape properties,
they might not always capture the intended high-level features. We show that a
genetic programming (GP) approach guided by these exploratory landscape
analysis (ELA) properties is not always able to find satisfying functions. Our
results suggest that careful considerations of the weighting of landscape
properties, as well as the distance measure used, might be required to evolve
functions that are sufficiently representative to the target landscape
Online Selection of CMA-ES Variants
In the field of evolutionary computation, one of the most challenging topics
is algorithm selection. Knowing which heuristics to use for which optimization
problem is key to obtaining high-quality solutions. We aim to extend this
research topic by taking a first step towards a selection method for adaptive
CMA-ES algorithms. We build upon the theoretical work done by van Rijn
\textit{et al.} [PPSN'18], in which the potential of switching between
different CMA-ES variants was quantified in the context of a modular CMA-ES
framework.
We demonstrate in this work that their proposed approach is not very
reliable, in that implementing the suggested adaptive configurations does not
yield the predicted performance gains. We propose a revised approach, which
results in a more robust fit between predicted and actual performance. The
adaptive CMA-ES approach obtains performance gains on 18 out of 24 tested
functions of the BBOB benchmark, with stable advantages of up to 23\%. An
analysis of module activation indicates which modules are most crucial for the
different phases of optimizing each of the 24 benchmark problems. The module
activation also suggests that additional gains are possible when including the
(B)IPOP modules, which we have excluded for this present work.Comment: To appear at Genetic and Evolutionary Computation Conference
(GECCO'19) Appendix will be added in due tim
- …