113 research outputs found
CMA-ES with Restarts for Solving CEC 2013 Benchmark Problems
This paper investigates the performance of 6 versions of Covariance Matrix Adaptation Evolution Strategy (CMAES) with restarts on a set of 28 noiseless optimization problems (including 23 multi-modal ones) designed for the special session on real-parameter optimization of CEC 2013. The experimental validation of the restart strategies shows that: i). the versions of CMA-ES with weighted active covariance matrix update outperform the original versions of CMA-ES, especially on illconditioned problems; ii). the original restart strategies with increasing population size (IPOP) are usually outperformed by the bi-population restart strategies where the initial mutation stepsize is also varied; iii). the recently proposed alternative restart strategies for CMA-ES demonstrate a competitive performance and are ranked first w.r.t. the proportion of function-target pairs solved after the full run on all 10-, 30- and 50-dimensional problems
On-line Search History-assisted Restart Strategy for Covariance Matrix Adaptation Evolution Strategy
Restart strategy helps the covariance matrix adaptation evolution strategy
(CMA-ES) to increase the probability of finding the global optimum in
optimization, while a single run CMA-ES is easy to be trapped in local optima.
In this paper, the continuous non-revisiting genetic algorithm (cNrGA) is used
to help CMA-ES to achieve multiple restarts from different sub-regions of the
search space. The CMA-ES with on-line search history-assisted restart strategy
(HR-CMA-ES) is proposed. The entire on-line search history of cNrGA is stored
in a binary space partitioning (BSP) tree, which is effective for performing
local search. The frequently sampled sub-region is reflected by a deep position
in the BSP tree. When leaf nodes are located deeper than a threshold, the
corresponding sub-region is considered a region of interest (ROI). In
HR-CMA-ES, cNrGA is responsible for global exploration and suggesting ROI for
CMA-ES to perform an exploitation within or around the ROI. CMA-ES restarts
independently in each suggested ROI. The non-revisiting mechanism of cNrGA
avoids to suggest the same ROI for a second time. Experimental results on the
CEC 2013 and 2017 benchmark suites show that HR-CMA-ES performs better than
both CMA-ES and cNrGA. A positive synergy is observed by the memetic
cooperation of the two algorithms.Comment: 8 pages, 9 figure
Static and Dynamic Multimodal Optimization by Improved Covariance Matrix Self-Adaptation Evolution Strategy with Repelling Subpopulations
The covariance matrix self-adaptation evolution strategy with repelling subpopulations (RS-CMSA-ES) is one of the most successful multimodal optimization (MMO) methods currently available. However, some of its components may become inefficient in certain situations. This study introduces the second variant of this method, called RS-CMSA-ESII. It improves the adaptation schemes for the normalized taboo distances of the archived solutions and the covariance matrix of the subpopulation, the termination criteria for the subpopulations, and the way in which the infeasible solutions are treated. It also improves the time complexity of RS-CMSA-ES by updating the initialization procedure of a subpopulation and developing a more accurate metric for determining critical taboo regions. The effects of these modifications are illustrated by designing controlled numerical simulations. RS-CMSA-ESII is then compared with the most successful and recent niching methods for MMO on a widely adopted test suite. The results obtained reveal the superiority of RS-CMSA-ESII over these methods, including the winners of the competition on niching methods for MMO in previous years. Besides, this study extends RS-CMSA-ESII to dynamic MMO and compares it with a few recently proposed methods on the modified moving peak benchmark functions
The True Destination of EGO is Multi-local Optimization
Efficient global optimization is a popular algorithm for the optimization of
expensive multimodal black-box functions. One important reason for its
popularity is its theoretical foundation of global convergence. However, as the
budgets in expensive optimization are very small, the asymptotic properties
only play a minor role and the algorithm sometimes comes off badly in
experimental comparisons. Many alternative variants have therefore been
proposed over the years. In this work, we show experimentally that the
algorithm instead has its strength in a setting where multiple optima are to be
identified
Niching an estimation-of-distribution algorithm by hierarchical Gaussian mixture learning
Estimation-of-Distribution Algorithms (EDAs) have been applied with quite some success when solving real-valued optimization problems, especially in the case of Black Box Optimization (BBO). Generally, the performance of an EDA depends on the match between its driving probability distribution and the landscape of the problem being solved. Because most well-known EDAs, including CMA-ES, NES, and AMaLGaM, use a uni-modal search distribution, they have a high risk of getting trapped in local optima when a problem is multi-modal with a (moderate) number of relatively comparable modes. This risk could potentially be mitigated using niching methods that define multiple regions of interest where separate search distributions govern sub-populations. However, a key question is how to determine a suitable number of niches, especially in BBO. In this paper, we present a novel, adaptive niching approach that determines the niches through hierarchical clustering based on the correlation between the probability densities and fitness values of solutions. We test the performance of a combination of this niching approach with AMaLGaM on both new and well-known niching benchmark problems and ind that the new approach properly identifies multiple landscape modes, leading to much beter performance on multi-modal problems than with a non-niched, uni-modal EDA
COCO: Performance Assessment
We present an any-time performance assessment for benchmarking numerical
optimization algorithms in a black-box scenario, applied within the COCO
benchmarking platform. The performance assessment is based on runtimes measured
in number of objective function evaluations to reach one or several quality
indicator target values. We argue that runtime is the only available measure
with a generic, meaningful, and quantitative interpretation. We discuss the
choice of the target values, runlength-based targets, and the aggregation of
results by using simulated restarts, averages, and empirical distribution
functions
A Computationally Efficient Limited Memory CMA-ES for Large Scale Optimization
We propose a computationally efficient limited memory Covariance Matrix
Adaptation Evolution Strategy for large scale optimization, which we call the
LM-CMA-ES. The LM-CMA-ES is a stochastic, derivative-free algorithm for
numerical optimization of non-linear, non-convex optimization problems in
continuous domain. Inspired by the limited memory BFGS method of Liu and
Nocedal (1989), the LM-CMA-ES samples candidate solutions according to a
covariance matrix reproduced from direction vectors selected during the
optimization process. The decomposition of the covariance matrix into Cholesky
factors allows to reduce the time and memory complexity of the sampling to
, where is the number of decision variables. When is large
(e.g., > 1000), even relatively small values of (e.g., ) are
sufficient to efficiently solve fully non-separable problems and to reduce the
overall run-time.Comment: Genetic and Evolutionary Computation Conference (GECCO'2014) (2014
- …