1,122 research outputs found
A Computationally Efficient Limited Memory CMA-ES for Large Scale Optimization
We propose a computationally efficient limited memory Covariance Matrix
Adaptation Evolution Strategy for large scale optimization, which we call the
LM-CMA-ES. The LM-CMA-ES is a stochastic, derivative-free algorithm for
numerical optimization of non-linear, non-convex optimization problems in
continuous domain. Inspired by the limited memory BFGS method of Liu and
Nocedal (1989), the LM-CMA-ES samples candidate solutions according to a
covariance matrix reproduced from direction vectors selected during the
optimization process. The decomposition of the covariance matrix into Cholesky
factors allows to reduce the time and memory complexity of the sampling to
, where is the number of decision variables. When is large
(e.g., > 1000), even relatively small values of (e.g., ) are
sufficient to efficiently solve fully non-separable problems and to reduce the
overall run-time.Comment: Genetic and Evolutionary Computation Conference (GECCO'2014) (2014
An Asynchronous Implementation of the Limited Memory CMA-ES
We present our asynchronous implementation of the LM-CMA-ES algorithm, which
is a modern evolution strategy for solving complex large-scale continuous
optimization problems. Our implementation brings the best results when the
number of cores is relatively high and the computational complexity of the
fitness function is also high. The experiments with benchmark functions show
that it is able to overcome its origin on the Sphere function, reaches certain
thresholds faster on the Rosenbrock and Ellipsoid function, and surprisingly
performs much better than the original version on the Rastrigin function.Comment: 9 pages, 4 figures, 4 tables; this is a full version of a paper which
has been accepted as a poster to IEEE ICMLA conference 201
A hybrid swarm-based algorithm for single-objective optimization problems involving high-cost analyses
In many technical fields, single-objective optimization procedures in
continuous domains involve expensive numerical simulations. In this context, an
improvement of the Artificial Bee Colony (ABC) algorithm, called the Artificial
super-Bee enhanced Colony (AsBeC), is presented. AsBeC is designed to provide
fast convergence speed, high solution accuracy and robust performance over a
wide range of problems. It implements enhancements of the ABC structure and
hybridizations with interpolation strategies. The latter are inspired by the
quadratic trust region approach for local investigation and by an efficient
global optimizer for separable problems. Each modification and their combined
effects are studied with appropriate metrics on a numerical benchmark, which is
also used for comparing AsBeC with some effective ABC variants and other
derivative-free algorithms. In addition, the presented algorithm is validated
on two recent benchmarks adopted for competitions in international conferences.
Results show remarkable competitiveness and robustness for AsBeC.Comment: 19 pages, 4 figures, Springer Swarm Intelligenc
The True Destination of EGO is Multi-local Optimization
Efficient global optimization is a popular algorithm for the optimization of
expensive multimodal black-box functions. One important reason for its
popularity is its theoretical foundation of global convergence. However, as the
budgets in expensive optimization are very small, the asymptotic properties
only play a minor role and the algorithm sometimes comes off badly in
experimental comparisons. Many alternative variants have therefore been
proposed over the years. In this work, we show experimentally that the
algorithm instead has its strength in a setting where multiple optima are to be
identified
Importance mixing: Improving sample reuse in evolutionary policy search methods
Deep neuroevolution, that is evolutionary policy search methods based on deep
neural networks, have recently emerged as a competitor to deep reinforcement
learning algorithms due to their better parallelization capabilities. However,
these methods still suffer from a far worse sample efficiency. In this paper we
investigate whether a mechanism known as "importance mixing" can significantly
improve their sample efficiency. We provide a didactic presentation of
importance mixing and we explain how it can be extended to reuse more samples.
Then, from an empirical comparison based on a simple benchmark, we show that,
though it actually provides better sample efficiency, it is still far from the
sample efficiency of deep reinforcement learning, though it is more stable
- …