31,391 research outputs found
A Computationally Efficient Limited Memory CMA-ES for Large Scale Optimization
We propose a computationally efficient limited memory Covariance Matrix
Adaptation Evolution Strategy for large scale optimization, which we call the
LM-CMA-ES. The LM-CMA-ES is a stochastic, derivative-free algorithm for
numerical optimization of non-linear, non-convex optimization problems in
continuous domain. Inspired by the limited memory BFGS method of Liu and
Nocedal (1989), the LM-CMA-ES samples candidate solutions according to a
covariance matrix reproduced from direction vectors selected during the
optimization process. The decomposition of the covariance matrix into Cholesky
factors allows to reduce the time and memory complexity of the sampling to
, where is the number of decision variables. When is large
(e.g., > 1000), even relatively small values of (e.g., ) are
sufficient to efficiently solve fully non-separable problems and to reduce the
overall run-time.Comment: Genetic and Evolutionary Computation Conference (GECCO'2014) (2014
An Asynchronous Implementation of the Limited Memory CMA-ES
We present our asynchronous implementation of the LM-CMA-ES algorithm, which
is a modern evolution strategy for solving complex large-scale continuous
optimization problems. Our implementation brings the best results when the
number of cores is relatively high and the computational complexity of the
fitness function is also high. The experiments with benchmark functions show
that it is able to overcome its origin on the Sphere function, reaches certain
thresholds faster on the Rosenbrock and Ellipsoid function, and surprisingly
performs much better than the original version on the Rastrigin function.Comment: 9 pages, 4 figures, 4 tables; this is a full version of a paper which
has been accepted as a poster to IEEE ICMLA conference 201
Cooperative Coevolution for Non-Separable Large-Scale Black-Box Optimization: Convergence Analyses and Distributed Accelerations
Given the ubiquity of non-separable optimization problems in real worlds, in
this paper we analyze and extend the large-scale version of the well-known
cooperative coevolution (CC), a divide-and-conquer optimization framework, on
non-separable functions. First, we reveal empirical reasons of why
decomposition-based methods are preferred or not in practice on some
non-separable large-scale problems, which have not been clearly pointed out in
many previous CC papers. Then, we formalize CC to a continuous game model via
simplification, but without losing its essential property. Different from
previous evolutionary game theory for CC, our new model provides a much simpler
but useful viewpoint to analyze its convergence, since only the pure Nash
equilibrium concept is needed and more general fitness landscapes can be
explicitly considered. Based on convergence analyses, we propose a hierarchical
decomposition strategy for better generalization, as for any decomposition
there is a risk of getting trapped into a suboptimal Nash equilibrium. Finally,
we use powerful distributed computing to accelerate it under the multi-level
learning framework, which combines the fine-tuning ability from decomposition
with the invariance property of CMA-ES. Experiments on a set of
high-dimensional functions validate both its search performance and scalability
(w.r.t. CPU cores) on a clustering computing platform with 400 CPU cores
- …