An empirical investigation of simplified step-size adapatation in evolution strategies with a view to theory

Randomized direct-search methods for the optimization of a function f: R^n -> R given by a black box for f-evaluations are investigated. We consider the cumulative step-size adaptation (CSA) for the variance of multivariate zero-mean normal distributions. Those are commonly used to sample new candidate solutions within metaheuristics, in particular within the CMA Evolution Strategy (CMA-ES), a state-of-the-art direct-search method. Though the CMA-ES is very successful in practical optimization, its theoretical foundations are very limited because of the complex stochastic process it induces. To forward the theory on this successful method, we propose two simplifications of the CSA used within CMA-ES for step-size control. We show by experimental and statistical evaluation that they perform sufficiently similarly to the original CSA (in the considered scenario), so that a further theoretical analysis is in fact reasonable. Furthermore, we outline in detail a probabilistic/theoretical runtime analysis for one of the two CSA-derivatives

When the plus strategy performs better than the comma strategy - and when not

Occasionally there have been long debates on whether to use elitist selection or not. In the present paper the simple (1,lambd) EA and (1+lambda) EA operating on {0,1}^n are compared by means of a rigorous runtime analysis. It turns out that only values for lambda that are logarithmic in n are interesting. An illustrative function is presented for which newly developed proof methods show that the (1,lambda) EA - where lambda is logarithmic in n - outperforms the (1+lambda) EA for any lambda. For smaller offspring populations the (1,lambda) EA is inefficient on every function with a unique optimum, whereas for larger lambda the two randomized search heuristics behave almost equivalently