12 research outputs found
Analysing Equilibrium States for Population Diversity
Population diversity is crucial in evolutionary algorithms as it helps with
global exploration and facilitates the use of crossover. Despite many runtime
analyses showing advantages of population diversity, we have no clear picture
of how diversity evolves over time. We study how population diversity of
algorithms, measured by the sum of pairwise Hamming distances,
evolves in a fitness-neutral environment. We give an exact formula for the
drift of population diversity and show that it is driven towards an equilibrium
state. Moreover, we bound the expected time for getting close to the
equilibrium state. We find that these dynamics, including the location of the
equilibrium, are unaffected by surprisingly many algorithmic choices. All
unbiased mutation operators with the same expected number of bit flips have the
same effect on the expected diversity. Many crossover operators have no effect
at all, including all binary unbiased, respectful operators. We review
crossover operators from the literature and identify crossovers that are
neutral towards the evolution of diversity and crossovers that are not.Comment: To appear at GECCO 202
How to Escape Local Optima in Black Box Optimisation: When Non-elitism Outperforms Elitism
Escaping local optima is one of the major obstacles to function optimisation. Using the metaphor of a fitness landscape, local optima correspond to hills separated by fitness valleys that have to be overcome. We define a class of fitness valleys of tunable difficulty by considering their length, representing the Hamming path between the two optima and their depth, the drop in fitness. For this function class we present a runtime comparison between stochastic search algorithms using different search strategies. The ((Formula presented.)) EA is a simple and well-studied evolutionary algorithm that has to jump across the valley to a point of higher fitness because it does not accept worsening moves (elitism). In contrast, the Metropolis algorithm and the Strong Selection Weak Mutation (SSWM) algorithm, a famous process in population genetics, are both able to cross the fitness valley by accepting worsening moves. We show that the runtime of the ((Formula presented.)) EA depends critically on the length of the valley while the runtimes of the non-elitist algorithms depend crucially on the depth of the valley. Moreover, we show that both SSWM and Metropolis can also efficiently optimise a rugged function consisting of consecutive valleys
Rigorous Runtime Analysis of Diversity Optimization with GSEMO on OneMinMax
The evolutionary diversity optimization aims at finding a diverse set of
solutions which satisfy some constraint on their fitness. In the context of
multi-objective optimization this constraint can require solutions to be
Pareto-optimal. In this paper we study how the GSEMO algorithm with additional
diversity-enhancing heuristic optimizes a diversity of its population on a
bi-objective benchmark problem OneMinMax, for which all solutions are
Pareto-optimal.
We provide a rigorous runtime analysis of the last step of the optimization,
when the algorithm starts with a population with a second-best diversity, and
prove that it finds a population with optimal diversity in expected time
, when the problem size is odd. For reaching our goal, we analyse
the random walk of the population, which reflects the frequency of changes in
the population and their outcomes.Comment: The full version of the paper accepted to FOGA 2023 conferenc
Lazy Parameter Tuning and Control:Choosing All Parameters Randomly from a Power-Law Distribution
Most evolutionary algorithms have multiple parameters and their values
drastically affect the performance. Due to the often complicated interplay of
the parameters, setting these values right for a particular problem (parameter
tuning) is a challenging task. This task becomes even more complicated when the
optimal parameter values change significantly during the run of the algorithm
since then a dynamic parameter choice (parameter control) is necessary.
In this work, we propose a lazy but effective solution, namely choosing all
parameter values (where this makes sense) in each iteration randomly from a
suitably scaled power-law distribution. To demonstrate the effectiveness of
this approach, we perform runtime analyses of the
genetic algorithm with all three parameters chosen in this manner. We show that
this algorithm on the one hand can imitate simple hill-climbers like the
EA, giving the same asymptotic runtime on problems like OneMax,
LeadingOnes, or Minimum Spanning Tree. On the other hand, this algorithm is
also very efficient on jump functions, where the best static parameters are
very different from those necessary to optimize simple problems. We prove a
performance guarantee that is comparable, sometimes even better, than the best
performance known for static parameters. We complement our theoretical results
with a rigorous empirical study confirming what the asymptotic runtime results
suggest.Comment: Extended version of the paper accepted to GECCO 2021, including all
the proofs omitted in the conference versio
When move acceptance selection hyper-heuristics outperform Metropolis and elitist evolutionary algorithms and when not
Selection hyper-heuristics (HHs) are automated algorithm selection methodologies that choose between different heuristics during the optimisation process. Recently, selection HHs choosing between a collection of elitist randomised local search heuristics with different neighbourhood sizes have been shown to optimise standard unimodal benchmark functions from evolutionary computation in the optimal expected runtime achievable with the available low-level heuristics. In this paper, we extend our understanding of the performance of HHs to the domain of multimodal optimisation by considering a Move Acceptance HH (MAHH) from the literature that can switch between elitist and non-elitist heuristics during the run. In essence, MAHH is a non-elitist search heuristic that differs from other search heuristics in the source of non-elitism.
We first identify the range of parameters that allow MAHH to hillclimb efficiently and prove that it can optimise the standard hillclimbing benchmark function OneMax in the best expected asymptotic time achievable by unbiased mutation-based randomised search heuristics. Afterwards, we use standard multimodal benchmark functions to highlight function characteristics where MAHH outperforms elitist evolutionary algorithms and the well-known Metropolis non-elitist algorithm by quickly escaping local optima, and ones where it does not. Since MAHH is essentially a non-elitist random local search heuristic, the paper is of independent interest to researchers in the fields of artificial intelligence and randomised search heuristics
Memetic algorithms outperform evolutionary algorithms in multimodal optimisation
Memetic algorithms integrate local search into an evolutionary algorithm to combine the advantages of rapid exploitation and global optimisation. We provide a rigorous runtime analysis of memetic algorithms on the Hurdle problem, a landscape class of tunable difficulty with a âbig valley structureâ, a characteristic feature of many hard combinatorial optimisation problems. A parameter called hurdle width describes the length of fitness valleys that need to be overcome. We show that the expected runtime of plain evolutionary algorithms like the (1+1) EA increases steeply with the hurdle width, yielding superpolynomial times to find the optimum, whereas a simple memetic algorithm, (1+1) MA, only needs polynomial expected time. Surprisingly, while increasing the hurdle width makes the problem harder for evolutionary algorithms, it becomes easier for memetic algorithms.
We further give the first rigorous proof that crossover can decrease the expected runtime in memetic algorithms. A (2+1) MA using mutation, crossover and local search outperforms any other combination of these operators. Our results demonstrate the power of memetic algorithms for problems with big valley structures and the benefits of hybridising multiple search operators
Tight bounds on the expected runtime of a standard steady state genetic algorithm
Recent progress in the runtime analysis of evolutionary algorithms (EAs) has allowed the derivation of upper bounds on the expected runtime of standard steady-state genetic algorithms (GAs). These upper bounds have shown speed-ups of the GAs using crossover and mutation over the same algorithms that only use mutation operators (i.e., steady-state EAs) both for standard unimodal (i.e., ONEMAX) and multimodal (i.e., JUMP) benchmark functions. The bounds suggest that populations are beneficial to the GA as well as higher mutation rates than the default 1/n rate. However, making rigorous claims was not possible because matching lower bounds were not available. Proving lower bounds on crossover-based EAs is a notoriously difficult task as it is hard to capture the progress that a diverse population can make. We use a potential function approach to prove a tight lower bound on the expected runtime of the (2+1) GA for ONEMAX for all mutation rates c/n with c<1.422. This provides the last piece of the puzzle that completes the proof that larger population sizes improve the performance of the standard steady-state GA for ONEMAX for various mutation rates, and it proves that the optimal mutation rate for the (2+1) GA on ONEMAX is (97ââââ5)/(4n)â1.2122/n
Complexity Theory for Discrete Black-Box Optimization Heuristics
A predominant topic in the theory of evolutionary algorithms and, more
generally, theory of randomized black-box optimization techniques is running
time analysis. Running time analysis aims at understanding the performance of a
given heuristic on a given problem by bounding the number of function
evaluations that are needed by the heuristic to identify a solution of a
desired quality. As in general algorithms theory, this running time perspective
is most useful when it is complemented by a meaningful complexity theory that
studies the limits of algorithmic solutions.
In the context of discrete black-box optimization, several black-box
complexity models have been developed to analyze the best possible performance
that a black-box optimization algorithm can achieve on a given problem. The
models differ in the classes of algorithms to which these lower bounds apply.
This way, black-box complexity contributes to a better understanding of how
certain algorithmic choices (such as the amount of memory used by a heuristic,
its selective pressure, or properties of the strategies that it uses to create
new solution candidates) influences performance.
In this chapter we review the different black-box complexity models that have
been proposed in the literature, survey the bounds that have been obtained for
these models, and discuss how the interplay of running time analysis and
black-box complexity can inspire new algorithmic solutions to well-researched
problems in evolutionary computation. We also discuss in this chapter several
interesting open questions for future work.Comment: This survey article is to appear (in a slightly modified form) in the
book "Theory of Randomized Search Heuristics in Discrete Search Spaces",
which will be published by Springer in 2018. The book is edited by Benjamin
Doerr and Frank Neumann. Missing numbers of pointers to other chapters of
this book will be added as soon as possibl