Theory and practice of population diversity in evolutionary computation

Divergence of character is a cornerstone of natural evolution. On the contrary, evolutionary optimization processes are plagued by an endemic lack of population diversity: all candidate solutions eventually crowd the very same areas in the search space. The problem is usually labeled with the oxymoron “premature convergence” and has very different consequences on the different applications, almost all deleterious. At the same time, case studies from theoretical runtime analyses irrefutably demonstrate the benefits of diversity. This tutorial will give an introduction into the area of “diversity promotion”: we will define the term “diversity” in the context of Evolutionary Computation, showing how practitioners tried, with mixed results, to promote it. Then, we will analyze the benefits brought by population diversity in specific contexts, namely global exploration and enhancing the power of crossover. To this end, we will survey recent results from rigorous runtime analysis on selected problems. The presented analyses rigorously quantify the performance of evolutionary algorithms in the light of population diversity, laying the foundation for a rigorous understanding of how search dynamics are affected by the presence or absence of diversity and the introduction of diversity mechanisms

Benchmarking a (μ+λ)(\mu+\lambda) Genetic Algorithm with Configurable Crossover Probability

We investigate a family of $(\mu+\lambda)$ Genetic Algorithms (GAs) which creates offspring either from mutation or by recombining two randomly chosen parents. By scaling the crossover probability, we can thus interpolate from a fully mutation-only algorithm towards a fully crossover-based GA. We analyze, by empirical means, how the performance depends on the interplay of population size and the crossover probability. Our comparison on 25 pseudo-Boolean optimization problems reveals an advantage of crossover-based configurations on several easy optimization tasks, whereas the picture for more complex optimization problems is rather mixed. Moreover, we observe that the ``fast'' mutation scheme with its are power-law distributed mutation strengths outperforms standard bit mutation on complex optimization tasks when it is combined with crossover, but performs worse in the absence of crossover. We then take a closer look at the surprisingly good performance of the crossover-based $(\mu+\lambda)$ GAs on the well-known LeadingOnes benchmark problem. We observe that the optimal crossover probability increases with increasing population size $\mu$. At the same time, it decreases with increasing problem dimension, indicating that the advantages of the crossover are not visible in the asymptotic view classically applied in runtime analysis. We therefore argue that a mathematical investigation for fixed dimensions might help us observe effects which are not visible when focusing exclusively on asymptotic performance bounds

How Crossover Speeds Up Building-Block Assembly in Genetic Algorithms

We re-investigate a fundamental question: how effective is crossover in Genetic Algorithms in combining building blocks of good solutions? Although this has been discussed controversially for decades, we are still lacking a rigorous and intuitive answer. We provide such answers for royal road functions and OneMax, where every bit is a building block. For the latter we show that using crossover makes every (\mu+\lambda) Genetic Algorithm at least twice as fast as the fastest evolutionary algorithm using only standard bit mutation, up to small-order terms and for moderate \mu and \lambda. Crossover is beneficial because it effectively turns fitness-neutral mutations into improvements by combining the right building blocks at a later stage. Compared to mutation-based evolutionary algorithms, this makes multi-bit mutations more useful. Introducing crossover changes the optimal mutation rate on OneMax from 1/n to (1+\sqrt{5})/2 \cdot 1/n \approx 1.618/n. This holds both for uniform crossover and k-point crossover. Experiments and statistical tests confirm that our findings apply to a broad class of building-block functions

Analyses of evolutionary algorithms

Evolutionary algorithms (EAs) are a highly successful tool commonly used in practice to solve algorithmic problems. This remarkable practical value, however, is not backed up by a deep theoretical understanding. Such an understanding would facilitate the application of EAs to further problems. Runtime analyses of EAs are one way to expand the theoretical knowledge in this field. This thesis presents runtime analyses for three prominent problems in combinatorial optimization. Additionally, it provides probability theoretical tools that will simplify future runtime analyses of EAs. The first problem considered is the Single Source Shortest Path problem. The task is to find in a weighted graph for a given source vertex shortest paths to all other vertices. Developing a new analysis method we can give tight bounds on the runtime of a previously designed and analyzed EA for this problem. The second problem is the All-Pairs Shortest Path problem. Given a weighted graph, one has to find a shortest path for every pair of vertices in the graph. For this problem we show that adding a crossover operator to a natural EA using only mutation provably decreases the runtime. This is the first time that the usefulness of a crossover operator was shown for a combinatorial problem. The third problem considered is the Sorting problem. For this problem, we design a new representation based on trees. We show that the EA nat- urally arising from this representation has a better runtime than previously analyzed EAs.Evolutionäre Algorithmen (EAs) werden in der Praxis sehr erfolgreich eingesetzt. Bisher werden die theoretischen Grundlagen von EAs jedoch nicht zufriedenstellend verstanden. Laufzeitanalysen für einfache EAs sollen dieses Verständnis erweitern. Diese Dissertation enthält Laufzeitanalysen für EAs für drei wohlbekannte kombinatorische Probleme. Zusätzlich werden wahrscheinlichkeitstheoretische Hilfsmittel zur Analyse von EAs eingeführt. Zuerst behandeln wir das Single Source Shortest Path Problem. Die Aufgabe besteht darin, in einem gewichteten Graphen einen kürzesten Weg von einem Startknoten zu jedem anderen Knoten zu finden. Durch die Entwick- lung einer neuen Analysemethode konnten wir scharfe Schranken für die Laufzeit eines bereits zuvor präsentierten und analysierten EAs angeben. Als nächstes betrachten wir das All-Pairs Shortest Path Problem. Hierbei will man für jedes Paar von Knoten in einem gewichteten Graphen einen kürzesten Weg berechnen. Für dieses Problem zeigen wir, dass das Hinzufügen eines Crossover Operators die Laufzeit gegenüber einem natürlichen EA, der nur Mutationen nutzt, verbessert. Dies ist das erste Mal, dass für ein kombinatorisches Problem bewiesen wurde, dass ein Crossover Operator die Laufzeit reduziert. Für das Sortierproblem entwickeln wir eine neue, auf Bäumen beruhende Repräsentation und zeigen, dass der natürlich daraus entstehende EA eine bessere Laufzeit hat als vorherige EAs