From Understanding Genetic Drift to a Smart-Restart Parameter-less Compact Genetic Algorithm

One of the key difficulties in using estimation-of-distribution algorithms is choosing the population size(s) appropriately: Too small values lead to genetic drift, which can cause enormous difficulties. In the regime with no genetic drift, however, often the runtime is roughly proportional to the population size, which renders large population sizes inefficient. Based on a recent quantitative analysis which population sizes lead to genetic drift, we propose a parameter-less version of the compact genetic algorithm that automatically finds a suitable population size without spending too much time in situations unfavorable due to genetic drift. We prove a mathematical runtime guarantee for this algorithm and conduct an extensive experimental analysis on four classic benchmark problems both without and with additive centered Gaussian posterior noise. The former shows that under a natural assumption, our algorithm has a performance very similar to the one obtainable from the best problem-specific population size. The latter confirms that missing the right population size in the original cGA can be detrimental and that previous theory-based suggestions for the population size can be far away from the right values; it also shows that our algorithm as well as a previously proposed parameter-less variant of the cGA based on parallel runs avoid such pitfalls. Comparing the two parameter-less approaches, ours profits from its ability to abort runs which are likely to be stuck in a genetic drift situation.Comment: 4 figures. Extended version of a paper appearing at GECCO 202

Fast Mutation in Crossover-based Algorithms

The heavy-tailed mutation operator proposed in Doerr, Le, Makhmara, and Nguyen (GECCO 2017), called \emph{fast mutation} to agree with the previously used language, so far was proven to be advantageous only in mutation-based algorithms. There, it can relieve the algorithm designer from finding the optimal mutation rate and nevertheless obtain a performance close to the one that the optimal mutation rate gives. In this first runtime analysis of a crossover-based algorithm using a heavy-tailed choice of the mutation rate, we show an even stronger impact. For the $(1+(\lambda,\lambda))$ genetic algorithm optimizing the OneMax benchmark function, we show that with a heavy-tailed mutation rate a linear runtime can be achieved. This is asymptotically faster than what can be obtained with any static mutation rate, and is asymptotically equivalent to the runtime of the self-adjusting version of the parameters choice of the $(1+(\lambda,\lambda))$ genetic algorithm. This result is complemented by an empirical study which shows the effectiveness of the fast mutation also on random satisfiable Max-3SAT instances.Comment: This is a version of the same paper presented at GECCO 2020 completed with the proofs which were missing because of the page limi

Coordination of Decisions in a Spatial Agent Model

For a binary choice problem, the spatial coordination of decisions in an agent community is investigated both analytically and by means of stochastic computer simulations. The individual decisions are based on different local information generated by the agents with a finite lifetime and disseminated in the system with a finite velocity. We derive critical parameters for the emergence of minorities and majorities of agents making opposite decisions and investigate their spatial organization. We find that dependent on two essential parameters describing the local impact and the spatial dissemination of information, either a definite stable minority/majority relation (single-attractor regime) or a broad range of possible values (multi-attractor regime) occurs. In the latter case, the outcome of the decision process becomes rather diverse and hard to predict, both with respect to the share of the majority and their spatial distribution. We further investigate how a dissemination of information on different time scales affects the outcome of the decision process. We find that a more ``efficient'' information exchange within a subpopulation provides a suitable way to stabilize their majority status and to reduce ``diversity'' and uncertainty in the decision process.Comment: submitted for publication in Physica A (31 pages incl. 17 multi-part figures

Genetic Algorithms

this paper. Bremermann's algorithm contained most of the ingredients of a good evolutionary algorithm. But because of limited computer experiments and a missing theory, he did not find a good combination of the ingredients. In the 70's two different evolutionary algorithms independently emerged - the genetic algorithm GA of Holland [1975] and the evolution strategies of Rechenberg [1973] and Schwefel [1981] . Holland was not so much interested in optimization, but in adaptation. He investigated the genetic algorithm with decision theory for discrete domains. Holland emphasized the importance of recombination in large populations, whereas Rechenberg and Schwefel mainly investigated mutation in very small populations for continuous parameter optimization

Towards A Theory Of Organisms And Evolving Automata Open Problems And Ways To Explore

We present 14 challenging problems of evolutionary computation, most of them derived from unfinished research work of outstanding scientists such as Charles Darwin, John von Neumann, Alan Turing, Claude Shannon, and Anatol Rapaport. The problems have one common theme: Can we develop a unifying theory or computational model of organisms (natural and artificial) which combines the properties structure, function, development, and evolution? There exist theories for each property separately and for some combinations of two. But the combination of all four properties seems necessary for understanding living organisms or evolving automata. We discuss promising approaches which aim in this research direction. We propose stochastic methods as a foundation for a unifying theory

The Science of Breeding and its Application to the Breeder Genetic Algorithm

Introduction There exists at least three kinds of theories in science ffl theories which are able to predict the outcome of experiments ffl theories which conceptually describe experimentally observed phaenomenen ffl theories of a "third kind" which do not have any predictive capabilities In classical science theories of a third kind have been rejected. Unfortunately the popular theory of genetic algorithms is of this kind. It is based on a fundamental theorem, the schema theorem. It cannot be applied to any given fitness function. The theory of the breeder genetic algorithm is of the first kind. It models artificial selection as performed by human breeders. The science of breeding is based on advanced statistical methods. In fact, quantitative genetics was the driving force behind modern statistics. The development started with Galton and Pearson, who invented the scatter diagram, regression and correlation i

Convergence of Estimation of Distribution Algorithms for Finite Samples

Estimation of Distribution Algorithms (EDA) have been proposed as an extension of genetic algorithms. Our algorithm FDA assumes that the function to be optimized is additively decomposed (ADF). The interaction graph GADF is used to create exact or approximate factorizations of the Boltzmann distribution. Using Gibbs sampling instead of probabilistic logic sampling is investigated. We also discuss the algorithm LFDA which learns a Bayesian network from data. For both algorithms estimates of the necessary sample size N to find the optimum are derived. The bounds are based on statistical learning theory and PAC learning. If the assumptions of a factorization theorem are fulfilled, the upper bound of the sample size N of FDA is of order O(n ln n) where n is the size of the problem. The computational complexity per generation is O(N ∗ n). For LFDA a bound cannot be proven because the network learned might be far from optimal. In many applications the optimal network is not necessary for converge to the global optima. For the 2D Ising model only 60 % of the edges of GADF need to be contained in the learned graph. Bounds can be obtained for two new learning methods. The first one learns factor graphs instead of Bayesian networks, the second one detects the structure of the function by computing its Walsh or Fourier coefficients. The computational complexity to compute the Walsh coefficients is O(n 2 ln n). The networks computed by FDA and LFDA are analyzed for a set of benchmark functions