Runtime Analysis of Probabilistic Crowding and Restricted Tournament Selection for Bimodal Optimisation

Many real optimisation problems lead to multimodal domains and so require the identifi- cation of multiple optima. Niching methods have been developed to maintain the population diversity, to investigate many peaks in parallel and to reduce the effect of genetic drift. Using rigorous runtime analysis, we analyse for the first time two well known niching methods: probabilistic crowding and restricted tournament selection (RTS). We incorporate both methods into a (µ+1) EA on the bimodal function Twomax where the goal is to find two optima at opposite ends of the search space. In probabilistic crowding, the offspring compete with their parents and the survivor is chosen proportionally to its fitness. On Twomax probabilistic crowding fails to find any reasonable solution quality even in exponential time. In RTS the offspring compete against the closest individual amongst w (window size) individuals. We prove that RTS fails if w is too small, leading to exponential times with high probability. However, if w is chosen large enough, it finds both optima for Twomax in time O(µn log n) with high probability. Our theoretical results are accompanied by experimental studies that match the theoretical results and also shed light on parameters not covered by the theoretical results

Theory and practice of population diversity in evolutionary computation

Divergence of character is a cornerstone of natural evolution. On the contrary, evolutionary optimization processes are plagued by an endemic lack of population diversity: all candidate solutions eventually crowd the very same areas in the search space. The problem is usually labeled with the oxymoron “premature convergence” and has very different consequences on the different applications, almost all deleterious. At the same time, case studies from theoretical runtime analyses irrefutably demonstrate the benefits of diversity. This tutorial will give an introduction into the area of “diversity promotion”: we will define the term “diversity” in the context of Evolutionary Computation, showing how practitioners tried, with mixed results, to promote it. Then, we will analyze the benefits brought by population diversity in specific contexts, namely global exploration and enhancing the power of crossover. To this end, we will survey recent results from rigorous runtime analysis on selected problems. The presented analyses rigorously quantify the performance of evolutionary algorithms in the light of population diversity, laying the foundation for a rigorous understanding of how search dynamics are affected by the presence or absence of diversity and the introduction of diversity mechanisms

Empirical analysis of diversity-preserving mechanisms on example landscapes for multimodal optimisation

Many diversity-preserving mechanisms have been developed to reduce the risk of premature convergence in evolutionary algorithms and it is not clear which mechanism is best. Most multimodal optimisation problems studied empirically are restricted to real-parameter problems and are not accessible to theoretical analysis, while theoreticians analyse the simple bimodal function TwoMax. This paper looks to narrow the gap between both approaches. We perform an extensive empirical study involving 9 common diversity mechanisms on Jansen-Zarges multimodal function classes (Jansen and Zarges, PPSN 2016) that allow to control important problem features while still being amenable to theoretical analysis. This allows us to study functions with various degrees of multimodality and to explain the results in the light of previous theoretical works. We show which mechanisms are able to find and maintain a large number of distant optima, escape from local optima, and which fail to locate even a single peak

Runtime analysis of crowding mechanisms for multimodal optimisation

Many real-world optimisation problems lead to multimodal domains and require the identification of multiple optima. Crowding methods have been developed to maintain population diversity, to investigate many peaks in parallel and to reduce genetic drift. We present the first rigorous runtime analyses of probabilistic crowding and generalised crowding, embedded in a (mu+1)EA. In probabilistic crowding the offspring compete with their parent in a fitness-proportional selection. Generalised crowding decreases the fitness of the inferior solution by a scaling factor during selection. We consider the bimodal function TwoMax and introduce a novel and natural notion for functions with bounded gradients. For a broad range of such functions we prove that probabilistic crowding needs exponential time with overwhelming probability to find solutions significantly closer to any global optimum than those found by random search. Even when the fitness function is scaled exponentially, probabilistic crowding still fails badly. Only if the exponential's base is linear in the problem size, probabilistic crowding becomes efficient on TwoMax. A similar threshold behaviour holds for generalised crowding on TwoMax with respect to the scaling factor. Our theoretical results are accompanied by experiments for TwoMax showing that the threshold behaviours also apply to the best fitness found

Theoretical and Empirical Evaluation of Diversity-preserving Mechanisms in Evolutionary Algorithms: On the Rigorous Runtime Analysis of Diversity-preserving Mechanisms in Evolutionary Algorithms

Evolutionary algorithms (EAs) simulate the natural evolution of species by iteratively applying evolutionary operators such as mutation, recombination, and selection to a set of solutions for a given problem. One of the major advantages of these algorithms is that they can be easily implemented when the optimisation problem is not well understood, and the design of problem-specific algorithms cannot be performed due to lack of time, knowledge, or expertise to design problem-specific algorithms. Also, EAs can be used as a first step to get insights when the problem is just a black box to the developer/programmer. In these cases, by evaluating candidate solutions it is possible to gain knowledge on the problem at hand. EAs are well suited to dealing with multimodal problems due to their use of a population. A diverse population can explore several hills in the fitness landscape simultaneously and offer several good solutions to the user, a feature desirable for decision making, multi-objective optimisation and dynamic optimisation. However, a major difficulty when applying EAs is that the population may converge to a sub-optimal individual before the fitness landscape is explored properly. Many diversity-preserving mechanisms have been developed to reduce the risk of such premature convergence and given such a variety of mechanisms to choose from, it is often not clear which mechanism is the best choice for a particular problem. We study the (expected/average) time for such algorithms to find satisfactory solutions for multimodal and multi-objective problems and to extract guidelines for the informed design of efficient and effective EAs. The resulting runtime bounds are used to predict and to judge the performance of algorithms for arbitrary problem sizes, further used to clarify important design issues from a theoretical perspective. We combine theoretical research with empirical applications to test the theoretical recommendations for their practicality, and to engage in rapid knowledge transfer from theory to practice. With this approach, we provide a better understanding of the working principles of EAs with diversity-preserving mechanisms. We provide theoretical foundations and we explain when and why certain diversity mechanisms are effective, and when they are not. It thus contributes to the informed design of better EAs

When hillclimbers beat genetic algorithms in multimodal optimization

This paper investigates the performance of multistart next ascent hillclimbing and well-known evolutionary algorithms incorporating diversity preservation techniques on instances of the multimodal problem generator. This generator induces a class of problems in the bitstringdomain which is interesting to study from a theoretical perspective in the context of multimodal optimization, as it is a generalization of the classical OneMax and TwoMax functions for an arbitrary number of peaks. An average-case runtime analysis for multistart next ascent hill-climbing is presented for uniformly distributed equal-height instances of this class of problems. It is shown empirically that conventional niching and mating restriction techniques incorporated in an evolutionary algorithm are not sufficient to make them competitive with the hillclimbing strategy. We conjecture the reason for this behaviour is the lack of structure in the space of local optima on instances of this problem class, which makes an optimization algorithm unable to exploit information from one optimum to infer where another optimum might be. When no such structure exist, it seems that the best strategy for discovering all optima is a brute-force one. Overall, our study gives insights with respect to the adequacy of hillclimbers and evolutionary algorithms for multimodal optimization, depending on properties of the fitness landscape.info:eu-repo/semantics/publishedVersio

An exploration of evolutionary computation applied to frequency modulation audio synthesis parameter optimisation

With the ever-increasing complexity of sound synthesisers, there is a growing demand for automated parameter estimation and sound space navigation techniques. This thesis explores the potential for evolutionary computation to automatically map known sound qualities onto the parameters of frequency modulation synthesis. Within this exploration are original contributions in the domain of synthesis parameter estimation and, within the developed system, evolutionary computation, in the form of the evolutionary algorithms that drive the underlying optimisation process. Based upon the requirement for the parameter estimation system to deliver multiple search space solutions, existing evolutionary algorithmic architectures are augmented to enable niching, while maintaining the strengths of the original algorithms. Two novel evolutionary algorithms are proposed in which cluster analysis is used to identify and maintain species within the evolving populations. A conventional evolution strategy and cooperative coevolution strategy are defined, with cluster-orientated operators that enable the simultaneous optimisation of multiple search space solutions at distinct optima. A test methodology is developed that enables components of the synthesis matching problem to be identified and isolated, enabling the performance of different optimisation techniques to be compared quantitatively. A system is consequently developed that evolves sound matches using conventional frequency modulation synthesis models, and the effectiveness of different evolutionary algorithms is assessed and compared in application to both static and timevarying sound matching problems. Performance of the system is then evaluated by interview with expert listeners. The thesis is closed with a reflection on the algorithms and systems which have been developed, discussing possibilities for the future of automated synthesis parameter estimation techniques, and how they might be employed

An exploration of evolutionary computation applied to frequency modulation audio synthesis parameter optimisation

With the ever-increasing complexity of sound synthesisers, there is a growing demand for automated parameter estimation and sound space navigation techniques. This thesis explores the potential for evolutionary computation to automatically map known sound qualities onto the parameters of frequency modulation synthesis. Within this exploration are original contributions in the domain of synthesis parameter estimation and, within the developed system, evolutionary computation, in the form of the evolutionary algorithms that drive the underlying optimisation process. Based upon the requirement for the parameter estimation system to deliver multiple search space solutions, existing evolutionary algorithmic architectures are augmented to enable niching, while maintaining the strengths of the original algorithms. Two novel evolutionary algorithms are proposed in which cluster analysis is used to identify and maintain species within the evolving populations. A conventional evolution strategy and cooperative coevolution strategy are defined, with cluster-orientated operators that enable the simultaneous optimisation of multiple search space solutions at distinct optima. A test methodology is developed that enables components of the synthesis matching problem to be identified and isolated, enabling the performance of different optimisation techniques to be compared quantitatively. A system is consequently developed that evolves sound matches using conventional frequency modulation synthesis models, and the effectiveness of different evolutionary algorithms is assessed and compared in application to both static and timevarying sound matching problems. Performance of the system is then evaluated by interview with expert listeners. The thesis is closed with a reflection on the algorithms and systems which have been developed, discussing possibilities for the future of automated synthesis parameter estimation techniques, and how they might be employed.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

An exploration of evolutionary computation applied to frequency modulation audio synthesis parameter optimisation

With the ever-increasing complexity of sound synthesisers, there is a growing demand for automated parameter estimation and sound space navigation techniques. This thesis explores the potential for evolutionary computation to automatically map known sound qualities onto the parameters of frequency modulation synthesis. Within this exploration are original contributions in the domain of synthesis parameter estimation and, within the developed system, evolutionary computation, in the form of the evolutionary algorithms that drive the underlying optimisation process. Based upon the requirement for the parameter estimation system to deliver multiple search space solutions, existing evolutionary algorithmic architectures are augmented to enable niching, while maintaining the strengths of the original algorithms. Two novel evolutionary algorithms are proposed in which cluster analysis is used to identify and maintain species within the evolving populations. A conventional evolution strategy and cooperative coevolution strategy are defined, with cluster-orientated operators that enable the simultaneous optimisation of multiple search space solutions at distinct optima. A test methodology is developed that enables components of the synthesis matching problem to be identified and isolated, enabling the performance of different optimisation techniques to be compared quantitatively. A system is consequently developed that evolves sound matches using conventional frequency modulation synthesis models, and the effectiveness of different evolutionary algorithms is assessed and compared in application to both static and timevarying sound matching problems. Performance of the system is then evaluated by interview with expert listeners. The thesis is closed with a reflection on the algorithms and systems which have been developed, discussing possibilities for the future of automated synthesis parameter estimation techniques, and how they might be employed.EThOS - Electronic Theses Online ServiceGBUnited Kingdo