From Understanding Genetic Drift to a Smart-Restart Parameter-less Compact Genetic Algorithm

One of the key difficulties in using estimation-of-distribution algorithms is choosing the population size(s) appropriately: Too small values lead to genetic drift, which can cause enormous difficulties. In the regime with no genetic drift, however, often the runtime is roughly proportional to the population size, which renders large population sizes inefficient. Based on a recent quantitative analysis which population sizes lead to genetic drift, we propose a parameter-less version of the compact genetic algorithm that automatically finds a suitable population size without spending too much time in situations unfavorable due to genetic drift. We prove a mathematical runtime guarantee for this algorithm and conduct an extensive experimental analysis on four classic benchmark problems both without and with additive centered Gaussian posterior noise. The former shows that under a natural assumption, our algorithm has a performance very similar to the one obtainable from the best problem-specific population size. The latter confirms that missing the right population size in the original cGA can be detrimental and that previous theory-based suggestions for the population size can be far away from the right values; it also shows that our algorithm as well as a previously proposed parameter-less variant of the cGA based on parallel runs avoid such pitfalls. Comparing the two parameter-less approaches, ours profits from its ability to abort runs which are likely to be stuck in a genetic drift situation.Comment: 4 figures. Extended version of a paper appearing at GECCO 202

From Understanding Genetic Drift to a Smart-Restart Mechanism for Estimation-of-Distribution Algorithms

Estimation-of-distribution algorithms (EDAs) are optimization algorithms that learn a distribution on the search space from which good solutions can be sampled easily. A key parameter of most EDAs is the sample size (population size). If the population size is too small, the update of the probabilistic model builds on few samples, leading to the undesired effect of genetic drift. Too large population sizes avoid genetic drift, but slow down the process. Building on a recent quantitative analysis of how the population size leads to genetic drift, we design a smart-restart mechanism for EDAs. By stopping runs when the risk for genetic drift is high, it automatically runs the EDA in good parameter regimes. Via a mathematical runtime analysis, we prove a general performance guarantee for this smart-restart scheme. This in particular shows that in many situations where the optimal (problem-specific) parameter values are known, the restart scheme automatically finds these, leading to the asymptotically optimal performance. We also conduct an extensive experimental analysis. On four classic benchmark problems, we clearly observe the critical influence of the population size on the performance, and we find that the smart-restart scheme leads to a performance close to the one obtainable with optimal parameter values. Our results also show that previous theory-based suggestions for the optimal population size can be far from the optimal ones, leading to a performance clearly inferior to the one obtained via the smart-restart scheme. We also conduct experiments with PBIL (cross-entropy algorithm) on two combinatorial optimization problems from the literature, the max-cut problem and the bipartition problem. Again, we observe that the smart-restart mechanism finds much better values for the population size than those suggested in the literature, leading to a much better performance.Comment: Accepted for publication in "Journal of Machine Learning Research". Extended version of our GECCO 2020 paper. This article supersedes arXiv:2004.0714

State-of-the-art in aerodynamic shape optimisation methods

Aerodynamic optimisation has become an indispensable component for any aerodynamic design over the past 60 years, with applications to aircraft, cars, trains, bridges, wind turbines, internal pipe flows, and cavities, among others, and is thus relevant in many facets of technology. With advancements in computational power, automated design optimisation procedures have become more competent, however, there is an ambiguity and bias throughout the literature with regards to relative performance of optimisation architectures and employed algorithms. This paper provides a well-balanced critical review of the dominant optimisation approaches that have been integrated with aerodynamic theory for the purpose of shape optimisation. A total of 229 papers, published in more than 120 journals and conference proceedings, have been classified into 6 different optimisation algorithm approaches. The material cited includes some of the most well-established authors and publications in the field of aerodynamic optimisation. This paper aims to eliminate bias toward certain algorithms by analysing the limitations, drawbacks, and the benefits of the most utilised optimisation approaches. This review provides comprehensive but straightforward insight for non-specialists and reference detailing the current state for specialist practitioners

Genetic Improvement of Software for Energy E ciency in Noisy and Fragmented Eco-Systems

Software has made its way to every aspect of our daily life. Users of smart devices expect almost continuous availability and uninterrupted service. However, such devices operate on restricted energy resources. As energy eficiency of software is relatively a new concern for software practitioners, there is a lack of knowledge and tools to support the development of energy eficient software. Optimising the energy consumption of software requires measuring or estimating its energy use and then optimising it. Generalised models of energy behaviour suffer from heterogeneous and fragmented eco-systems (i.e. diverse hardware and operating systems). The nature of such optimisation environments favours in-vivo optimisation which provides the ground-truth for energy behaviour of an application on a given platform. One key challenge in in-vivo energy optimisation is noisy energy readings. This is because complete isolation of the effects of software optimisation is simply infeasible, owing to random and systematic noise from the platform. In this dissertation we explore in-vivo optimisation using Genetic Improvement of Software (GI) for energy eficiency in noisy and fragmented eco-systems. First, we document expected and unexpected technical challenges and their solutions when conducting energy optimisation experiments. This can be used as guidelines for software practitioners when conducting energy related experiments. Second, we demonstrate the technical feasibility of in-vivo energy optimisation using GI on smart devices. We implement a new approach for mitigating noisy readings based on simple code rewrite. Third, we propose a new conceptual framework to determine the minimum number of samples required to show significant differences between software variants competing in tournaments. We demonstrate that the number of samples can vary drastically between different platforms as well as from one point of time to another within a single platform. It is crucial to take into consideration these observations when optimising in the wild or across several devices in a control environment. Finally, we implement a new validation approach for energy optimisation experiments. Through experiments, we demonstrate that the current validation approaches can mislead software practitioners to draw wrong conclusions. Our approach outperforms the current validation techniques in terms of specificity and sensitivity in distinguishing differences between validation solutions.Thesis (Ph.D.) -- University of Adelaide, School of Computer Science, 202

Flow-Based Optimization of Products or Devices

Flow-based optimization of products and devices is an immature field compared to the corresponding topology optimization based on solid mechanics. However, it is an essential part of component development with both internal and/or external flow. The aim of this book is two-fold: (i) to provide state-of-the-art examples of flow-based optimization and (ii) to present a review of topology optimization for fluid-based problems

Theoretical and Empirical Evaluation of Diversity-preserving Mechanisms in Evolutionary Algorithms: On the Rigorous Runtime Analysis of Diversity-preserving Mechanisms in Evolutionary Algorithms

Evolutionary algorithms (EAs) simulate the natural evolution of species by iteratively applying evolutionary operators such as mutation, recombination, and selection to a set of solutions for a given problem. One of the major advantages of these algorithms is that they can be easily implemented when the optimisation problem is not well understood, and the design of problem-specific algorithms cannot be performed due to lack of time, knowledge, or expertise to design problem-specific algorithms. Also, EAs can be used as a first step to get insights when the problem is just a black box to the developer/programmer. In these cases, by evaluating candidate solutions it is possible to gain knowledge on the problem at hand. EAs are well suited to dealing with multimodal problems due to their use of a population. A diverse population can explore several hills in the fitness landscape simultaneously and offer several good solutions to the user, a feature desirable for decision making, multi-objective optimisation and dynamic optimisation. However, a major difficulty when applying EAs is that the population may converge to a sub-optimal individual before the fitness landscape is explored properly. Many diversity-preserving mechanisms have been developed to reduce the risk of such premature convergence and given such a variety of mechanisms to choose from, it is often not clear which mechanism is the best choice for a particular problem. We study the (expected/average) time for such algorithms to find satisfactory solutions for multimodal and multi-objective problems and to extract guidelines for the informed design of efficient and effective EAs. The resulting runtime bounds are used to predict and to judge the performance of algorithms for arbitrary problem sizes, further used to clarify important design issues from a theoretical perspective. We combine theoretical research with empirical applications to test the theoretical recommendations for their practicality, and to engage in rapid knowledge transfer from theory to practice. With this approach, we provide a better understanding of the working principles of EAs with diversity-preserving mechanisms. We provide theoretical foundations and we explain when and why certain diversity mechanisms are effective, and when they are not. It thus contributes to the informed design of better EAs

An overview of the main machine learning models - from theory to algorithms

Dissertation presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced AnalyticsIn the context of solving highly complex problems, Artificial Intelligence shows an exponential growth over the past years allowing the Machine Learning to augment and sometimes to outperform the human learning. From driverless cars to automatic recommendation on Netflix, we are surrounded by AI, even if we do not notice it. Furthermore, companies have recently adopted new frameworks in their routines which are mainly composed by algorithms able to solve complex problems in a short period of time. The growth of AI technologies has been absolutely stunning and yes, it is only possible because a sub-field of AI called Machine Learning is growing even faster. In a small scale, Machine Learning may be seen as a simple system able to find patterns on data and learn from it. However, it is precisely that learning process that in a large scale will allow machines to mimic the human behavior and perform tasks that would eventually require human intelligence. Just for us to have an idea, according to Forbes the global Machine Learning market was evaluated in $1.7B in 2017 and it is expected to reach almost$21B in 2024. Naturally, Machine Learning has become an attractive and profitable scientific area that demands continuous learning since there is always something new being discovered. During the last decades, a huge number of algorithms have been proposed by the research community, which sometimes may cause some confusion of how and when to use each one of them. That is exactly what is pretended in this thesis, over the next chapters we are going to review the main Machine Learning models and their respective advantages/disadvantages

Evolutionary computation applied to combinatorial optimisation problems

This thesis addresses the issues associated with conventional genetic algorithms (GA) when applied to hard optimisation problems. In particular it examines the problem of selecting and implementing appropriate genetic operators in order to meet the validity constraints for constrained optimisation problems. The problem selected is the travelling salesman problem (TSP), a well known NP-hard problem. Following a review of conventional genetic algorithms, this thesis advocates the use of a repair technique for genetic algorithms: GeneRepair. We evaluate the effectiveness of this operator against a wide range of benchmark problems and compare these results with conventional genetic algorithm approaches. A comparison between GeneRepair and the conventional GA approaches is made in two forms: firstly a handcrafted approach compares GAs without repair against those using GeneRepair. A second automated approach is then presented. This meta-genetic algorithm examines different configurations of operators and parameters. Through the use of a cost/benefit (Quality-Time Tradeoff) function, the user can balance the computational effort against the quality of the solution and thus allow the user to specify exactly what the cost benefit point should be for the search. Results have identified the optimal configuration settings for solving selected TSP problems. These results show that GeneRepair when used consistently generates very good TSP solutions for 50, 70 and 100 city problems. GeneRepair assists in finding TSP solutions in an extremely efficient manner, in both time and number of evaluations required