A survey of techniques for characterising fitness landscapes and some possible ways forward

Real-world optimisation problems are often very complex. Metaheuristics have been successful in solving many of these problems, but the difficulty in choosing the best approach can be a huge challenge for practitioners. One approach to this dilemma is to use fitness landscape analysis to better understand problems before deciding on approaches to solving the problems. However, despite extensive research on fitness landscape analysis and a large number of developed techniques, very few techniques are used in practice. This could be because fitness landscape analysis in itself can be complex. In an attempt to make fitness landscape analysis techniques accessible, this paper provides an overview of techniques from the 1980s to the present. Attributes that are important for practical implementation are highlighted and ways of adapting techniques to be more feasible or appropriate are suggested. The survey reveals the wide range of factors that can influence problem difficulty, emphasising the need for a shift in focus away from predicting problem hardness towards measuring characteristics. It is hoped that this survey will invoke renewed interest in the field of understanding complex optimisation problems and ultimately lead to better decision making on the use of appropriate metaheuristics.http://www.elsevier.com/locate/inshb201

Nature-inspired optimisation: Improvements to the Particle Swarm Optimisation Algorithm and the Bees Algorithm

This research focuses on nature-inspired optimisation algorithms, in particular, the Particle Swarm Optimisation (PSO) Algorithm and the Bees Algorithm. The PSO Algorithm is a population-based stochastic optimisation technique first invented in 1995. It was inspired by the social behaviour of birds flocking or a school of fish. The Bees Algorithm is a population-based search algorithm initially proposed in 2005. It mimics the food foraging behaviour of swarms of honey bees. The thesis presents three algorithms. The first algorithm called the PSO-Bees Algorithm is a cross between the PSO Algorithm and the Bees Algorithm. The PSO-Bees Algorithm enhanced the PSO Algorithm with techniques derived from the Bees Algorithm. The second algorithm called the improved Bees Algorithm is a version of the Bees Algorithm that incorporates techniques derived from the PSO Algorithm. The third algorithm called the SNTO-Bees Algorithm enhanced the Bees Algorithm using techniques derived from the Sequential Number-Theoretic Optimisation (SNTO) Algorithm. To demonstrate the capability of the proposed algorithms, they were applied to different optimisation problems. The PSO-Bees Algorithm is used to train neural networks for two problems, Control Chart Pattern Recognition and Wood Defect Classification. The results obtained and those from tests on well known benchmark functions provide an indication of the performance of the algorithm relative to that of other swarm-based stochastic optimisation algorithms. The improved Bees Algorithm was applied to mechanical design optimisation problems (design of welded beams and coil springs) and the mathematical benchmark problems used previously to test the PSO-Bees Algorithm. The algorithm incorporates cooperation and communication between different neighbourhoods. The results obtained show that the proposed cooperation and communication strategies adopted enhanced the performance and convergence of the algorithm. The SNTO-Bees Algorithm was applied to a set of mechanical design optimisation problems (design of welded beams, coil springs and pressure vessel) and mathematical benchmark functions used previously to test the PSO-Bees Algorithm and the improved Bees Algorithm. In addition, the algorithm was tested with a number of deceptive multi modal benchmark functions. The results obtained help to validate the SNTO-Bees Algorithm as an effective global optimiser capable of handling problems that are deceptive in nature with high dimensions

Characterising the searchability of continuous optimisation problems for PSO

The focus of research in swarm intelligence has been largely on the algorithmic side with relatively little attention being paid to the study of problems and the behaviour of algorithms in relation to problems. When a new algorithm or variation on an existing algorithm is proposed in the literature, there is seldom any discussion or analysis of algorithm weaknesses and on what kinds of problems the algorithm is expected to fail. Fitness landscape analysis is an approach that can be used to analyse optimisation problems. By characterising problems in terms of fitness landscape features, the link between problem types and algorithm performance can be studied. This article investigates a number of measures for analysing the ability of a search process to improve fitness on a particular problem (called evolvability in literature but referred to as searchability in this study to broaden the scope to non-evolutionary-based search techniques). A number of existing fitness landscape analysis techniques originally proposed for discrete problems are adapted towork in continuous search spaces. For a range of benchmark problems, the proposed searchability measures are viewed alongside performance measures for a traditional global best particle swarm optimisation (PSO) algorithm. Empirical results show that no single measure can be used as a predictor of PSO performance, but that multiple measures of different fitness landscape features can be used together to predict PSO failure.http://link.springer.com/journal/117212015-12-31hb201

Hyperparameter optimisation in differential evolution using Summed Local Difference Strings, a rugged but easily calculated landscape for combinatorial search problems

AbstractWe analyse the effectiveness of differential evolution hyperparameters in large-scale search problems, i.e. those with very many variables or vector elements, using a novel objective function that is easily calculated from the vector/string itself. The objective function is simply the sum of the differences between adjacent elements. For both binary and real-valued elements whose smallest and largest values are min and max in a vector of length N, the value of the objective function ranges between 0 and(N-1) × (max-min)and can thus easily be normalised if desired. This provides for a conveniently rugged landscape. Using this we assess how effectively search varies with both the values of fixed hyperparameters for Differential Evolution and the string length. String length, population size and generations for computational iterations have been studied. Finally, a neural network is trained by systematically varying three hyper-parameters, viz population (NP), mutation factor (F) and crossover rate (CR), and two output target variables are collected (a) median and (b) maximum cost function values from 10-trial experiments. This neural system is then tested on an extended range of data points generated by varying the three parameters on a finer scale to predict bothmedianandmaximumfunction costs. The results obtained from the machine learning model have been validated with actual runs using Pearson’s coefficient to justify the reliability to motivate the use of machine learning techniques over grid search for hyper-parameter search for numerical optimisation algorithms. The performance has also been compared with SMAC3 and OPTUNA in addition to grid search and random search.</jats:p

Evolutionary algorithms and other metaheuristics in water resources: Current status, research challenges and future directions

Copyright © 2014 Elsevier. NOTICE: this is the author’s version of a work that was accepted for publication in Environmental Modelling and Software. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Environmental Modelling and Software Vol. 62 (2014), DOI: 10.1016/j.envsoft.2014.09.013The development and application of evolutionary algorithms (EAs) and other metaheuristics for the optimisation of water resources systems has been an active research field for over two decades. Research to date has emphasized algorithmic improvements and individual applications in specific areas (e.g. model calibration, water distribution systems, groundwater management, river-basin planning and management, etc.). However, there has been limited synthesis between shared problem traits, common EA challenges, and needed advances across major applications. This paper clarifies the current status and future research directions for better solving key water resources problems using EAs. Advances in understanding fitness landscape properties and their effects on algorithm performance are critical. Future EA-based applications to real-world problems require a fundamental shift of focus towards improving problem formulations, understanding general theoretic frameworks for problem decompositions, major advances in EA computational efficiency, and most importantly aiding real decision-making in complex, uncertain application contexts

Fitness landscape-based analysis of nature-inspired algorithms

As the number of nature-inspired algorithms increases so does the need to characterise these algorithms. A rigorous process to characterise algorithms helps practitioners decide which algorithms may offer a good fit for their given problem. One approach is to relate the characteristics of a problem's associated fitness landscape with the performance of an algorithm. The aim of this thesis is to capitalise on the notion of fitness landscape characteristics as a technique for analysing algorithm performance, and to provide a novel algorithm- and problem-independent methodology that can be used to present the strengths and weaknesses of an algorithm. The methodology was tested by developing a portfolio of six nature-inspired algorithms commonly used to solve continuous optimisation problems. This portfolio includes the performance of these algorithms with parameters both “out of the box" and after they have been tuned using an automated tuning technique. Each of the algorithms shows a different “resilience" profile to the landscape characteristics, and responds differently to the tuning process. In order to provide a more practical way to utilise the portfolio an automated “ranking" methodology based on two machine learning techniques was developed. Using estimates of the fitness landscape characteristics on benchmark problems, the best algorithm to use is estimated, and compared with the actual performance of each algorithm. While results show that predicting algorithm performance is difficult, the results are promising, and show that this is an area worth exploring further. This methodology has significant advantages over the current practice of demonstrating novel algorithm performance on benchmark problems, most importantly offering a practical, generalised overview of the algorithm to a potential practitioner. Choosing to use a technique such as the one demonstrated here when presenting a novel algorithm could greatly ease the problem of algorithm selection

Landscape-based Evolutionary Algorithms for Dynamic Optimization Problems

In real-world structured optimization problems, specific objective functions, decision variables, constraints, data and/or parameters may vary over time. These problems are generally recognized as dynamic optimization problems (DOPs). Evolutionary computation (EC) is a stochastic global search approach that has been successfully used to find optimal or near-optimal solutions for a wide range of optimization problems. EC is conceptually simple and imposes no specific mathematical properties requirement, thus showing competitive performance in dealing with static optimization problems. However, EC encounters challenges in dynamic problems on adaptability and efficiency. For the employment of EC in DOPs, two key points should be considered: the nature of optimization problems to be solved and the class of algorithms to be designed, where the crucial element of the former is landscape analysis and the latter frequently leads to the type of the algorithm. A new approach named Landscape Influenced Dynamic Optimization Algorithm (LIDOA) is proposed to incorporate landscape analysis information into the search process, where a landscape-based strategy is integrated with appropriately designed evolutionary algorithms. In LIDOA, the knowledge learned in each landscape is archived and re-utilized in the new environment. Several classical evolutionary algorithms, including genetic algorithm (GA), self-adaptive differential evolution algorithm (jDE) and covariance matrix adaptation evolution strategy (CMA-ES), are employed to examine the efficiency of LIDOA, and four landscape measures are considered. Experimental results showed the overall advantage of LIDOA. LIDOA with a single landscape measure is then expanded to multiple landscape measures. Three multi-measure methods are designed that are able to achieve good performance on evolutionary algorithms with appropriately integrated multiple landscape measures. According to the experimental results, LIDOA with multi-measure methods also improves the performance of GA, jDE and CMA-ES. The second key point in employing multiple evolutionary algorithms in DOPs is also studied. Three multi-algorithm methods are investigated based on jDE and GA, where an information sharing strategy and a self-adjusted parameter strategy are designed. Experimental results show that with an appropriate integration mechanism, all three multi-algorithm methods can obtain better performance over a single algorithm. Two key parameters in multi-algorithm methods are discussed. The similarity check strategy with multi-measure is also integrated with three multi-algorithm methods, and experimental results demonstrate the efficacy of both multi-algorithm methods and multi-measure strategies. Furthermore, to show the applicability of the concept in other algorithms, it is tested on quantum-inspired evolutionary algorithms. The performance of LIDOA with quantum-inspired evolutionary algorithms shows that LIDOA and quantum operators are beneficial for jDE, GA and CMA-ES, though their contributions vary. Finally, the proposed algorithms are applied to two practical problems (parameter estimation for frequency-modulated (FM) sound waves and spread spectrum radar polyphase code design). With appropriately selected landscape measure(s), LIDOA is able to improve the performance on both problems. When the complexity of the two applicable problems increases, the proposed hybrid framework with a multi-algorithm and multi-measure method is more reliable

Bio-inspired computation: where we stand and what's next

In recent years, the research community has witnessed an explosion of literature dealing with the adaptation of behavioral patterns and social phenomena observed in nature towards efficiently solving complex computational tasks. This trend has been especially dramatic in what relates to optimization problems, mainly due to the unprecedented complexity of problem instances, arising from a diverse spectrum of domains such as transportation, logistics, energy, climate, social networks, health and industry 4.0, among many others. Notwithstanding this upsurge of activity, research in this vibrant topic should be steered towards certain areas that, despite their eventual value and impact on the field of bio-inspired computation, still remain insufficiently explored to date. The main purpose of this paper is to outline the state of the art and to identify open challenges concerning the most relevant areas within bio-inspired optimization. An analysis and discussion are also carried out over the general trajectory followed in recent years by the community working in this field, thereby highlighting the need for reaching a consensus and joining forces towards achieving valuable insights into the understanding of this family of optimization techniques

Bio-inspired computation: where we stand and what's next

In recent years, the research community has witnessed an explosion of literature dealing with the adaptation of behavioral patterns and social phenomena observed in nature towards efficiently solving complex computational tasks. This trend has been especially dramatic in what relates to optimization problems, mainly due to the unprecedented complexity of problem instances, arising from a diverse spectrum of domains such as transportation, logistics, energy, climate, social networks, health and industry 4.0, among many others. Notwithstanding this upsurge of activity, research in this vibrant topic should be steered towards certain areas that, despite their eventual value and impact on the field of bio-inspired computation, still remain insufficiently explored to date. The main purpose of this paper is to outline the state of the art and to identify open challenges concerning the most relevant areas within bio-inspired optimization. An analysis and discussion are also carried out over the general trajectory followed in recent years by the community working in this field, thereby highlighting the need for reaching a consensus and joining forces towards achieving valuable insights into the understanding of this family of optimization techniques