Understanding Optimisation Processes with Biologically-Inspired Visualisations

Evolutionary algorithms (EAs) constitute a branch of artificial intelligence utilised to evolve solutions to solve optimisation problems abound in industry and research. EAs often generate many solutions and visualisation has been a primary strategy to display EA solutions, given that visualisation is a multi-domain well-evaluated medium to comprehend extensive data. The endeavour of visualising solutions is inherent with challenges resulting from high dimensional phenomenons and the large number of solutions to display. Recently, scholars have produced methods to mitigate some of these known issues when illustrating solutions. However, one key consideration is that displaying the final subset of solutions exclusively (rather than the whole population) discards most of the informativeness of the search, creating inadequate insight into the black-box EA. There is an unequivocal knowledge gap and requirement for methods which can visualise the whole population of solutions from an optimiser and subjugate the high-dimensional problems and scaling issues to create interpretability of the EA search process. Furthermore, a requirement for explainability in evolutionary computing has been demanded by the evolutionary computing community, which could take the form of visualisations, to support EA comprehension much like the support explainable artificial intelligence has brought to artificial intelligence. In this thesis, we report novel visualisation methods that can be used to visualise large and high-dimensional optimiser populations with the aim of creating greater interpretability during a search. We consider the nascent intersection of visualisation and explainability in evolutionary computing. The potential high informativeness of a visualisation method from an early chapter of this work forms an effective platform to develop an explainability visualisation method, namely the population dynamics plot, to attempt to inject explainability into the inner workings of the search process. We further support the visualisation of populations using machine learning to construct models which can capture the characteristics of an EA search and develop intelligent visualisations which use artificial intelligence to potentially enhance and support visualisation for a more informative search process. The methods developed in this thesis are evaluated both quantitatively and qualitatively. We use multi-feature benchmark problems to show the method’s ability to reveal specific problem characteristics such as disconnected fronts, local optima and bias, as well as potentially creating a better understanding of the problem landscape and optimiser search for evaluating and comparing algorithm performance (we show the visualisation method to be more insightful than conventional metrics like hypervolume alone). One of the most insightful methods developed in this thesis can produce a visualisation requiring less than 1% of the time and memory necessary to produce a visualisation of the same objective space solutions using existing methods. This allows for greater scalability and the use in short compile time applications such as online visualisations. Predicated by an existing visualisation method in this thesis, we then develop and apply an explainability method to a real-world problem and evaluate it to show the method to be highly effective at explaining the search via solutions in the objective spaces, solution lineage and solution variation operators to compactly comprehend, evaluate and communicate the search of an optimiser, although we note the explainability properties are only evaluated against the author’s ability and could be evaluated further in future work with a usability study. The work is then supported by the development of intelligent visualisation models that may allow one to predict solutions in optima (importantly local optima) in unseen problems by using a machine learning model. The results are effective, with some models able to predict and visualise solution optima with a balanced F1 accuracy metric of 96%. The results of this thesis provide a suite of visualisations which aims to provide greater informativeness of the search and scalability than previously existing literature. The work develops one of the first explainability methods aiming to create greater insight into the search space, solution lineage and reproductive operators. The work applies machine learning to potentially enhance EA understanding via visualisation. These models could also be used for a number of applications outside visualisation. Ultimately, the work provides novel methods for all EA stakeholders which aims to support understanding, evaluation and communication of EA processes with visualisation

Semantic Evolutionary Visualization

The Evolutionary optimization (EO) field has become an active area of research for handling complex optimization problems. However, EO techniques need tuning to obtain better results. Visualization is one approach used by EA researchers to identify early stagnation, lossof diversity, and other indicators that can help them to guide evolutionary search to better areas. In this paper, a Semantic EvolutionaryVisualization framework (SEV) is proposed for analysing and exploringthe potential EA dynamics. Empirical results have shown that the SEVcan help to reveal and monitor information on evolutionary dynamics;thus, it can assist researchers in adapting the evolutionary parametersto obtain better performance

Hidden-Markov-Based Self-adaptive Differential Evolution

Heuristic search is an efficient way to solve complex optimization problems, and sometimes it is the only way to do so. Differential Evolution (DE) is a population-based heuristic search suitable mostly for continuous optimization problems. The efficiency of DE to optimize a problem can largely degrade if the right values for its parameters are not chosen. Finding the right values for DE’s parameters is a non-trivial task. Many researchers resort to parameter tuning and self-adaptation mechanisms. Existing methods vary in their performance and design philosophies.In this thesis, I start by introducing a semantic evolutionary visualization framework to investigate evolutionary dynamics. The different visualizations track the ongoing changes within an evolutionary run by exploring pedigree trees and the fitness landscapes. The visualization alone was not sufficient to shed light on a very high dimensional space. Consequently, I resorted to introducing a new self-adaptive algorithm using Hidden Markov Models (HMMs).Markov models have been used extensively in the past to analyze convergence of evolutionary optimization methods. I have leveraged this opportunity to introduce a new algorithm that we call DE-HMM, where HMMs is used for real-time learning of evolutionary dynamics to allow for dynamic adjustment of the two intrinsic DE parameters: F and CR.DE-HMM categorizes each evolutionary transition into two discrete states; low and high, representing the rate of change in a population over time. The HMM posterior and likelihood ratios are estimated to assign the values for F and CR during the evolutionary process. Two unconstrained benchmark set are used to assess DE-HMM performance, demonstrating its overall superiority in terms of solution quality and computational resources, when compared to other state-of-the art algorithms.The self-adaptive DE-HMM is then augmented with local search to solve constrained optimization problems. A two-stage method is introduced; with the two states being either global or local based on the degree of feasibility and rate of diversity. The methodology demonstrated competitive results when tested on the constrained CEC2010 benchmark dataset