The Local Optima Level in Chemotherapy Schedule Optimisation

In this paper a multi-drug Chemotherapy Schedule Optimisation Problem (CSOP) is subject to Local Optima Network (LON) analysis. LONs capture global patterns in fitness landscapes. CSOPs have not previously been subject to fitness landscape analysis. We fill this gap: LONs are constructed and studied for meaningful structure. The CSOP formulation presents novel challenges and questions for the LON model because there are infeasible regions in the fitness landscape and an unknown global optimum; it also brings a topic from healthcare to LON analysis. Two LON Construction algorithms are proposed for sampling CSOP fitness landscapes: a Markov-Chain Construction Algorithm and a Hybrid Construction Algorithm. The results provide new insight into LONs of highly-constrained spaces, and into the proficiency of search operators on the CSOP. Iterated Local Search and Memetic Search, which are the foundations for the LON algorithms, are found to markedly out-perform a Genetic Algorithm from the literature

Inferring Future Landscapes: Sampling the Local Optima Level

Connection patterns among Local Optima Networks (LONs) can inform heuristic design for optimisation. LON research has predominantly required complete enumeration of a fitness landscape, thereby restricting analysis to problems diminutive in size compared to real-life situations. LON sampling algorithms are therefore important. In this paper, we study LON construction algorithms for the Quadratic Assignment Problem (QAP). Using machine learning, we use estimated LON features to predict search performance for competitive heuristics used in the QAP domain. The results show that by using random forest regression, LON construction algorithms produce fitness landscape features which can explain almost all search variance. We find that LON samples better relate to search than enumerated LONs do. The importance of fitness levels of sampled LONs in search predictions is crystallised. Features from LONs produced by different algorithms are combined in predictions for the first time, with promising results for this ‘super-sampling’: a model to predict tabu search success explained 99% of variance. Arguments are made for the use-case of each LON algorithm and for combining the exploitative process of one with the exploratory optimisation of the other

HPO: We won’t get fooled again

Hyperparameter optimization (HPO) is a well-studied research field. However, the effects and interactions of the components in an HPO pipeline are not yet well investigated. Then, we ask ourselves: Can the landscape of HPO be biased by the pipeline used to evaluate individual configurations? To address this question, we proposed to analyze the effect of the HPO pipeline on HPO problems using fitness landscape analysis. Particularly, we studied the DS-2019 HPO benchmark data set, looking for patterns that could indicate evaluation pipeline malfunction, and relate them to HPO performance. Our main findings are: (i) In most instances, large groups of diverse hyperparameters (i.e., multiple configurations) yield the same \emph{ill} performance, most likely associated with majority class prediction models; (ii) in these cases, a worsened correlation between the observed fitness and average fitness in the neighborhood is observed, potentially making harder the deployment of local-search-based HPO strategies. Finally, we concluded that the HPO pipeline definition might negatively affect the HPO landscape

Anatomy of the Local Optima Level in Combinatorial Optimisation

Many situations in daily life represent complex combinatorial optimisation problems. These include issues such as efficient fuel consumption, nurse scheduling, or distribution of humanitarian aid. There are many algorithms that attempt to solve these problems but the ability to understand their likely performance on a given problem is still lacking. Fitness landscape analysis identifies some of the reasons why metaheuristic algorithms behave in a particular way. The Local Optima Network (LON) model, proposed in 2008, encodes local optima connectivity in fitness landscapes. In this approach, nodes are local optima and edges encode transitions between these optima. A LON provides a static image of the dynamics of algorithm-problem inter- play. Analysing these structures provides insights into the reactions between optimisation problems and metaheuristic search algorithms. This thesis proposes that analysis of the local optima space of combinatorial fitness landscapes encoded using a LON provides important information concerning potential search algorithm performance. It considers the question as to whether or not features of LONs can contribute to explaining or predicting the outcome of trying to optimise an associated combinatorial problem. Topological landscape features of LONs are proposed, analysed and compared. Benchmark and novel problem instances are studied; both types of problem are sampled and in some cases exhaustively-enumerated such that LONs can be extracted for analysis. Investigations into the nature and biases of LON construction algorithms are conducted and compared. Contributions include aligning fractal geometry to the study of LONs; proposals for novel ways to compute fractal dimension from these structures; comparing the power of different LON construction algorithms for explaining algorithm performances; and analysing the interplay between algorithmic operations and infeasible regions in the local optima space using LONs as a tool. Throughout the thesis, large scale structural patterns in fitness landscapes are shown to be strongly linked with metaheuristic algorithm performance. This includes arrangements of local optima funnel structures; spatial and geometric complexity in the LON (measured by their fractal dimensionality) and fitness levels in the space of local optima. These features are demonstrated to have explanatory or predictive ability with respect to algorithm performance for the underlying combinatorial problems. The results presented here indicate that large topological patterns in fitness landscapes are important during metaheuristic search algorithm design. In many cases they are incontrovertibly linked to the success of the algorithm. These results indicate that use of the suggested fitness landscape measures would be highly beneficial when considering the design of search algorithms for a given problem domain

Community-driven dispersal in an individual-based predator-prey model

We present a spatial, individual-based predator-prey model in which dispersal is dependent on the local community. We determine species suitability to the biotic conditions of their local environment through a time and space varying fitness measure. Dispersal of individuals to nearby communities occurs whenever their fitness falls below a predefined tolerance threshold. The spatiotemporal dynamics of the model is described in terms of this threshold. We compare this dynamics with the one obtained through density-independent dispersal and find marked differences. In the community-driven scenario, the spatial correlations in the population density do not vary in a linear fashion as we increase the tolerance threshold. Instead we find the system to cross different dynamical regimes as the threshold is raised. Spatial patterns evolve from disordered, to scale-free complex patterns, to finally becoming well-organized domains. This model therefore predicts that natural populations, the dispersal strategies of which are likely to be influenced by their local environment, might be subject to complex spatiotemporal dynamics.Comment: 43 pages, 7 figures, vocabulary modifications, discussion expanded, references added, Ecological Complexity accepte

Who Replaces Whom? Local versus Non-local Replacement in Social and Evolutionary Dynamics

In this paper, we inspect well-known population genetics and social dynamics models. In these models, interacting individuals, while participating in a self-organizing process, give rise to the emergence of complex behaviors and patterns. While one main focus in population genetics is on the adaptive behavior of a population, social dynamics is more often concerned with the splitting of a connected array of individuals into a state of global polarization, that is, the emergence of speciation. Applying computational and mathematical tools we show that the way the mechanisms of selection, interaction and replacement are constrained and combined in the modeling have an important bearing on both adaptation and the emergence of speciation. Differently (un)constraining the mechanism of individual replacement provides the conditions required for either speciation or adaptation, since these features appear as two opposing phenomena, not achieved by one and the same model. Even though natural selection, operating as an external, environmental mechanism, is neither necessary nor sufficient for the creation of speciation, our modeling exercises highlight the important role played by natural selection in the interplay of the evolutionary and the self-organization modeling methodologies.Comment: 14 pages, 11 figure

Visualising Basins of Attraction for the Cross-Entropy and the Squared Error Neural Network Loss Functions

Quantification of the stationary points and the associated basins of attraction of neural network loss surfaces is an important step towards a better understanding of neural network loss surfaces at large. This work proposes a novel method to visualise basins of attraction together with the associated stationary points via gradient-based random sampling. The proposed technique is used to perform an empirical study of the loss surfaces generated by two different error metrics: quadratic loss and entropic loss. The empirical observations confirm the theoretical hypothesis regarding the nature of neural network attraction basins. Entropic loss is shown to exhibit stronger gradients and fewer stationary points than quadratic loss, indicating that entropic loss has a more searchable landscape. Quadratic loss is shown to be more resilient to overfitting than entropic loss. Both losses are shown to exhibit local minima, but the number of local minima is shown to decrease with an increase in dimensionality. Thus, the proposed visualisation technique successfully captures the local minima properties exhibited by the neural network loss surfaces, and can be used for the purpose of fitness landscape analysis of neural networks.Comment: Preprint submitted to the Neural Networks journa

Policies for allocation of information in task-oriented groups: elitism and egalitarianism outperform welfarism

Communication or influence networks are probably the most controllable of all factors that are known to impact on the problem-solving capability of task-forces. In the case connections are costly, it is necessary to implement a policy to allocate them to the individuals. Here we use an agent-based model to study how distinct allocation policies affect the performance of a group of agents whose task is to find the global maxima of NK fitness landscapes. Agents cooperate by broadcasting messages informing on their fitness and use this information to imitate the fittest agent in their influence neighborhoods. The larger the influence neighborhood of an agent, the more links, and hence information, the agent receives. We find that the elitist policy in which agents with above-average fitness have their influence neighborhoods amplified, whereas agents with below-average fitness have theirs deflated, is optimal for smooth landscapes, provided the group size is not too small. For rugged landscapes, however, the elitist policy can perform very poorly for certain group sizes. In addition, we find that the egalitarian policy, in which the size of the influence neighborhood is the same for all agents, is optimal for both smooth and rugged landscapes in the case of small groups. The welfarist policy, in which the actions of the elitist policy are reversed, is always suboptimal, i.e., depending on the group size it is outperformed by either the elitist or the egalitarian policies