2,553 research outputs found
A survey of techniques for characterising fitness landscapes and some possible ways forward
Real-world optimisation problems are often very complex. Metaheuristics have been successful
in solving many of these problems, but the difficulty in choosing the best approach
can be a huge challenge for practitioners. One approach to this dilemma is to use fitness
landscape analysis to better understand problems before deciding on approaches to solving
the problems. However, despite extensive research on fitness landscape analysis and a
large number of developed techniques, very few techniques are used in practice. This could
be because fitness landscape analysis in itself can be complex. In an attempt to make fitness
landscape analysis techniques accessible, this paper provides an overview of techniques
from the 1980s to the present. Attributes that are important for practical
implementation are highlighted and ways of adapting techniques to be more feasible or
appropriate are suggested. The survey reveals the wide range of factors that can influence
problem difficulty, emphasising the need for a shift in focus away from predicting problem
hardness towards measuring characteristics. It is hoped that this survey will invoke
renewed interest in the field of understanding complex optimisation problems and ultimately
lead to better decision making on the use of appropriate metaheuristics.http://www.elsevier.com/locate/inshb201
Tools for Landscape Analysis of Optimisation Problems in Procedural Content Generation for Games
The term Procedural Content Generation (PCG) refers to the (semi-)automatic
generation of game content by algorithmic means, and its methods are becoming
increasingly popular in game-oriented research and industry. A special class of
these methods, which is commonly known as search-based PCG, treats the given
task as an optimisation problem. Such problems are predominantly tackled by
evolutionary algorithms.
We will demonstrate in this paper that obtaining more information about the
defined optimisation problem can substantially improve our understanding of how
to approach the generation of content. To do so, we present and discuss three
efficient analysis tools, namely diagonal walks, the estimation of high-level
properties, as well as problem similarity measures. We discuss the purpose of
each of the considered methods in the context of PCG and provide guidelines for
the interpretation of the results received. This way we aim to provide methods
for the comparison of PCG approaches and eventually, increase the quality and
practicality of generated content in industry.Comment: 30 pages, 8 figures, accepted for publication in Applied Soft
Computin
Visualising the Global Structure of Search Landscapes: Genetic Improvement as a Case Study
The search landscape is a common metaphor to describe the structure of computational search spaces. Different landscape metrics can be computed and used to predict search difficulty. Yet, the metaphor falls short in visualisation terms because it is hard to represent complex landscapes, both in terms of size and dimensionality. This paper combines Local Optima Networks, as a compact representation of the global structure of a search space, and dimensionality reduction, using the t-Distributed Stochastic Neighbour Embedding (t-SNE) algorithm, in order to both bring the metaphor to life and convey new insight into the search process. As a case study, two benchmark programs, under a Genetic Improvement bug-fixing scenario, are analysed and visualised using the proposed method. Local Optima Networks for both iterated local search and a hybrid genetic algorithm, across different neighbourhoods, are compared, highlighting the differences in how the landscape is explored
Insights into the feature selection problem using local optima networks
The binary feature selection problem is investigated in this paper. Feature selection fitness landscape analysis is done, which allows for a better understanding of the behaviour of feature selection algorithms. Local optima networks are employed as a tool to visualise and characterise the fitness landscapes of the feature selection problem in the context of classification. An analysis of the fitness landscape global structure is provided, based on seven real-world datasets with up to 17 features. Formation of neutral global optima plateaus are shown to indicate the existence of irrelevant features in the datasets. Removal of irrelevant features resulted in a reduction of neutrality and the ratio of local optima to the size of the search space, resulting in improved performance of genetic algorithm search in finding the global optimum
Empirical Loss Landscape Analysis of Neural Network Activation Functions
Activation functions play a significant role in neural network design by
enabling non-linearity. The choice of activation function was previously shown
to influence the properties of the resulting loss landscape. Understanding the
relationship between activation functions and loss landscape properties is
important for neural architecture and training algorithm design. This study
empirically investigates neural network loss landscapes associated with
hyperbolic tangent, rectified linear unit, and exponential linear unit
activation functions. Rectified linear unit is shown to yield the most convex
loss landscape, and exponential linear unit is shown to yield the least flat
loss landscape, and to exhibit superior generalisation performance. The
presence of wide and narrow valleys in the loss landscape is established for
all activation functions, and the narrow valleys are shown to correlate with
saturated neurons and implicitly regularised network configurations.Comment: Accepted for publication in Genetic and Evolutionary Computation
Conference Companion, July 15--19, 2023, Lisbon, Portuga
- …