Improving Scalability of Evolutionary Robotics with Reformulation

Creating systems that can operate autonomously in complex environments is a challenge for contemporary engineering techniques. Automatic design methods offer a promising alternative, but so far they have not been able to produce agents that outperform manual designs. One such method is evolutionary robotics. It has been shown to be a robust and versatile tool for designing robots to perform simple tasks, but more challenging tasks at present remain out of reach of the method. In this thesis I discuss and attack some problems underlying the scalability issues associated with the method. I present a new technique for evolving modular networks. I show that the performance of modularity-biased evolution depends heavily on the morphology of the robot’s body and present a new method for co-evolving morphology and modular control. To be able to reason about the new technique I develop reformulation framework: a general way to describe and reason about metaoptimization approaches. Within this framework I describe a new heuristic for developing metaoptimization approaches that is based on the technique for co-evolving morphology and modularity. I validate the framework by applying it to a practical task of zero-g autonomous assembly of structures with a fleet of small robots. Although this work focuses on the evolutionary robotics, methods and approaches developed within it can be applied to optimization problems in any domain

Lexicase Selection for Multi-Task Evolutionary Robotics

In Evolutionary Robotics, Lexicase selection has proven effective when a single task is broken down into many individual parameterizations. Evolved individuals have generalized across unique configurations of an overarching task. Here, we investigate the ability of Lexicase selection to generalize across multiple tasks, with each task again broken down into many instances. There are three objectives: to determine the feasibility of introducing additional tasks to the existing platform; to investigate any consequential effects of introducing these additional tasks during evolutionary adaptation; and to explore whether the schedule of presentation of the additional tasks over evolutionary time affects the final outcome. To address these aims we use a quadruped animat controlled by a feed-forward neural network with joint-angle, bearing-to-target, and spontaneous sinusoidal inputs. Weights in this network are trained using evolution with Lexicase-based parent selection. Simultaneous adaptation in a wall crossing task (labelled wall-cross) is explored when one of two different alternative tasks is also present: turn-and-seek or cargo-carry. Each task is parameterized into 100 distinct variants, and these variants are used as environments for evaluation and selection with Lexicase. We use performance in a single-task wall-cross environment as a baseline against which to examine the multi-task configurations. In addition, the objective sampling strategy (the manner in which tasks are presented over evolutionary time) is varied, and so data for treatments implementing uniform sampling, even sampling, or degrees of generational sampling are also presented. The Lexicase mechanism successfully integrates evolution of both turn-and-seek and cargo-carry with wall-cross, though there is a performance penalty compared to single task evolution. The size of the penalty depends on the similarity of the tasks. Complementary tasks (wallcross/turn-and-seek) show better performance than antagonistic tasks (wall-cross/cargo-carry). In complementary tasks performance is not affected by the sampling strategy. Where tasks are antagonistic, uniform and even sampling strategies yield significantly better performance than generational sampling. In all cases the generational sampling requires more evaluations and consequently more computational resources. The results indicate that Lexicase is a viable mechanism for multitask evolution of animat neurocontrollers, though the degree of interference between tasks is a key consideration. The results also support the conclusion that the naive, uniform random sampling strategy is the best choice when considering final task performance, simplicity of implementation, and computational efficiency

Diversifying Emergent Behaviours with Age-Layered MAP-Elites

Emergent behaviour can arise unexpectedly as a by-product of the complex interactions of an autonomous system, and with the increasing desire for such systems, emergent behaviour has become an important area of interest for AI research. One aspect of this research is in searching for a diverse set of emergent behaviours which not only provides a useful tool for finding unwanted emergent behaviour, but also in finding interesting emergent behaviour. The multi-dimensional archive of phenotypic elites (MAP-Elites) algorithm is a popular evolutionary algorithm which returns a highly diverse set of elite solutions at the end of a run. The population is separated into a grid-like feature space defined by a set of behaviour dimensions specified by the user where each cell of the grid corresponds to a unique behaviour combination. The algorithm is conceptually simple and effective at producing high-quality, diverse solutions, but it comes with a major limitation on its exploratory capabilities. With each additional behaviour, the set of solutions grows exponentially, making high-dimensional feature spaces infeasible. This thesis proposes an option for increasing behaviours with a novel Age-Layered MAP-Elites (ALME) algorithm where the population is separated into age layers and each layer has its own feature space. By using different behaviours in the different layers, the population migrates up through the layers experiencing selective pressure towards different behaviours. This algorithm is applied to a simulated intelligent agent environment to observe interesting emergent behaviours. It is observed that ALME is capable of producing a set of solutions with diversity in all behaviour dimensions while keeping the final population size low. It is also observed that ALME is capable of filling its top layer feature space more consistently than MAP-Elites with the same behaviour dimensions

Meta-Learning for Symbolic Hyperparameter Defaults

Hyperparameter optimization in machine learning (ML) deals with the problem of empirically learning an optimal algorithm configuration from data, usually formulated as a black-box optimization problem. In this work, we propose a zero-shot method to meta-learn symbolic default hyperparameter configurations that are expressed in terms of the properties of the dataset. This enables a much faster, but still data-dependent, configuration of the ML algorithm, compared to standard hyperparameter optimization approaches. In the past, symbolic and static default values have usually been obtained as hand-crafted heuristics. We propose an approach of learning such symbolic configurations as formulas of dataset properties from a large set of prior evaluations on multiple datasets by optimizing over a grammar of expressions using an evolutionary algorithm. We evaluate our method on surrogate empirical performance models as well as on real data across 6 ML algorithms on more than 100 datasets and demonstrate that our method indeed finds viable symbolic defaults.Comment: Pieter Gijsbers and Florian Pfisterer contributed equally to the paper. V1: Two page GECCO poster paper accepted at GECCO 2021. V2: The original full length paper (8 pages) with appendi

A Survey of Monte Carlo Tree Search Methods

Monte Carlo tree search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. It has received considerable interest due to its spectacular success in the difficult problem of computer Go, but has also proved beneficial in a range of other domains. This paper is a survey of the literature to date, intended to provide a snapshot of the state of the art after the first five years of MCTS research. We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarize the results from the key game and nongame domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work