Vector Field Embryogeny

We present a novel approach toward evolving artificial embryogenies, which omits the graph representation of gene regulatory networks and directly shapes the dynamics of a system, i.e., its phase space. We show the feasibility of the approach by evolving cellular differentiation, a basic feature of both biological and artificial development. We demonstrate how a spatial hierarchy formulation can be integrated into the framework and investigate the evolution of a hierarchical system. Finally, we show how the framework allows the investigation of allometry, a biological phenomenon, and its role for evolution. We find that direct evolution of allometric change, i.e., the evolutionary adaptation of the speed of system states on transient trajectories in phase space, is advantageous for a cellular differentiation task

Evolutionary development of tensegrity structures

Contributions from the emerging fields of molecular genetics and evo-devo (evolutionary developmental biology) are greatly benefiting the field of evolutionary computation, initiating a promise of renewal in the traditional methodology. While direct encoding has constituted a dominant paradigm, indirect ways to encode the solutions have been reported, yet little attention has been paid to the benefits of the proposed methods to real problems. In this work, we study the biological properties that emerge by means of using indirect encodings in the context of form-finding problems. A novel indirect encoding model for artificial development has been defined and applied to an engineering structural-design problem, specifically to the discovery of tensegrity structures. This model has been compared with a direct encoding scheme. While the direct encoding performs similarly well to the proposed method, indirect-based results typically outperform the direct-based results in aspects not directly linked to the nature of the problem itself, but to the emergence of properties found in biological organisms, like organicity, generalization capacity, or modularity aspects which are highly valuable in engineering

Robust Multi-Cellular Developmental Design

This paper introduces a continuous model for Multi-cellular Developmental Design. The cells are fixed on a 2D grid and exchange "chemicals" with their neighbors during the growth process. The quantity of chemicals that a cell produces, as well as the differentiation value of the cell in the phenotype, are controlled by a Neural Network (the genotype) that takes as inputs the chemicals produced by the neighboring cells at the previous time step. In the proposed model, the number of iterations of the growth process is not pre-determined, but emerges during evolution: only organisms for which the growth process stabilizes give a phenotype (the stable state), others are declared nonviable. The optimization of the controller is done using the NEAT algorithm, that optimizes both the topology and the weights of the Neural Networks. Though each cell only receives local information from its neighbors, the experimental results of the proposed approach on the 'flags' problems (the phenotype must match a given 2D pattern) are almost as good as those of a direct regression approach using the same model with global information. Moreover, the resulting multi-cellular organisms exhibit almost perfect self-healing characteristics

Evolvability signatures of generative encodings: beyond standard performance benchmarks

Evolutionary robotics is a promising approach to autonomously synthesize machines with abilities that resemble those of animals, but the field suffers from a lack of strong foundations. In particular, evolutionary systems are currently assessed solely by the fitness score their evolved artifacts can achieve for a specific task, whereas such fitness-based comparisons provide limited insights about how the same system would evaluate on different tasks, and its adaptive capabilities to respond to changes in fitness (e.g., from damages to the machine, or in new situations). To counter these limitations, we introduce the concept of "evolvability signatures", which picture the post-mutation statistical distribution of both behavior diversity (how different are the robot behaviors after a mutation?) and fitness values (how different is the fitness after a mutation?). We tested the relevance of this concept by evolving controllers for hexapod robot locomotion using five different genotype-to-phenotype mappings (direct encoding, generative encoding of open-loop and closed-loop central pattern generators, generative encoding of neural networks, and single-unit pattern generators (SUPG)). We observed a predictive relationship between the evolvability signature of each encoding and the number of generations required by hexapods to adapt from incurred damages. Our study also reveals that, across the five investigated encodings, the SUPG scheme achieved the best evolvability signature, and was always foremost in recovering an effective gait following robot damages. Overall, our evolvability signatures neatly complement existing task-performance benchmarks, and pave the way for stronger foundations for research in evolutionary robotics.Comment: 24 pages with 12 figures in the main text, and 4 supplementary figures. Accepted at Information Sciences journal (in press). Supplemental videos are available online at, see http://goo.gl/uyY1R

Behavior finding: Morphogenetic Designs Shaped by Function

Evolution has shaped an incredible diversity of multicellular living organisms, whose complex forms are self-made through a robust developmental process. This fundamental combination of biological evolution and development has served as an inspiration for novel engineering design methodologies, with the goal to overcome the scalability problems suffered by classical top-down approaches. Top-down methodologies are based on the manual decomposition of the design into modular, independent subunits. In contrast, recent computational morphogenetic techniques have shown that they were able to automatically generate truly complex innovative designs. Algorithms based on evolutionary computation and artificial development have been proposed to automatically design both the structures, within certain constraints, and the controllers that optimize their function. However, the driving force of biological evolution does not resemble an enumeration of design requirements, but much rather relies on the interaction of organisms within the environment. Similarly, controllers do not evolve nor develop separately, but are woven into the organism’s morphology. In this chapter, we discuss evolutionary morphogenetic algorithms inspired by these important aspects of biological evolution. The proposed methodologies could contribute to the automation of processes that design “organic” structures, whose morphologies and controllers are intended to solve a functional problem. The performance of the algorithms is tested on a class of optimization problems that we call behavior-finding. These challenges are not explicitly based on morphology or controller constraints, but only on the solving abilities and efficacy of the design. Our results show that morphogenetic algorithms are well suited to behavior-finding

Evolutionary Reinforcement Learning: A Survey

Reinforcement learning (RL) is a machine learning approach that trains agents to maximize cumulative rewards through interactions with environments. The integration of RL with deep learning has recently resulted in impressive achievements in a wide range of challenging tasks, including board games, arcade games, and robot control. Despite these successes, there remain several crucial challenges, including brittle convergence properties caused by sensitive hyperparameters, difficulties in temporal credit assignment with long time horizons and sparse rewards, a lack of diverse exploration, especially in continuous search space scenarios, difficulties in credit assignment in multi-agent reinforcement learning, and conflicting objectives for rewards. Evolutionary computation (EC), which maintains a population of learning agents, has demonstrated promising performance in addressing these limitations. This article presents a comprehensive survey of state-of-the-art methods for integrating EC into RL, referred to as evolutionary reinforcement learning (EvoRL). We categorize EvoRL methods according to key research fields in RL, including hyperparameter optimization, policy search, exploration, reward shaping, meta-RL, and multi-objective RL. We then discuss future research directions in terms of efficient methods, benchmarks, and scalable platforms. This survey serves as a resource for researchers and practitioners interested in the field of EvoRL, highlighting the important challenges and opportunities for future research. With the help of this survey, researchers and practitioners can develop more efficient methods and tailored benchmarks for EvoRL, further advancing this promising cross-disciplinary research field

Meta-parametric design: Developing a computational approach for early stage collaborative practice

Computational design is the study of how programmable computers can be integrated into the process of design. It is not simply the use of pre-compiled computer aided design software that aims to replicate the drawing board, but rather the development of computer algorithms as an integral part of the design process. Programmable machines have begun to challenge traditional modes of thinking in architecture and engineering, placing further emphasis on process ahead of the final result. Just as Darwin and Wallace had to think beyond form and inquire into the development of biological organisms to understand evolution, so computational methods enable us to rethink how we approach the design process itself. The subject is broad and multidisciplinary, with influences from design, computer science, mathematics, biology and engineering. This thesis begins similarly wide in its scope, addressing both the technological aspects of computational design and its application on several case study projects in professional practice. By learning through participant observation in combination with secondary research, it is found that design teams can be most effective at the early stage of projects by engaging with the additional complexity this entails. At this concept stage, computational tools such as parametric models are found to have insufficient flexibility for wide design exploration. In response, an approach called Meta-Parametric Design is proposed, inspired by developments in genetic programming (GP). By moving to a higher level of abstraction as computational designers, a Meta-Parametric approach is able to adapt to changing constraints and requirements whilst maintaining an explicit record of process for collaborative working

Spicing up map generation

We describe a search-based map generator for the classic real-time strategy game Dune 2. The generator is capable of creating playable maps in seconds, which can be used with a partial recreation of Dune 2 that has been implemented using the Strategy Game Description Language. Map genotypes are represented as low-resolution matrices, which are then converted to higher-resolution maps through a stochastic process involving cellular automata. Map phenotypes are evaluated using a set of heuristics based on the gameplay requirements of Dune 2.peer-reviewe

The Two Regimes of Neutral Evolution: Localization on Hubs and Delocalized Diffusion

It has been argued that much of evolution takes place in the absence of fitness gradients. Such periods of evolution can be analysed by examining the mutational network formed by sequences of equal fitness, that is the neutral network. It has been demonstrated that, in large populations under a high mutation rate, the population distribution over the neutral network and average mutational robustness are given by the principle eigenvector and eigenvalue, respectively, of the network’s adjacency matrix. However, little progress has been made towards understanding the manner in which the topology of the neutral network influences the resulting population distribution and robustness. In this work, we build on recent results from spectral graph theory and utilize numerical methods to demonstrate that there exist two regimes of behaviour: convergence on hubs and diffusion over the network. We also derive approximations for the population’s behaviour under these regimes. This challenges the widespread assumption that neutral evolution always leads to exploration of the neutral network and elucidates the conditions which result in the evolution of robust organisms