Acquiring moving skills in robots with evolvable morphologies: Recent results and outlook

© 2017 ACM. We construct and investigate a strongly embodied evolutionary system, where not only the controllers but also the morphologies undergo evolution in an on-line fashion. In these studies, we have been using various types of robot morphologies and controller architectures in combination with several learning algorithms, e.g. evolutionary algorithms, reinforcement learning, simulated annealing, and HyperNEAT. This hands-on experience provides insights and helps us elaborate on interesting research directions for future development

Lamarck's Revenge: Inheritance of Learned Traits Can Make Robot Evolution Better

Evolutionary robot systems offer two principal advantages: an advanced way of developing robots through evolutionary optimization and a special research platform to conduct what-if experiments regarding questions about evolution. Our study sits at the intersection of these. We investigate the question ``What if the 18th-century biologist Lamarck was not completely wrong and individual traits learned during a lifetime could be passed on to offspring through inheritance?'' We research this issue through simulations with an evolutionary robot framework where morphologies (bodies) and controllers (brains) of robots are evolvable and robots also can improve their controllers through learning during their lifetime. Within this framework, we compare a Lamarckian system, where learned bits of the brain are inheritable, with a Darwinian system, where they are not. Analyzing simulations based on these systems, we obtain new insights about Lamarckian evolution dynamics and the interaction between evolution and learning. Specifically, we show that Lamarckism amplifies the emergence of `morphological intelligence', the ability of a given robot body to acquire a good brain by learning, and identify the source of this success: `newborn' robots have a higher fitness because their inherited brains match their bodies better than those in a Darwinian system.Comment: preprint-nature scientific report. arXiv admin note: text overlap with arXiv:2303.1259

Exploring Robot Morphology Spaces through Breadth-First Search and Random Query

Evolutionary robotics offers a powerful framework for designing and evolving robot morphologies, particularly in the context of modular robots. However, the role of query mechanisms during the genotype-to-phenotype mapping process has been largely overlooked. This research addresses this gap by conducting a comparative analysis of query mechanisms in the brain-body co-evolution of modular robots. Using two different query mechanisms, Breadth-First Search (BFS) and Random Query, within the context of evolving robot morphologies using CPPNs and robot controllers using tensors, and testing them in two evolutionary frameworks, Lamarckian and Darwinian systems, this study investigates their influence on evolutionary outcomes and performance. The findings demonstrate the impact of the two query mechanisms on the evolution and performance of modular robot bodies, including morphological intelligence, diversity, and morphological traits. This study suggests that BFS is both more effective and efficient in producing highly performing robots. It also reveals that initially, robot diversity was higher with BFS compared to Random Query, but in the Lamarckian system, it declines faster, converging to superior designs, while in the Darwinian system, BFS led to higher end-process diversity.Comment: arXiv admin note: text overlap with arXiv:2303.12594. substantial text overlap with arXiv:2309.1309

Objective versus Non-Objective Search in Evolving Morphologically Robust Robot Controllers

This study evaluates objective versus non-objective based evolutionary search methods for behavior evolution in robot teams. The goal is to evaluate the morphological robustness of evolved controllers, where controllers are evolved for specific robot sensory-motor configurations (morphologies) but must continue to function as these morphologies degrade. Robots use artificial neural network controllers where behavior evolution is directed by developmental neuro-evolution. Guiding evolutionary controller design we use objective (fitness function) versus nonobjective (novelty) search. The former optimizes for behavioral fitness and the latter for behavioral novelty. These methods are evaluated across varying robot morphologies and increasing task complexity. Results indicate that novelty search yields no benefits over objective search, in terms of evolving morphologically robust controllers. That is, both novelty and objective search evolve team controllers that are morphologically robust given varying robot morphologies and increasing task complexity. Results thus suggest behavioral diversity methods such as novelty search may not be suitable for generating robot behaviors that can continue functioning given changing robot morphologies, for example, due to damaged or disabled sensors and actuators

Lamarckian Evolution of Simulated Modular Robots

We study evolutionary robot systems where not only the robot brains but also the robot bodies are evolvable. Such systems need to include a learning period right after ‘birth' to acquire a controller that fits the newly created body. In this paper we investigate the possibility of bootstrapping infant robot learning through employing Lamarckian inheritance of parental controllers. In our system controllers are encoded by a combination of a morphology dependent component, a Central Pattern Generator (CPG), and a morphology independent part, a Compositional Pattern Producing Network (CPPN). This makes it possible to transfer the CPPN part of controllers between different morphologies and to create a Lamarckian system. We conduct experiments with simulated modular robots whose fitness is determined by the speed of locomotion, establish the benefits of inheriting optimized parental controllers, shed light on the conditions that influence these benefits, and observe that changing the way controllers are evolved also impacts the evolved morphologies

A comparison of controller architectures and learning mechanisms for arbitrary robot morphologies

The main question this paper addresses is: What combination of a robot controller and a learning method should be used, if the morphology of the learning robot is not known in advance? Our interest is rooted in the context of morphologically evolving modular robots, but the question is also relevant in general, for system designers interested in widely applicable solutions. We perform an experimental comparison of three controller-and-learner combinations: one approach where controllers are based on modelling animal locomotion (Central Pattern Generators, CPG) and the learner is an evolutionary algorithm, a completely different method using Reinforcement Learning (RL) with a neural network controller architecture, and a combination `in-between' where controllers are neural networks and the learner is an evolutionary algorithm. We apply these three combinations to a test suite of modular robots and compare their efficacy, efficiency, and robustness. Surprisingly, the usual CPG-based and RL-based options are outperformed by the in-between combination that is more robust and efficient than the other two setups

Is Novelty Search Good for Evolving Morphologically Robust Robot Controllers?

This study evaluates comparative behavioral search methods for evolutionary controller design in robot teams, where the goal is to evaluate the morphological robustness of evolved controllers. That is, where controllers are evolved for specific robot sensory-motor configurations (morphologies) but must continue to function as these morphologies degrade. Robots use neural controllers where behavior evolution is directed by developmental Neuro-Evolution (HyperNEAT). Guiding evolutionary controller design we use objective (fitness function) versus non-objective (novelty) search. The former optimizes for behavioral fitness and the latter for behavioral novelty. These search methods are evaluated across varying robot morphologies and increasing task complexity. Results indicate that both novelty and objective search evolve team controllers (behaviors) that are morphologically robust given degrading robot morphologies and increasing task complexity. Results thus suggest that novelty search is not necessarily suitable for generating robot team behaviors that are robust to changes in robot morphologies (for example, due to damaged or disabled sensors and actuators)

A Comparative Analysis of Darwinian Asexual and Sexual Reproduction in Evolutionary Robotics

Evolutionary Robotics systems draw inspiration from natural evolution to solve the problem of robot design. A key moment in the evolutionary process is reproduction, when the genotype of one or more parents is inherited by their offspring. Existent approaches have used both sexual and asexual reproduction but a comparison between the two is still missing. In this work, we study the effects of sexual and asexual reproduction on the controllers of an Evolutionary Robotics system. In our system, both morphologies and controllers are jointly evolved to solve two separate tasks. We adopt the Triangle of Life framework, in which the controllers go through a phase of learning before reproduction. Using extensive simulations we show that sexual reproduction of the robots' brains is preferable over asexual reproduction as it obtains better robots in terms of fitness. Moreover, we show that sexually reproducing robots present different morphologies and behaviors than the asexually reproducing ones, even though the reproduction mechanism only affects their brains. Finally, we study the effects of the reproduction mechanism on the robots' learning capabilities. By measuring the difference between the inherited and the learned brain we find that robots that evolved using sexual reproduction have better inherited brains and are also better learners

How the Morphology Encoding Influences the Learning Ability in Body-Brain Co-Optimization

Embedding the learning of controllers within the evolution of morphologies has emerged as an effective strategy for the co-optimization of agents' bodies and brains. Intuitively, that is how nature shaped animal life on Earth. Still, the design of such co-optimization is a complex endeavor; one issue is the choice of the genetic encoding for the morphology. Such choice can be crucial for the effectiveness of learning, i.e., how fast and to what degree agents adapt, through learning, during their life. Here we evolve the morphologies of voxel-based soft agents with two different encodings, direct and indirect while learning the controllers with reinforcement learning. We experiment with three tasks, ranging from cave crawling to beam toppling, and study how the encoding influences the learning outcome. Our results show that the direct encoding corresponds to increased ability to learn, mostly in terms of learning speed. The same is not always true for the indirect one. We link these results to different shades of the Baldwin effect, consisting of morphologies being selected for increasing an agent’s ability to learn during its lifetime