Evolving a Behavioral Repertoire for a Walking Robot

Numerous algorithms have been proposed to allow legged robots to learn to walk. However, the vast majority of these algorithms is devised to learn to walk in a straight line, which is not sufficient to accomplish any real-world mission. Here we introduce the Transferability-based Behavioral Repertoire Evolution algorithm (TBR-Evolution), a novel evolutionary algorithm that simultaneously discovers several hundreds of simple walking controllers, one for each possible direction. By taking advantage of solutions that are usually discarded by evolutionary processes, TBR-Evolution is substantially faster than independently evolving each controller. Our technique relies on two methods: (1) novelty search with local competition, which searches for both high-performing and diverse solutions, and (2) the transferability approach, which com-bines simulations and real tests to evolve controllers for a physical robot. We evaluate this new technique on a hexapod robot. Results show that with only a few dozen short experiments performed on the robot, the algorithm learns a repertoire of con-trollers that allows the robot to reach every point in its reachable space. Overall, TBR-Evolution opens a new kind of learning algorithm that simultaneously optimizes all the achievable behaviors of a robot.Comment: 33 pages; Evolutionary Computation Journal 201

Darwin's Avatars: a Novel Combination of Gameplay and Procedural Content Generation

The co-evolution of morphology and control for virtual crea-tures enables the creation of a novel form of gameplay and procedural content generation. Starting with a creature evolved to perform a simple task such as locomotion and removing its brain, the remaining body can be employed in a compelling interactive control problem. Just as we en-joy the challenge and reward of mastering helicopter flight or learning to play a musical instrument, learning to con-trol such a creature through manual activation of its actu-ators presents an engaging and rewarding puzzle. Impor-tantly, the novelty of this challenge is inexhaustible, since the evolution of virtual creatures provides a way to proce-durally generate content for such a game. An endless series of creatures can be evolved for a task, then have their brains removed to become the game’s next human-control chal-lenge. To demonstrate this new form of gameplay and con-tent generation, a proof-of-concept game—tentatively titled Darwin’s Avatars—was implemented using evolved creature content, and user tested. This implementation also provided a unique opportunity to compare human and evolved control of evolved virtual creatures, both qualitatively and quanti-tatively, with interesting implications for improvements and future work

Open-ended search for environments and adapted agents using MAP-Elites

Creatures in the real world constantly encounter new and diverse challenges they have never seen before. They will often need to adapt to some of these tasks and solve them in order to survive. This almost endless world of novel challenges is not as common in virtual environments, where artificially evolving agents often have a limited set of tasks to solve. An exception to this is the field of open-endedness where the goal is to create unbounded exploration of interesting artefacts. We want to move one step closer to creating simulated environments similar to the diverse real world, where agents can both find solvable tasks, and adapt to them. Through the use of MAP-Elites we create a structured repertoire, a map, of terrains and virtual creatures that locomote through them. By using novelty as a dimension in the grid, the map can continuously develop to encourage exploration of new environments. The agents must adapt to the environments found, but can also search for environments within each cell of the grid to find the one that best fits their set of skills. Our approach combines the structure of MAP-Elites, which can allow the virtual creatures to use adjacent cells as stepping stones to solve increasingly difficult environments, with open-ended innovation. This leads to a search that is unbounded, but still has a clear structure. We find that while handcrafted bounded dimensions for the map lead to quicker exploration of a large set of environments, both the bounded and unbounded approach manage to solve a diverse set of terrains

Material properties affect evolution's ability to exploit morphological computation in growing soft-bodied creatures

The concept of morphological computation holds that the body of an agent can, under certain circumstances, exploit the interaction with the environment to achieve useful behavior, potentially reducing the computational burden of the brain/controller. The conditions under which such phenomenon arises are, however, unclear. We hypothesize that morphological computation will be facilitated by body plans with appropriate geometric, material, and growth properties, while it will be hindered by other body plans in which one or more of these three properties is not well suited to the task. We test this by evolving the geometries and growth processes of soft robots, with either manually-set softer or stiffer material properties. Results support our hypothesis: we find that for the task investigated, evolved softer robots achieve better performances with simpler growth processes than evolved stiffer ones. We hold that the softer robots succeed because they are better able to exploit morphological computation. This four-way interaction among geometry, growth, material properties and morphological computation is but one example phenomenon that can be investigated using the system here introduced, that could enable future studies on the evolution and development of generic soft-bodied creatures

Simultaneous incremental neuroevolution of motor control, navigation and object manipulation in 3D virtual creatures

There have been numerous attempts to develop 3D virtual agents by applying evolutionary processes to populations that exist in a realistic physical simulation. Whilst often contributing useful knowledge, no previous work has demonstrated the capacity to evolve a sequence of increasingly complex behaviours in a single, unified system. This thesis has this demonstration as its primary aim. A rigorous exploration of one aspect of incremental artificial evolution was carried out to understand how subtask presentations affect the whole-task generalisation performance of evolved, fixed-morphology 3D agents. Results from this work led to the design of an environment–body–control architecture that can be used as a base for evolving multiple behaviours incrementally. A simulation based on this architecture with a more complex environment was then developed and explored. This system was then adapted to include elements of physical manipulation as a first step toward a fully physical virtual creature environment demonstrating advanced evolved behaviours. The thesis demonstrates that incremental evolutionary systems can be subject to problems of forgetting and loss of gradient, and that different complexification strategies have a strong bearing on the management of these issues. Presenting successive generations of the population to a full range of objective functions (covering and revisiting the range of complexity) outperforms straightforward linear or direct presentations, establishing a more robust approach to the evolution of naturalistic embodied agents. When combining this approach with a bespoke control architecture in a problem requiring reactive and deliberative behaviours, we see results that not only demonstrate success at the tasks, but also show a variety of intricate behaviours being used. This is the first ever example of the simultaneous incremental evolution in 3D of composite behaviours more complex than simple locomotion. Finally, the architecture demonstrably supports extension to manipulation in a feedback control task. Given the problem-agnostic controller architecture, these results indicate a system with potential for discovering yet more advanced behaviours in yet more complex environments

Digital control networks for virtual creatures

Robot control systems evolved with genetic algorithms traditionally take the form of floating-point neural network models. This thesis proposes that digital control systems, such as quantised neural networks and logical networks, may also be used for the task of robot control. The inspiration for this is the observation that the dynamics of discrete networks may contain cyclic attractors which generate rhythmic behaviour, and that rhythmic behaviour underlies the central pattern generators which drive lowlevel motor activity in the biological world. To investigate this a series of experiments were carried out in a simulated physically realistic 3D world. The performance of evolved controllers was evaluated on two well known control tasks—pole balancing, and locomotion of evolved morphologies. The performance of evolved digital controllers was compared to evolved floating-point neural networks. The results show that the digital implementations are competitive with floating-point designs on both of the benchmark problems. In addition, the first reported evolution from scratch of a biped walker is presented, demonstrating that when all parameters are left open to evolutionary optimisation complex behaviour can result from simple components

On the Evolutionary Co-Adaptation of Morphology and Distributed Neural Controllers in Adaptive Agents

The attempt to evolve complete embodied and situated artificial creatures in which both morphological and control characteristics are adapted during the evolutionary process has been and still represents a long term goal key for the artificial life and the evolutionary robotics community. Loosely inspired by ancient biological organisms which are not provided with a central nervous system and by simple organisms such as stick insects, this thesis proposes a new genotype encoding which allows development and evolution of mor- phology and neural controller in artificial agents provided with a distributed neural network. In order to understand if this kind of network is appropriate for the evolution of non trivial behaviours in artificial agents, two experiments (description and results will be shown in chapter 3) in which evolution was applied only to the controller’s parameters were performed. The results obtained in the first experiment demonstrated how distributed neural networks can achieve a good level of organization by synchronizing the output of oscillatory elements exploiting acceleration/deceleration mechanisms based on local interactions. In the second experiment few variants on the topology of neural architecture were introduced. Results showed how this new control system was able to coordinate the legs of a simulated hexapod robot on two different gaits on the basis of the external circumstances. After this preliminary and successful investigation, a new genotype encoding able to develop and evolve artificial agents with no fixed morphology and with a distributed neural controller was proposed. A second set of experiments was thus performed and the results obtained confirmed both the effectiveness of genotype encoding and the ability of distributed neural network to perform the given task. The results have also shown the strength of genotype both in generating a wide range of different morphological structures and in favouring a direct co-adaptation between neural controller and morphology during the evolutionary process. Furthermore the simplicity of the proposed model has showed the effective role of specific elements in evolutionary experiments. In particular it has demonstrated the importance of the environment and its complexity in evolving non-trivial behaviours and also how adding an independent component to the fitness function could help the evolutionary process exploring a larger space solutions avoiding a premature convergence towards suboptimal solutions