Scalable Co-Optimization of Morphology and Control in Embodied Machines

Evolution sculpts both the body plans and nervous systems of agents together over time. In contrast, in AI and robotics, a robot's body plan is usually designed by hand, and control policies are then optimized for that fixed design. The task of simultaneously co-optimizing the morphology and controller of an embodied robot has remained a challenge. In psychology, the theory of embodied cognition posits that behavior arises from a close coupling between body plan and sensorimotor control, which suggests why co-optimizing these two subsystems is so difficult: most evolutionary changes to morphology tend to adversely impact sensorimotor control, leading to an overall decrease in behavioral performance. Here, we further examine this hypothesis and demonstrate a technique for "morphological innovation protection", which temporarily reduces selection pressure on recently morphologically-changed individuals, thus enabling evolution some time to "readapt" to the new morphology with subsequent control policy mutations. We show the potential for this method to avoid local optima and converge to similar highly fit morphologies across widely varying initial conditions, while sustaining fitness improvements further into optimization. While this technique is admittedly only the first of many steps that must be taken to achieve scalable optimization of embodied machines, we hope that theoretical insight into the cause of evolutionary stagnation in current methods will help to enable the automation of robot design and behavioral training -- while simultaneously providing a testbed to investigate the theory of embodied cognition

A Minimal Developmental Model Can Increase Evolvability in Soft Robots

Different subsystems of organisms adapt over many time scales, such as rapid changes in the nervous system (learning), slower morphological and neurological change over the lifetime of the organism (postnatal development), and change over many generations (evolution). Much work has focused on instantiating learning or evolution in robots, but relatively little on development. Although many theories have been forwarded as to how development can aid evolution, it is difficult to isolate each such proposed mechanism. Thus, here we introduce a minimal yet embodied model of development: the body of the robot changes over its lifetime, yet growth is not influenced by the environment. We show that even this simple developmental model confers evolvability because it allows evolution to sweep over a larger range of body plans than an equivalent non-developmental system, and subsequent heterochronic mutations 'lock in' this body plan in more morphologically-static descendants. Future work will involve gradually complexifying the developmental model to determine when and how such added complexity increases evolvability

Unshackling evolution: evolving soft robots with multiple materials and a powerful generative encoding

In 1994 Karl Sims showed that computational evolution can produce interesting morphologies that resemble natural organisms. Despite nearly two decades of work since, evolved morphologies are not obviously more complex or natural, and the field seems to have hit a complexity ceiling. One hypothesis for the lack of increased complexity is that most work, including Sims’, evolves morphologies composed of rigid elements, such as solid cubes and cylinders, limiting the design space. A second hypothesis is that the encodings of previous work have been overly regular, not allowing complex regularities with variation. Here we test both hypotheses by evolving soft robots with multiple materials and a powerful generative encoding called a compositional pattern-producing network (CPPN). Robots are selected for locomotion speed. We find that CPPNs evolve faster robots than a direct encoding and that the CPPN morphologies appear more natural. We also find that locomotion performance increases as more materials are added, that diversity of form and behavior can be increased with di↵erent cost functions without stifling performance, and that organisms can be evolved at di↵erent levels of resolution. These findings suggest the ability of generative soft-voxel systems to scale towards evolving a large diversity of complex, natural, multi-material creatures. Our results suggest that future work that combines the evolution of CPPNencoded soft, multi-material robots with modern diversityencouraging techniques could finally enable the creation of creatures far more complex and interesting than those produced by Sims nearly twenty years ago

Material properties affect evolution's ability to exploit morphological computation in growing soft-bodied creatures

The concept of morphological computation holds that the body of an agent can, under certain circumstances, exploit the interaction with the environment to achieve useful behavior, potentially reducing the computational burden of the brain/controller. The conditions under which such phenomenon arises are, however, unclear. We hypothesize that morphological computation will be facilitated by body plans with appropriate geometric, material, and growth properties, while it will be hindered by other body plans in which one or more of these three properties is not well suited to the task. We test this by evolving the geometries and growth processes of soft robots, with either manually-set softer or stiffer material properties. Results support our hypothesis: we find that for the task investigated, evolved softer robots achieve better performances with simpler growth processes than evolved stiffer ones. We hold that the softer robots succeed because they are better able to exploit morphological computation. This four-way interaction among geometry, growth, material properties and morphological computation is but one example phenomenon that can be investigated using the system here introduced, that could enable future studies on the evolution and development of generic soft-bodied creatures

Towards Multi-Morphology Controllers with Diversity and Knowledge Distillation

Finding controllers that perform well across multiple morphologies is an important milestone for large-scale robotics, in line with recent advances via foundation models in other areas of machine learning. However, the challenges of learning a single controller to control multiple morphologies make the `one robot one task' paradigm dominant in the field. To alleviate these challenges, we present a pipeline that: (1) leverages Quality Diversity algorithms like MAP-Elites to create a dataset of many single-task/single-morphology teacher controllers, then (2) distills those diverse controllers into a single multi-morphology controller that performs well across many different body plans by mimicking the sensory-action patterns of the teacher controllers via supervised learning. The distilled controller scales well with the number of teachers/morphologies and shows emergent properties. It generalizes to unseen morphologies in a zero-shot manner, providing robustness to morphological perturbations and instant damage recovery. Lastly, the distilled controller is also independent of the teacher controllers -- we can distill the teacher's knowledge into any controller model, making our approach synergistic with architectural improvements and existing training algorithms for teacher controllers.Comment: Accepted at the Genetic and Evolutionary Computation Conference 2024 Evolutionary Machine Learning track as a full pape

No-brainer: Morphological Computation driven Adaptive Behavior in Soft Robots

It is prevalent in contemporary AI and robotics to separately postulate a brain modeled by neural networks and employ it to learn intelligent and adaptive behavior. While this method has worked very well for many types of tasks, it isn\u27t the only type of intelligence that exists in nature. In this work, we study the ways in which intelligent behavior can be created without a separate and explicit brain for robot control, but rather solely as a result of the computation occurring within the physical body of a robot. Specifically, we show that adaptive and complex behavior can be created in voxel-based virtual soft robots by using simple reactive materials that actively change the shape of the robot, and thus its behavior, under different environmental cues. We demonstrate a proof of concept for the idea of closed-loop morphological computation, and show that in our implementation, it enables behavior mimicking logic gates, enabling us to demonstrate how such behaviors may be combined to build up more complex collective behaviors.Accepted to the From Animals to Animats: 17th International Conference on the Simulation of Adaptive Behavior (SAB 2024) conferenc

Evolutionary Developmental Soft Robotics As a Framework to Study Intelligence and Adaptive Behavior in Animals and Plants

In this paper, a comprehensive methodology and simulation framework will be reviewed, designed in order to study the emergence of adaptive and intelligent behavior in generic soft-bodied creatures. By incorporating artificial evolutionary and developmental processes, the system allows to evolve complete creatures (brain, body, developmental properties, sensory, control system, etc.) for different task environments. Whether the evolved creatures will resemble animals or plants is in general not known a priori, and depends on the specific task environment set up by the experimenter. In this regard, the system may offer a unique opportunity to explore differences and similarities between these two worlds. Different material properties can be simulated and optimized, from a continuum of soft/stiff materials, to the interconnection of heterogeneous structures, both found in animals and plants alike. The adopted genetic encoding and simulation environment are particularly suitable in order to evolve distributed sensory and control systems, which play a particularly important role in plants. After a general description of the system some case studies will be presented, focusing on the emergent properties of the evolved creatures. Particular emphasis will be on some unifying concepts that are thought to play an important role in the emergence of intelligent and adaptive behavior across both the animal and plant kingdoms, such as morphological computation and morphological developmental plasticity. Overall, with this paper, we hope to draw attention on set of tools, methodologies, ideas and results, which may be relevant to researchers interested in plant-inspired robotics and intelligence

The Genomic Code: The genome instantiates a generative model of the organism

How does the genome encode the form of the organism? What is the nature of this genomic code? Common metaphors, such as a blueprint or program, fail to capture the complex, indirect, and evolutionarily dynamic relationship between the genome and organismal form, or the constructive, interactive processes that produce it. Such metaphors are also not readily formalised, either to treat empirical data or to simulate genomic encoding of form in silico. Here, we propose a new analogy, inspired by recent work in machine learning and neuroscience: that the genome encodes a generative model of the organism. In this scheme, by analogy with variational autoencoders, the genome does not encode either organismal form or developmental processes directly, but comprises a compressed space of latent variables. These latent variables are the DNA sequences that specify the biochemical properties of encoded proteins and the relative affinities between trans-acting regulatory factors and their target sequence elements. Collectively, these comprise a connectionist network, with weights that get encoded by the learning algorithm of evolution and decoded through the processes of development. The latent variables collectively shape an energy landscape that constrains the self-organising processes of development so as to reliably produce a new individual of a certain type, providing a direct analogy to Waddingtons famous epigenetic landscape. The generative model analogy accounts for the complex, distributed genetic architecture of most traits and the emergent robustness and evolvability of developmental processes. It also provides a new way to explain the independent selectability of specific traits, drawing on the idea of multiplexed disentangled representations observed in artificial and neural systems and lends itself to formalisation.Comment: 31 pages, 4 figure

Evolving soft locomotion in aquatic and terrestrial environments: effects of material properties and environmental transitions

Designing soft robots poses considerable challenges: automated design approaches may be particularly appealing in this field, as they promise to optimize complex multi-material machines with very little or no human intervention. Evolutionary soft robotics is concerned with the application of optimization algorithms inspired by natural evolution in order to let soft robots (both morphologies and controllers) spontaneously evolve within physically-realistic simulated environments, figuring out how to satisfy a set of objectives defined by human designers. In this paper a powerful evolutionary system is put in place in order to perform a broad investigation on the free-form evolution of walking and swimming soft robots in different environments. Three sets of experiments are reported, tackling different aspects of the evolution of soft locomotion. The first two sets explore the effects of different material properties on the evolution of terrestrial and aquatic soft locomotion: particularly, we show how different materials lead to the evolution of different morphologies, behaviors, and energy-performance tradeoffs. It is found that within our simplified physics world stiffer robots evolve more sophisticated and effective gaits and morphologies on land, while softer ones tend to perform better in water. The third set of experiments starts investigating the effect and potential benefits of major environmental transitions (land - water) during evolution. Results provide interesting morphological exaptation phenomena, and point out a potential asymmetry between land-water and water-land transitions: while the first type of transition appears to be detrimental, the second one seems to have some beneficial effects.Comment: 37 pages, 22 figures, currently under review (journal