Scalable Co-Optimization of Morphology and Control in Embodied Machines

Evolution sculpts both the body plans and nervous systems of agents together over time. In contrast, in AI and robotics, a robot's body plan is usually designed by hand, and control policies are then optimized for that fixed design. The task of simultaneously co-optimizing the morphology and controller of an embodied robot has remained a challenge. In psychology, the theory of embodied cognition posits that behavior arises from a close coupling between body plan and sensorimotor control, which suggests why co-optimizing these two subsystems is so difficult: most evolutionary changes to morphology tend to adversely impact sensorimotor control, leading to an overall decrease in behavioral performance. Here, we further examine this hypothesis and demonstrate a technique for "morphological innovation protection", which temporarily reduces selection pressure on recently morphologically-changed individuals, thus enabling evolution some time to "readapt" to the new morphology with subsequent control policy mutations. We show the potential for this method to avoid local optima and converge to similar highly fit morphologies across widely varying initial conditions, while sustaining fitness improvements further into optimization. While this technique is admittedly only the first of many steps that must be taken to achieve scalable optimization of embodied machines, we hope that theoretical insight into the cause of evolutionary stagnation in current methods will help to enable the automation of robot design and behavioral training -- while simultaneously providing a testbed to investigate the theory of embodied cognition

Impact of Morphology Variations on Evolved Neural Controllers for Modular Robots

Modular robots, in particular those in which the modules are physically interchangeable, are suitable to be evolved because they allow for many different designs. Moreover, they can constitute ecosystems where “old” robots are disassembled and the resulting modules are composed together, either within an external assembling facility or by self-assembly procedures, to form new robots. However, in practical settings, self-assembly may result in morphologies that are slightly different from the expected ones: this may cause a detrimental misalignment between controller and morphology. Here, we characterize experimentally the robustness of neural controllers for Voxel-based Soft Robots, a kind of modular robots, with respect to small variations in the morphology. We employ evolutionary computation for optimizing the controllers and assess the impact of morphology variations along two axes: kind of morphology and size of the robot. Moreover, we quantify the advantage of performing a re-optimization of the controller for the varied morphology. Our results show that small variations in the morphology are in general detrimental for the performance of the evolved neural controller. Yet, a short re-optimization is often sufficient for aligning back the performance of the modified robot to the original one

Collective control of modular soft robots via embodied Spiking Neural Cellular Automata

Voxel-based Soft Robots (VSRs) are a form of modular soft robots, composed of several deformable cubes, i.e., voxels. Each VSR is thus an ensemble of simple agents, namely the voxels, which must cooperate to give rise to the overall VSR behavior. Within this paradigm, collective intelligence plays a key role in enabling the emerge of coordination, as each voxel is independently controlled, exploiting only the local sensory information together with some knowledge passed from its direct neighbors (distributed or collective control). In this work, we propose a novel form of collective control, influenced by Neural Cellular Automata (NCA) and based on the bio-inspired Spiking Neural Networks: the embodied Spiking NCA (SNCA). We experiment with different variants of SNCA, and find them to be competitive with the state-of-the-art distributed controllers for the task of locomotion. In addition, our findings show significant improvement with respect to the baseline in terms of adaptability to unforeseen environmental changes, which could be a determining factor for physical practicability of VSRs.Comment: Workshop on "From Cells to Societies: Collective Learning across Scales" at the International Conference on Learning Representations (Cells2Societies@ICLR

Evolving Modularity in Soft Robots Through an Embodied and Self-Organizing Neural Controller

Modularity is a desirable property for embodied agents, as it could foster their suitability to different domains by disassembling them into transferable modules that can be reassembled differently. We focus on a class of embodied agents known as voxel-based soft robots (VSRs). They are aggregations of elastic blocks of soft material; as such, their morphologies are intrinsically modular. Nevertheless, controllers used until now for VSRs act as abstract, disembodied processing units: Disassembling such controllers for the purpose of module transferability is a challenging problem. Thus, the full potential of modularity for VSRs still remains untapped. In this work, we propose a novel self-organizing, embodied neural controller for VSRs. We optimize it for a given task and morphology by means of evolutionary computation: While evolving, the controller spreads across the VSR morphology in a way that permits emergence of modularity. We experimentally investigate whether such a controller (i) is effective and (ii) allows tuning of its degree of modularity, and with what kind of impact. To this end, we consider the task of locomotion on rugged terrains and evolve controllers for two morphologies. Our experiments confirm that our self-organizing, embodied controller is indeed effective. Moreover, by mimicking the structural modularity observed in biological neural networks, different levels of modularity can be achieved. Our findings suggest that the self-organization of modularity could be the basis for an automatic pipeline for assembling, disassembling, and reassembling embodied agents

Harnessing the Power of Collective Intelligence: the Case Study of Voxel-based Soft Robots

The field of Evolutionary Robotics (ER) is concerned with the evolution of artificial agents---robots. Albeit groundbreaking, progress in the field has recently stagnated. In the research community, there is a strong feeling that a paradigm change has become necessary to disentangle ER. In particular, a solution has emerged from ideas from Collective Intelligence (CI). In CI---which has many relevant examples in nature---behavior emerges from the interaction between several components. In the absence of central intelligence, collective systems are usually more adaptable. In this thesis, we set out to harness the power of CI, focusing on the case study of simulated Voxel-based Soft Robots (VSRs): they are aggregations of homogeneous and soft cubic blocks that actuate by altering their volume. We investigate two axes. First, the morphologies of VSRs are intrinsically modular and an ideal substrate for CI; nevertheless, controllers employed until now do not take advantage of such modularity. Our results prove that VSRs can truly be controlled by the CI of their modules. Second, we investigate the spatial and time scales of CI. In particular, we evolve a robot to detect its global body properties given only local information processing, and, in a different study, generalize better to unseen environmental conditions through Hebbian learning. We also consider how evolution and learning interact in VSRs. Looking beyond VSRs, we propose a novel soft robot formalism that more closely resembles natural tissues and blends local with global actuation.The field of Evolutionary Robotics (ER) is concerned with the evolution of artificial agents---robots. Albeit groundbreaking, progress in the field has recently stagnated. In the research community, there is a strong feeling that a paradigm change has become necessary to disentangle ER. In particular, a solution has emerged from ideas from Collective Intelligence (CI). In CI---which has many relevant examples in nature---behavior emerges from the interaction between several components. In the absence of central intelligence, collective systems are usually more adaptable. In this thesis, we set out to harness the power of CI, focusing on the case study of simulated Voxel-based Soft Robots (VSRs): they are aggregations of homogeneous and soft cubic blocks that actuate by altering their volume. We investigate two axes. First, the morphologies of VSRs are intrinsically modular and an ideal substrate for CI; nevertheless, controllers employed until now do not take advantage of such modularity. Our results prove that VSRs can truly be controlled by the CI of their modules. Second, we investigate the spatial and time scales of CI. In particular, we evolve a robot to detect its global body properties given only local information processing, and, in a different study, generalize better to unseen environmental conditions through Hebbian learning. We also consider how evolution and learning interact in VSRs. Looking beyond VSRs, we propose a novel soft robot formalism that more closely resembles natural tissues and blends local with global actuation

Morphology Choice Affects the Evolution of Affordance Detection in Robots

A vital component of intelligent action is affordance detection: understanding what actions external objects afford the viewer. This requires the agent to understand the physical nature of the object being viewed, its own physical nature, and the potential relationships possible when they interact. Although robotics researchers have investigated affordance detection, the way in which the morphology of the robot facilitates, obstructs, or otherwise influences the robot’s ability to detect affordances has yet to be studied. We do so here and find that a robot with an appropriate morphology can evolve to predict whether it will fit through an aperture with just minimal tactile feedback. We also find that some robot morphologies facilitate the evolution of more accurate affordance detection, while others do not if all have the same evolutionary optimization budget. This work demonstrates that sensation, thought, and action are necessary but not sufficient for understanding how affordance detection may evolve in organisms or robots: morphology must also be taken into account. It also suggests that, in the future, we may optimize morphology along with control in order to facilitate affordance detection in robots, and thus improve their reliable and safe action in the world

On the Schedule for Morphological Development of Evolved Modular Soft Robots

Development is fundamental for living beings. As robots are often designed to mimic biological organisms, development is believed to be crucial for achieving successful results in robotic agents, as well. What is not clear, though, is the most appropriate scheduling for development. While in real life systems development happens mostly during the initial growth phase of organisms, it has not yet been investigated whether such assumption holds also for artificial creatures. In this paper, we employ a evolutionary approach to optimize the development—according to different representations—of Voxel-based Soft Robots (VSRs), a kind of modular robots. In our study, development consists in the addition of new voxels to the VSR, at fixed time instants, depending on the development schedule. We experiment with different schedules and show that, similarly to living organisms, artificial agents benefit from development occurring at early stages of life more than from development lasting for their entire life

A Fully-distributed Shape-aware Neural Controller for Modular Robots

Modular robots are promising for their versatility and large design freedom. Modularity can also enable automatic assembly and reconfiguration, be it autonomous or via external machinery. However, these procedures are error-prone and often result in misassemblings. This, in turn, can cause catastrophic effects on the robot functionality, as the controller deployed in each module is optimized for a different robot shape than the actual one. In this work, we address such shortcoming by proposing a shape-aware modular controller, operating with (1) a self-discovery phase, in which each module controller identifies the shape it is assembled in, followed by (2) a parameter selection phase, where the controller selects its parameters according to the inferred shape. We deploy a self-classifying neural cellular automaton for phase (1), and we leverage evolutionary optimization for implementing a library of controller parameters for phase (2). We test the validity of the proposed method considering voxel-based soft robots, a class of modular soft robots, and the task of locomotion. Our findings confirm the effectiveness of such a controller paradigm, and also show that it can be used to partially overcome unforeseen damages or assembly mistakes

Evolving modular soft robots without explicit inter-module communication using local self-attention

Modularity in robotics holds great potential. In principle, modular robots can be disassembled and reassembled in different robots, and possibly perform new tasks. Nevertheless, actually exploiting modularity is yet an unsolved problem: controllers usually rely on inter-module communication, a practical requirement that makes modules not perfectly interchangeable and thus limits their flexibility. Here, we focus on Voxel-based Soft Robots (VSRs), aggregations of mechanically identical elastic blocks. We use the same neural controller inside each voxel, but without any inter-voxel communication, hence enabling ideal conditions for modularity: modules are all equal and interchangeable. We optimize the parameters of the neural controller—shared among the voxels—by evolutionary computation. Crucially, we use a local self-attention mechanism inside the controller to overcome the absence of inter-module communication channels, thus enabling our robots to truly be driven by the collective intelligence of their modules. We show experimentally that the evolved robots are effective in the task of locomotion: thanks to self-attention, instances of the same controller embodied in the same robot can focus on different inputs. We also find that the evolved controllers generalize to unseen morphologies, after a short fine-tuning, suggesting that an inductive bias related to the task arises from true modularity