Evolutionary robotics: model or design?

In this paper, I review recent work in evolutionary robotics (ER), and discuss the perspectives and future directions of the field. First, I propose to draw a crisp distinction between studies that exploit ER as a design methodology on the one hand, and studies that instead use ER as a modeling tool to better understand phenomena observed in biology. Such a distinction is not always that obvious in the literature, however. It is my conviction that ER would profit from an explicit commitment to one or the other approach. Indeed, I believe that the constraints imposed by the specific approach would guide the experimental design and the analysis of the results obtained, therefore reducing arbitrary choices and promoting the adoption of principled methods that are common practice in the target domain, be it within engineering or the life sciences. Additionally, this would improve dissemination and the impact of ER studies on other disciplines, leading to the establishment of ER as a valid tool either for design or modeling purposes

Cooperative coevolution of partially heterogeneous multiagent systems

Cooperative coevolution algorithms (CCEAs) facilitate the evolution of heterogeneous, cooperating multiagent systems. Such algorithms are, however, subject to inherent scalability issues, since the number of required evaluations increases with the number of agents. A possible solution is to use partially heterogeneous (hybrid) teams: behaviourally heterogeneous teams composed of homogeneous sub-teams. By having different agents share controllers, the number of coevolving populations in the system is reduced. We propose HybCCEA, an extension of cooperative coevolution to partially heterogeneous multiagent systems. In Hyb-CCEA, both the agent controllers and the team composition are under evolutionary control. During the evolutionary process, we rely on measures of behaviour similarity for the formation of homogeneous sub-teams (merging), and propose a stochastic mechanism to increase heterogeneity (splitting). We evaluate Hyb-CCEA in multiple variants of a simulated herding task, and compare it with a fully heterogeneous CCEA. Our results show that Hyb-CCEA can achieve solutions of similar quality using significantly fewer evaluations, and in most setups, Hyb-CCEA even achieves significantly higher fitness scores than the CCEA. Overall, we show that merging and splitting populations are viable mechanisms for the cooperative coevolution of hybrid teams.info:eu-repo/semantics/publishedVersio

Behavioral Specialization in Embodied Evolutionary Robotics: Why So Difficult?

Embodied evolutionary robotics is an on-line distributed learning method used in collective robotics where robots are facing open environments. This paper focuses on learning behavioral specialization, as defined by robots being able to demonstrate different kind of behaviors at the same time (e.g., division of labor). Using a foraging task with two resources available in limited quantities, we show that behavioral specialization is unlikely to evolve in the general case, unless very specific conditions are met regarding interactions between robots (a very sparse communication network is required) and the expected outcome of specialization (specialization into groups of similar sizes is easier to achieve). We also show that the population size (the larger the better) as well as the selection scheme used (favoring exploration over exploitation) both play important – though not always mandatory – roles. This research sheds light on why existing embodied evolution algorithms are limited with respect to learning efficient division of labor in the general case, i.e., where it is not possible to guess before deployment if behavioral specialization is required or not, and gives directions to overcome current limitations.This work is supported by the European Unions Horizon 2020 research and innovation programme under grant agreement No 640891, and the ERC Advanced Grant EPNet (340828). Part of the experiments presented in this paper were carried out using the Grid’5000 experimental testbed, being developed under the INRIA ALADDIN development action with support from CNRS, RENATER, and several Universities as well as other funding bodies (see https://www.grid5000.fr). The other parts of the simulations have been done in the supercomputer MareNostrum at Barcelona Supercomputing Center – Centro Nacional de Supercomputacion (The Spanish National Supercomputing Center).Peer ReviewedPostprint (published version

Avoiding convergence in cooperative coevolution with novelty search

Cooperative coevolution is an approach for evolving solutions composed of coadapted components. Previous research has shown, however, that cooperative coevolutionary algorithms are biased towards stability: they tend to converge prematurely to equilibrium states, instead of converging to optimal or near-optimal solutions. In single-population evolutionary algorithms, novelty search has been shown capable of avoiding premature convergence to local optima — a pathology similar to convergence to equilibrium states. In this study, we demonstrate how novelty search can be applied to cooperative coevolution by proposing two new algorithms. The first algorithm promotes behavioural novelty at the team level (NS-T), while the second promotes novelty at the individual agent level (NS-I). The proposed algorithms are evaluated in two popular multiagent tasks: predator-prey pursuit and keepaway soccer. An analysis of the explored collaboration space shows that (i) fitnessbased evolution tends to quickly converge to poor equilibrium states, (ii) NS-I almost never reaches any equilibrium state due to constant change in the individual populations, while (iii) NS-T explores a variety of equilibrium states in each evolutionary run and thus significantly outperforms both fitness-based evolution and NS-I.info:eu-repo/semantics/acceptedVersio

Novel approaches to cooperative coevolution of heterogeneous multiagent systems

Tese de doutoramento, Informática (Engenharia Informática), Universidade de Lisboa, Faculdade de Ciências, 2017Heterogeneous multirobot systems are characterised by the morphological and/or behavioural heterogeneity of their constituent robots. These systems have a number of advantages over the more common homogeneous multirobot systems: they can leverage specialisation for increased efficiency, and they can solve tasks that are beyond the reach of any single type of robot, by combining the capabilities of different robots. Manually designing control for heterogeneous systems is a challenging endeavour, since the desired system behaviour has to be decomposed into behavioural rules for the individual robots, in such a way that the team as a whole cooperates and takes advantage of specialisation. Evolutionary robotics is a promising alternative that can be used to automate the synthesis of controllers for multirobot systems, but so far, research in the field has been mostly focused on homogeneous systems, such as swarm robotics systems. Cooperative coevolutionary algorithms (CCEAs) are a type of evolutionary algorithm that facilitate the evolution of control for heterogeneous systems, by working over a decomposition of the problem. In a typical CCEA application, each agent evolves in a separate population, with the evaluation of each agent depending on the cooperation with agents from the other coevolving populations. A CCEA is thus capable of projecting the large search space into multiple smaller, and more manageable, search spaces. Unfortunately, the use of cooperative coevolutionary algorithms is associated with a number of challenges. Previous works have shown that CCEAs are not necessarily attracted to the global optimum, but often converge to mediocre stable states; they can be inefficient when applied to large teams; and they have not yet been demonstrated in real robotic systems, nor in morphologically heterogeneous multirobot systems. In this thesis, we propose novel methods for overcoming the fundamental challenges in cooperative coevolutionary algorithms mentioned above, and study them in multirobot domains: we propose novelty-driven cooperative coevolution, in which premature convergence is avoided by encouraging behavioural novelty; and we propose Hyb-CCEA, an extension of CCEAs that places the team heterogeneity under evolutionary control, significantly improving its scalability with respect to the team size. These two approaches have in common that they take into account the exploration of the behaviour space by the evolutionary process. Besides relying on the fitness function for the evaluation of the candidate solutions, the evolutionary process analyses the behaviour of the evolving agents to improve the effectiveness of the evolutionary search. The ultimate goal of our research is to achieve general methods that can effectively synthesise controllers for heterogeneous multirobot systems, and therefore help to realise the full potential of this type of systems. To this end, we demonstrate the proposed approaches in a variety of multirobot domains used in previous works, and we study the application of CCEAs to new robotics domains, including a morphological heterogeneous system and a real robotic system.Fundação para a Ciência e a Tecnologia (FCT, PEst-OE/EEI/LA0008/2011