We elaborate on (future) evolutionary robot systems where morphologies and controllers of real robots are evolved in the real-world. We argue that such systems must contain a learning component where a newborn robot refines its inherited controller to align with its body, which will inevitably be different from its parents

Eiben, A. E.

Hart, Emma

English

We elaborate on (future) evolutionary robot systems where morphologies and controllers of real robots are evolved in the real-world. We argue that such systems must contain a learning component where a newborn robot refines its inherited controller to align with its body, which will inevitably be different from its parents.</p

VU Research Portal

VU Research PortalIf it evolves it needs to learnEiben, A. E.; Hart, Emmapublished inGECCO '202020DOI (link to publisher)10.1145/3377929.3398151document versionPublisher's PDF, also known as Version of recorddocument licenseArticle 25fa Dutch Copyright ActLink to publication in VU Research Portalcitation for published version (APA)Eiben, A. E., & Hart, E. (2020). If it evolves it needs to learn. In GECCO '20: Proceedings of the 2020 Geneticand Evolutionary Computation Conference Companion (pp. 1383-1384). Association for Computing Machinery,Inc. https://doi.org/10.1145/3377929.3398151General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.            • Users may download and print one copy of any publication from the public portal for the purpose of private study or research.            • You may not further distribute the material or use it for any profit-making activity or commercial gain            • You may freely distribute the URL identifying the publication in the public portal ?Take down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.E-mail address:vuresearchportal.ub@vu.nlDownload date: 22. May. 2021If it evolves it needs to learnA.E. EibenVrije Universiteit AmsterdamThe Netherlandsa.e.eiben@vu.nlEmma HartEdinburgh Napier UniversityScotland, UKe.hart@napier.ac.ukABSTRACTWe elaborate on (future) evolutionary robot systems where mor-phologies and controllers of real robots are evolved in the real-world.We argue that such systems must contain a learning componentwhere a newborn robot refines its inherited controller to align withits body, which will inevitably be different from its parents.CCS CONCEPTS• Computer systems organization → Evolutionary robotics;• Theory of computation → Evolutionary algorithms.KEYWORDSEvolutionary robotics, online learning, Lamarckian evolutionACM Reference Format:A.E. Eiben and EmmaHart. 2020. If it evolves it needs to learn. InGenetic andEvolutionary Computation Conference Companion (GECCO ’20 Companion),July 8–12, 2020, Cancún, Mexico. ACM, New York, NY, USA, 2 pages. https://doi.org/10.1145/3377929.33981511 INTRODUCTIONA general definition of Evolutionary Robotics (ER) states that it“aims to apply evolutionary computation techniques to evolve theoverall design or controllers, or both, for real and simulated au-tonomous robots” [11]. Although much of the past work in thefield of Evolutionary Robotics has been limited to evolving robotcontrollers, more recently ER systems in which morphologies andcontrollers of robots evolve simultaneously are beginning to emerge.However, the majority of such systems follow an approach of evolv-ing in simulation followed by construction in the real-world [9].Researchers have even recently demonstrated that is possible touse wet materials to construct by hand a physical incarnation ofan organism evolved in simulation [8], however this is even moretedious than than constructing from hardware. The ‘simulate-then-construct’ approach inevitably runs into the infamous reality-gapproblem [5], calling for systems in which bodies and brains areevolved together in the real-world. With the development of 3D-printing and rapid prototyping technology, this is now becoming aviable option [1, 4, 6, 12].Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from permissions@acm.org.GECCO ’20 Companion, July 8–12, 2020, Cancún, Mexico© 2020 Association for Computing Machinery.ACM ISBN 978-1-4503-7127-8/20/07. . . $15.00https://doi.org/10.1145/3377929.33981512 THREE STAGES OF A ROBOTIC LIFEA general architecture for “evolving robots in real-time and real-space” has been suggested in [3]. A tangible implementation iscaptured by the notion of an EvoSphere, cf. [2],MORPHOGENESISINFANCY2MATURE LIFE1 3survival, mating, learning,task executionlearningloopbirth processConception FertilityDeliveryFigure 1: Generic system architecture for robot evolutionconceptualized by the Triangle of Life, cf. [3].The EvoSphere consists of three components: the Robot Fabrica-tor, the Training Facility, and the Arena where the robots ‘live andwork’. The Robot Fabricator is where new robots are created (mor-phogenesis). The Training Facility is an environment for supervisedlearning during infancy, so individual robots can learn to controltheir –possibly unique– body to acquire basic skills (locomotion,object manipulation) and to perform complex tasks. Including an in-fancy stage mitigates a general problem: the potential that the brainin a newborn robot is sub-optimal for its body. This can manifestin different ways: in the worst-case, this may result in a completemis-match (e.g. the robot has more sensors than there are inputs inthe controllers), in the best-case it may simply require some tuningof the parameters of the brain. The use of indirect representationssuch as CPPN [10] for controllers is likely reduce this mis-match.However, evolving a completely general brain that works in everypossible body is very challenging.To acquire a brain that works well in the inherited body the learn-ing process starts from the inherited brain and improves it until arequired performance level is reached or a predefined learning bud-get is exhausted.1 If a robot reaches the required performance level,it is declared a fertile adult and released into the Arena, otherwiseit is removed from the system and recycled.3 TWO LOOPS OF ADAPTATIONIt is important to note that we have two loops within this evolvingrobot system: the main evolutionary loop that is evolving bodies1Infant learning can partly take place in simulation. However, simulated learningalways needs to be followed by learning / validation on the real robots to close thereality gap.1383GECCO ’20 Companion, July 8–12, 2020, Cancún, Mexico Eiben and Hartand brains and the secondary loop of the learning method. Theprimary/evolutionary loop is shown in green on the Triangle ofLife (Figure 1). It produces robots with new bodies and brains,materialised by the Robot Fabricator. The secondary / learning loopis depicted by the blue circle in Figure 1. It takes place in the bodyof a newborn robot and produces new brains for this body via alearning algorithm. This process starts with the inherited brain andends with what we call the learned brain.In principle, the learning method can be of any type, includingevolutionary algorithms, e.g., neuro-evolution or genetic program-ming. In this case we have two evolutionary loops and the termsfitness or individual will have a different meaning in each. There-fore we define the following terminology conventions for additionalclarity.Inherited body The body of a robot at birth. It is obtained byrecombination of genomes that (either directly or indirectly) definethe bodies of its parents. The body does not change during lifetime.Inherited brain The brain of a robot at birth. It is obtainedby recombination applied to the genomes that (either directly orindirectly) represent the brains of its parents. It forms the staringpoint for the infant learning process in the Training Centre.Learned brain The brain of a robot obtained by the learningprocess in the Training Center. The maturity test is based on be-haviour / performance produced by the learned brain.Task performance Quantity that reflects how well a task isperformed. In a system with multiple tasks there are multiple defi-nitions of task performance. Task performance drives learning inthe Training Centre. If the definition of fitness contains task-relatedmeasures, then task performance will drive evolution as well.Fitness Quantity that determines reproduction probabilities.Drives evolution. Task performance may be part of the fitnessdefinition.The essential difference between task performance and fitness isthat task performance is used to compare different brains in a givenbody thus driving the learning process in the infancy stage. In con-trast, fitness is used to compare different robots when performingselection during mature life. Poor task performance implies thatthe given ‘robot baby’ fails the maturity test and is recycled. Lowfitness in the Arena means that the given robot will not reproducefrequently, thus and its features will not be propagated to futuregenerations. Obviously, system designers may choose to maintaina fitness definition that is (partly) based on task performance.4 DARWINISM AND LAMARCKISMAll robots that make it to themature lifetime period will have passedthe learning stage. This means that they will possess a learned brainthat will be different from their inherited brain. This implies a choicewhen this robot is about to reproduce and conceive a child.• If the inherited brain is used by the reproduction operator,then we have a Darwinian system.• If the learned brain is used by the reproduction operator,then we have a Lamarckian system.2Observe that the fitness of the robot that determines its chancesto reproduce is always based on the behavior induced by the learned2Note that a Lamarckian approach is impossible if an indirect representation is usedfor the brain but the learning acts directly on the brain that is produced from this.brain, regardless of the above. Whether using a Lamarckian systemis advisable is an open question. As of today there are not manystudies of morphologically evolving robot systems with Lamarckianevolution of controllers, but the first results are encouraging [7].5 CONCLUDING REMARKSAdvances in rapid prototyping technology have opened up a newavenue in Evolutionary Robotics that takes place directly in hard-ware, but this also implies new challenges. We note that randomizedreproduction of robot bodies and brains can lead to a mis-matchbetween the body and the brain of newborn robots. We argue thatthe robotic life cycle should include a specific infancy stage where(supervised) learning takes place to mitigate this problem. Thisincreases the chances of success in the stage of maturity and itprevents reproduction of poorly performing robots and thereby itsaves resources. In summary, our main thesis is that when mor-phologies (and not only controllers) of robots are evolvable, thenthe evolutionary process must be augmented with lifetime learning:If it evolves it needs to learn.ACKNOWLEDGEMENTSThis work is funded by EPSRC (ARE project): EP/R03561X,EP/R035733, EP/R035679 , and the Vrije Universiteit Amsterdam.REFERENCES[1] L. Brodbeck, S. Hauser, and F. Iida. 2015. Morphological evolution of physi-cal robots through model-free phenotype development. PloS one 10, 6 (2015),e0128444.[2] A.E. Eiben. 2015. EvoSphere: The World of Robot Evolution. In Proc. of theTheory and Practice of Natural Computing 2015 (LNCS 9477). Springer, 3–19.https://doi.org/10.1007/978-3-319-26841-5_1[3] A.E. Eiben, N. Bredeche, M. Hoogendoorn, J. Stradner, J. Timmis, A.M. Tyrrell,and A. Winfield. 2013. The Triangle of Life: Evolving Robots in Real-time andReal-space. In Proc. of ECAL 2013. MIT Press, 1056–1063.[4] Matthew F Hale, Edgar Buchanan, Alan F Winfield, Jon Timmis, Emma Hart,Agoston E Eiben, Mike Angus, Frank Veenstra, Wei Li, Robert Woolley, et al. 2019.The ARE Robot Fabricator: How to (Re) produce Robots that Can Evolve in theReal World. In The 2019 Conference on Artificial Life. MIT Press, 95–102.[5] Nick Jakobi, Phil Husbands, and Inman Harvey. 1995. Noise and the realitygap: The use of simulation in evolutionary robotics. In European Conference onArtificial Life. Springer, 704–720.[6] Milan Jelisavcic, Matteo De Carlo, Elte Hupkes, Panagiotis Eustratiadis, JakubOrlowski, Evert Haasdijk, Joshua E Auerbach, and Agoston E Eiben. 2017. Real-world evolution of robot morphologies: A proof of concept. Artificial life 23, 2(2017), 206–235.[7] Milan Jelisavcic, Kyrre Glette, Evert Haasdijk, and A. E. Eiben. 2019. LamarckianEvolution of Simulated Modular Robots. Frontiers in Robotics and AI 6 (2019), 9.https://doi.org/10.3389/frobt.2019.00009[8] Sam Kriegman, Douglas Blackiston, Michael Levin, and Josh Bongard. 2020. Ascalable pipeline for designing reconfigurable organisms. Proceedings of theNational Academy of Sciences 117, 4 (2020), 1853–1859. https://doi.org/10.1073/pnas.1910837117 arXiv:https://www.pnas.org/content/117/4/1853.full.pdf[9] Hod Lipson and Jordan B Pollack. 2000. Automatic design and manufacture ofrobotic lifeforms. Nature 406, 6799 (2000), 974.[10] Kenneth O Stanley. 2007. Compositional pattern producing networks: A novelabstraction of development. Genetic programming and evolvable machines 8, 2(2007), 131–162.[11] Patricia A. Vargas, Ezequiel A. Di Paolo, Inman Harvey, and Phil Husbands. 2014.The Horizons of Evolutionary Robotics. The MIT Press.[12] V. Vujovic, A. Rosendo, L. Brodbeck, and F. Iida. 2017. Evolutionary Developmen-tal Robotics: Improving Morphology and Control of Physical Robots. ArtificialLife 23, 2 (2017), 169–185.1384

If it evolves it needs to learn

NARCIS 

http://www.scopus.com/inward/citedby.url?scp=85089748964&partnerID=8YFLogxK

If it evolves it needs to learn

Abstract

Similar works

Full text

Available Versions

VU Research Portal

NARCIS