Bonte, Bert

Boullart, Luc

Wyns, Bart

English

Ghent University Academic Bibliography

EVOLVING ROBUST ROBOT CONTROLLERS FOR CORRIDORFOLLOWING USING GENETIC PROGRAMMINGBart Wyns, Bert Bonte, Luc BoullartDept. of Electrical Energy, Systems and Automation, Ghent University, Technologiepark 913, Zwijnaarde, BelgiumBart.Wyns@UGent.beKeywords: Genetic Programming, Evolutionary Robotics, Corridor Following, EyeBot.Abstract: Designing robots and robot controllers is a highly complex and often expensive task. However, genetic pro-gramming provides an automated design strategy to evolve complex controllers based on evolution in nature.We show that, even with limited computational resources, genetic programming is able to evolve efficientrobot controllers for corridor following in a simulation environment. Therefore, a mixed and gradual formof layered learning is used, resulting in very robust and efficient controllers. Furthermore, the controller issuccessfully applied to real environments as well.1 INTRODUCTIONMany interesting and realistic applications whererobots can be used are too difficult for the currentstate-of-the-art. Robots are mainly used for relativelyeasy and repetitive tasks. Fully autonomous robotsin realistic applications are exceptions (Pollack et al.,2000). This is mainly caused by the highly complexdesign of such systems. More specifically, the devel-opment of robust controllers for real mobile robots ischallenging.The main objective of this contribution is devel-oping robust controllers for corridor following usinggenetic programming (GP), an evolutionary methodbased on program induction. The robot must navigatein a corridor system from start to end as efficiently aspossible and without collisions. The evolved mobilerobot controllers must be robust enough to navigatesuccessfully in corridor systems on which the robotwas trained during the evolutionary process as wellas new and unseen environments. Furthermore, thecontroller evolved in simulation must be transferableto the real robot preserving its behaviour learned insimulation.An example of using GP in a basic simulation en-vironment is found in (Lazarus and Hu, 2001). GPis used for evolving wall following-behaviour, whichis part of many higher level robot skills. Similarexperiments provide some basic proof for using GPin robotics but their practical use is questionable.In (Reynolds, 1994) a simplified but noisy simula-tion environment for evolving corridor following be-haviour with steady-state GP was used. In (Dupuisand Parizeau, 2006), a vision-based line-followingcontroller was evolved in simulation by incrementallyimproving the visual model in the simulation. Mainlycaused by some oversimplifications in the simulator,the authors were not able to successfully transfer thisbehaviour to the physical robot. Some experimentswere conducted directly on real robots. An exampleis in (Nordin and Banzhaf, 1995), where an obstacleavoidance controller for the Khepera-robot is evolved,using steady-state GP. The resulting program was ro-bust, as it was successful in other environments aswell.The remainder of this paper is composed as fol-lows. Section 2 provides a short introduction to theGP specifics used in the experiments. After that, inSection 3 the experimental setup is set out, includ-ing the simulation environment and the robot plat-form. Finally, Section 4 discusses the evolution ofrobot controllers in simulation and the transfer to re-ality.Figure 1: A representative environment for each of the five categories, as used in the standard approach. Each categoryconsists of increasingly difficult environments in terms of the number and type of turns and the length of the optimal path.2 GENETIC PROGRAMMINGDue to page restrictions this section will only describethe various parameter settings used in this study. Amore detailed overview of the GP evolutionary cycleis given in (Nordin and Banzhaf, 1995). 300 individu-als were initially created by the ramped half-and-halfmethod with depth ramps between 2 and 6 and wereallowed to evolve during 50 generations. Crossover(90%) and reproduction (10%) were used in combina-tion with tournament selection with seven individuals.The function set contains two functions: IfLess (arity4), and PROGN2 (arity 2), both well known. Termi-nals to move forward and backward over a distance of10 cm are also included. Whereas in the first experi-ments (Sections 4.1 and 4.2), turn left and right makesthe robot turn 90 degrees in place, further experi-ments (Section 4.3) employ 15 degree turns. Three in-frared distance sensors are used: front, left and right,each perpendicular on the robot. The frontal sensoris placed left of the center of the robot. Finally, threethreshold values are available: low, medium and high,respectively 75, 150 and 300 mm.Each generation all individuals are tested in threeenvironments and can perform 500 movements ineach environment. The fitness function is averagedover all three tested environments and consists ofthree components. A first, basic, component mea-sures the distance in bird’s eye perspective the robothas covered so far. This component mainly differen-tiates between controllers in initial generations. Thesecond component punishes every collision detectedusing either sensor on each side of the robot. Thepenalty consists of adding a fixed value to the num-ber of movements so far. Thirdly, a bonus componentis added when the robot reaches the end of the corri-dor system. This consists of a fixed part and a vari-able part. The variable part is relative to the numberof spare movements and thus rewards efficient con-trollers.While evaluating fitness on a single environmentmost likely leads to brittle strategies, averaging overmultiple fitness cases results in more general solu-tions. Therefore, we use a variant of the layeredlearning approach (Gustafson and Hsu, 2001). Cat-egories of environments with increasing level of diffi-culty (see Figure 1) are interchanged every numberof generations. In a standard setting, each of thefive categories consists of three equally difficult en-vironments. We also construct a mixed setting. Here,two environments from one category and one from aclearly more difficult or easier category are selected,however maintaining the overall increase in difficultytowards the end of a GP run. The gradual approachdoes just the same, but the number of generations thatis used to train on increases, leaving more time for theevolutionary process to learn more difficult behaviour.3 EXPERIMENTAL SETUPWe use the EyeBot-platform for our experiments(Bra¨unl, 2006). The evolutionary process is carriedout in the EyeSim, the simulation environment of theplatform because analysis of robot behaviour in soft-ware is straightforward whereas in reality image pro-cessing would be required. Main advantage is thatprograms for the EyeSim can be transferred immedi-ately for execution on the real EyeBot.Gaussian distributed noise is a good option tomodel realistic errors. The standard deviation we useis 3 for sensor noise and 2 for motor noise. Com-bined with this noise, the simulation environment fa-cilitates the evolution of controllers for real applica-tions, which is impracticable in simplified and naivesimulation environments.The GP process was handled by ECJ 1. ECJ con-structs controllers of which the fitness is evaluated inthe EyeSim and returned back to ECJ, which performsall evolutionary computations. Since GP is a prob-abilistic method, we consider three different runs ofeach experiment. This relatively low number is jus-tified because we are interested in the best controller,not some mean value. Moreover, when consideringtoo many runs, the computational cost becomes toohigh. Finally, the standard deviations for all success-ful runs turn out to be small.1http://www.cs.gmu.edu/ eclab/projects/ecj/4 EVOLVING ROBOTCONTROLLERSWe start with evolving robot controllers in simulationunder relatively simple conditions (Section 4.1). Toincrease realism we then add noise. Firstly, noise isadded to the sensory equipment. Secondly, noise isalso added to the steering mechanism (Section 4.2).To allow adjusting for incomplete turns, we also refinethe terminal set with 15 degree turns instead of 90degree turns. Eventually in Section 4.3 the evolvedcontroller is transfered to the real Eyebot.4.1 Evolution in simulationTable 1 lists the results of the experiments in simula-tion. The best controller of every experiment is ableto navigate through all corridor systems, except in ex-periments 4 and 6, where the controller solves 14 outof 15.Next to the more classic approach of changing thenumber of generations and population size, we willfocus mainly on combining different fitness cases.More precisely, we investigate if and how we can im-prove the resulting controllers and reduce the com-putation time by constructing well-considered cate-gories of training environments.Columns 7 and 8 display the results from the evo-lutionary process. From all three runs, the best per-forming controllers’ fitness on the last category islisted. Remark that the differences in absolute num-bers are small however significant, since a differenceof 0.001 can still be noticed by observation of the con-trollers behaviour. Furthermore, the best controllers’fitness of each run is averaged in the column mean.Columns 9–12 contains results of verification tests.Conclusions concerning the robustness and general-ity of the best controller require verification in newenvironments which were not used during evolution.We considered five environments with the same diffi-culty level of the environments in the last category offitness cases. The number of collisions (too close tothe wall) and the number of moves aggregated over allfive environments are listed. The number of environ-ments (out of five) in which the robot reaches the endin less than 500 moves is indicated as well. Finallythe fitness averaged over all 5 verification environ-ments is found in the last column. Note that this wascalculated in order to make them comparable over allpenalty values.For the noiseless simulations, experiment 1 us-ing the mixed approach clearly yields the best results.Even with significant time reduction compared to ex-periments 2 and 3, this setup results in a more robust(all verification environments are successfully com-pleted) and efficient (small number of movements)controller. Therefore it is very beneficial using mixedcategories containing enough diversity and increas-ingly difficult fitness cases. This diversity enables theevolutionary process to build further on more generalcontrollers.4.2 Preparing for RealityAs stated in literature, noise can improve simulationresults and lead to more robust controllers (Jakobiet al., 1995; Bra¨unl, 2006; Reynolds, 1994). Mainargument is that with noise, evolution is no longerable to exploit coincidences in fitness cases only validin simulation and therefore leads to more robust con-trollers. Indeed, in the second series of experiments,with noise added to the sensor values, results im-proved significantly.Experiment 5 in Table 1 leads to a reasonable con-troller without collisions, though two verification en-vironments were not solved successfully. However,since entropy is still fairly high, further improvementscan be expected. To verify this, in experiment 6 thebest run from experiment 5 was allowed to evolve forsome more generations resulting in a successful con-troller which navigates very robustly and efficientlyin all verification environments. Important remark isthat the mean fitness value from experiment 5 is ex-tremely high, even better than the best results in allother experiments. Therefore the gradual approach isvery interesting for this problem domain since mostruns will lead to excellent controllers.This intermediary setup thus provides sufficientknowledge to move on for a more realistic simula-tion. Next to the noisy sensor values, noise is addedto the steering mechanism as well. Furthermore, turnsof 15 degrees are used instead of 90. This way, thecontroller will be able to navigate smoother and ad-just for incomplete turns, which are very common inreality. This clearly is a scaled up version of the pre-vious problem. Though, by using exactly the samesetup of experiment 5 and augmenting the populationsize and the number of generations, a very efficientand robust solution was evolved. This solution is ableto fulfil, with 4 collisions in total, all tested environ-ments, whether they were used during evolution ornot. An example trail of this controller is depicted inFigure 2(a).An interesting remark is that the underlying strat-egy of nearly all successful controllers of the experi-ments is wall following. Mostly, the left wall is fol-lowed, navigating to the end of the corridor systemwithout turning and proceeding in the wrong direc-Table 1: Results of the first series of experiments. The number of generations and the population size are denoted by Grespectively P, the penalty is referenced by Pen, and collisions is abbreviated to Coll. Experiments 1–4 don’t use the mediumconstant. Experiment 6 is a continued evolution of the best controller from experiment 5 and hence has no mean fitness.Parameters Evolution VerificationNr G P Pen. Fitness c. Best Mean Coll. Moves /5 Fitnessnonoise 1 50 300 1 Mixed 0.90769 0.86914 25 1087 5 0.914022 50 500 1 Standard 0.90933 0.85658 31 1667 3 0.884433 100 200 1 Standard 0.90341 0.85866 126 1356 5 0.89780noisy4 50 300 3 Mixed 0.90922 0.86332 25 1777 2 0.854975 50 300 3 Gradual 0.91174 0.91077 0 1575 3 0.885536 68 300 3 Gradual 0.91234 2 871 5 0.91711tion. This general strategy is a nice example of evolu-tionary processes to come up with simple yet generaland efficient solutions.(a) (b)Figure 2: Best controllor found. (a) The robot trail in simu-lation. (b) The real EyeBot in a test environment. The whiteline denotes the robot trajectory.4.3 Transfer to Real WorldWhen transferring the best controller thus far to real-ity, performance slightly decreased. This was mainlycaused by the fact that the real PSD sensors return in-creasing values when approaching a wall from a cer-tain distance. After increasing the lowest thresholdfrom 75 to 100 (the distance under which the sensorvalues become unreliable), this problem was solved.The robot was able to navigate efficiently throughpreviously unseen environments. The robot success-fully drives straight ahead, adjusts where necessaryand most curves are taken smoothly. Nevertheless wenoted slightly more collisions than in simulation. Fig-ure 2(b) shows the real robot in a test environment.Remark that the left wall following is illustrated byomitting some right walls, yet resulting in a success-ful navigation.5 CONCLUSIONSWe demonstrated that, even with a basic PC and lim-ited computation time, GP is able to evolve controllersfor corridor following in a simulation environment byusing a gradual form of layered learning. Moreover,this controller was transferred successfully to reality.REFERENCESBra¨unl, T. (2006). Embedded Robotics: Mobile RobotDesign and Applications With Embedded Systems.Springer-Verlag, 2nd edition edition.Dupuis, J. and Parizeau, M. (2006). Evolving a Vision-Based Line-Following Robot Controller. In Proceed-ings of the The 3rd Canadian Conference on Com-puter and Robot Vision, page 75. IEEE Computer So-ciety Washington, DC, USA.Gustafson, S. and Hsu, W. (2001). Layered Learning inGenetic Programming for a Cooperative Robot Soc-cer Problem. Lecture Notes in Computer Science,2038:291–301.Jakobi, N., Husbands, P., and Harvey, I. (1995). Noiseand the reality gap: The use of simulation in evolu-tionary robotics. Lecture Notes in Computer Science,929:704–720.Lazarus, C. and Hu, H. (2001). Using Genetic Program-ming to Evolve Robot Behaviours. In Proceedingsof the 3rd British Conference on Autonomous MobileRobotics & Autonomous Systems.Nordin, P. and Banzhaf, W. (1995). Genetic programmingcontrolling a miniature robot. In Working Notes forthe AAAI Symposium on Genetic Programming, pages61–67.Pollack, J., Lipson, H., Ficici, S., Funes, P., Hornby, G., andWatson, R. (2000). Evolutionary techniques in phys-ical robotics. In Evolvable Systems: From Biologyto Hardware: Third International Conference, ICES2000, Edinburgh, Scotland, UK, April 17-19, 2000:Proceedings. Springer.Reynolds, C. (1994). Evolution of corridor following be-havior in a noisy world. In Cliff, D., Husbands,P., Meyer, J.-A., and Wilson, S., editors, From Ani-mals to Animats 3: Proceedings of the third Interna-tional Conference on Simulation of Adaptive Behav-ior, pages 402–410. MIT Press.

Evolving robust robot controllers for corridor following using genetic programming

https://biblio.ugent.be/publication/978399/file/985375.pdf

Evolving robust robot controllers for corridor following using genetic programming

Abstract

Similar works

Full text

Available Versions

Ghent University Academic Bibliography