This article describes two vision-based behaviors designed for an autonomous legged robot. These behaviors have been designed in a modular way in order to be able to integrate them in an architecture named DSH (Dynamic Schema Hierarchies), which is also briefly described. These behaviors have been tested in office indoor environments and experiments carried out are also described in this paper. The platform used in these experiments carried out are also described in theis paper. The platform used in these experiments has been the Sony AIBO robot. The developed behaviors provide the robot with the follow wall and follow person capabilities, using as main sensor the robot on-board camer

Martín Rico, Francisco

Matellán Olivera, Vicente

Montero, Pablo

Ruiz, Juan V.

English

Leon University (Spain)

Vision based behaviors for a legged robotJuan V. Ruiz, Pablo Montero, Francisco Martín,Vicente MatellánRobotics Lab (GSyC)Universidad Rey Juan Carlos, Móstoles, España{jvicente,pmontero,fmartin,vmo}@gsyc.escet.urjc.esAbstractThis article describes two vision-basedbehaviors designed for an autonomous leggedrobot. These behaviors have been designed ina modular way in order to be able to integratethem in an architecture named DSH (DynamicSchema Hierarchies), which is also brieydescribed. These behaviors have been testedin oce indoor environments and experimentscarried out are also described in this paper.The platform used in these experiments hasbeen the Sony AIBO robot. The developedbehaviors provide the robot with the followwall and follow person capabilities, using asmain sensor the robot on-board camera.1 IntroductionOce environments represent a complexscenario where a robot can perform a lotof useful tasks for humans. This type ofenvironment represents a big challenge forthese robots due its complexity incrementedand the possibility of human interaction. Therobots must be able to perform task as, forexample, getting the mail form the mailboxto the addressee's oce, guiding across theoce to blindly persons, security, etc. Ourmain research interest focuses on developingarchitectures which get fully autonomousbehaviors in robots. In such a way, the robotcould carry out these tasks without humanguidance.Two basic vision-based behaviors arepresented in this work for their integrationwithin the DSH architecture[?]. Thesebehaviors are based in active vision and theyhave been built and tested in the Sony AIBOrobot ERS-210A, shown in gure 1. Thisrobot is equipped with a 352x288 pixel cameralocated in its head, giving color images at 25fps. This robot incorporates a 400MHz MIPSprocessor with 64 MB of main memory. It isalso equipped with a wide variety of sensors,such as an infrared sensor located in its head,an acceleration sensor, a vibration sensor, twoheadphones in both ears, contact sensor ineach leg, in its back and in its head. Theactuators are 20 servos ( 3 in each 4 legs, 3in the ears, 2 in its tail and one in its mouth)and one speaker located in its mouth. Therobot is also equipped with a 802.11b wirelesscard.Figure 1: Sony AIBO ERS-210ABoth behaviors have been designed usingthe DSH architecture. DSH is a ethologicalinspired, behavior based architecture forrobot control which uses dynamic hierarchiesof small schemas to generate autonomousbehavior. DSH architecture is based onthe schema idea from Arbib[?] and ideasfrom ethology for attention and actionselection, taking into account engineering andprogramming issues. Each schema in DSH canbe described as an independent module thatruns concurrently with another ones. Eachschema has several parameters which tune it.It can also be turned on and o. The schemascan be independently designed and they canbe reused by another higher level schemas.This architecture distinguishes betweentwo kinds of schemas: perceptual andmotor. The rst ones process informationthe environment. The information obtainedis used by motor schemas to generatecontrol commands for locomotion modules,or activation signal for other low levelschemas (perceptual or motor) as well as theirmodulation parameters.The hierarchies are dynamically built. Ifa high level motor schema needs use anotherlower lever schema, it activates and setup thelower schema. This low level schema can bemotor or perceptual one, and it can activateanother lower level schemas and so on.Figure 2: Schema hierarchy and pool of SLEPTschemasFigure 2 represents a DSH architecture.Circles represents motor schemas and squaresperception ones. Only one motor schema canbe active in each level. More informationabout DSH can be found in [?].In this work we will use the term behavioras a collection of schemas that collaborates toobtain a well dened goal. The two behaviorswe have implemented are follow wall andfollow person. The follow wall [?] behavior canbe used to navigate between rooms followinga path, or to patrol the environment. Followperson[?] behavior is activated and modulatedin order to follow a person.The main characteristic of the behaviorspresented in this work is that they use acamera as the main sensor.In order to obtain the information requiredto implement the control schema we haveused an HSV segmentation mechanism. Weuse this color space to avoid the noiseproduced by the illumination variations[?]and to obtain a robust method for detectingrelevant information from an image.The application has been developed usingSony Open-R framework1[?] for the AIBOrobot. This framework allows us to developapplications divided into several concurrentmono-thread objects with message-passingbased communication. This framework alsoprovides abstractions for robot sensors andactuators access. It also let us manage thewireless communication.Using this framework, DSH schemas can beimplemented as Open-R objects in a naturalway. Each object can be seen as a concurrentperceptual schema (if it gets information fromthe environment) or a motion schema (if itproduces output for the motion module or itactivates and modules another schemas). Theactivation and modulation is implemented bythe message passing communication.This paper is organized as follows: Insection 2 we will describe the follow wallbehavior. In section 3 will be described thefollow person behavior for human following.In section 4 will be described the experimentscarried out to validate these behaviors.,Finally, the results and the future guidelinesof this work are discussed in section 5.2 Follow wall behaviorThe environment where the robot mustperform its task is made up by corridors androoms. The robot must be able to navigatefrom one concrete room to another, or topatrol the environment.1http://openr.aibo.com/This behavior has been implemented as twoindependent schemas: one perceptual schemaand one control schema.The perceptual schema takes images fromthe camera and extracts useful information(named stimulus), such as the border limitbetween the oor and the wall. This schemawill also be in charge of getting the doorspositions.The control schema receives the stimulusfrom the perception schema and depending onthe mode selected, it can perform two dierenttypes of movements:Following a wall The border line betweenthe oor and the wall is used to determinethe distance to the wall. The robotcorrects its angular velocity for don'tcollide with the wall. If there is twoborder lines, the robot knows it is in frontto a corner and it turns to adjust itstrajectory.Following a path The robot uses the borderline between the oor and the wall andthe door information for following a givenpath. When the robot completes thepath, it stops.In the following sections we will detailthe two schemas developed for this behavior.Perception schema will obtain the informationneeded from the environment. The controlschema will use this information and it willactuate consistently.2.1 Perception schemaThis schema identies two dierent stimulus:doors and the border limit between the oorand the wall. Doors are identied too. Borderbetween the oor and the wall is used todetermine the distance and angle with thewall. This information will be useful to avoidcolliding with the wall.The identication of the doors in theenvironment will be used in the path modeas a basic element.Border detection is made through dierentconsecutive steps. These steps will get thestimulus from the raw image captured by theon-board camera(Figure 3). These steps are:Figure 3: Original image1. To convert the image from RGB (Red,Green, Blue) to HSV (Hue, saturation,Value) color space. The HSV color spaceis more convenient due to its robustnessto illumination variations. This colorspace let us decompose the image in amore convenient way for segmentation.Figure 4 shows the HSV components(represented as three dierent curves)variation from a portion (bottom of thegure) of the original image shown ingure 3.Figure 4: HSV color space2. Then, the normalized image is lteredusing a Sobel convolution lter for gettingborders (Figure 5). Matrix M is used toobtain the horizontal borders.M =12(1 2 10 0 0−1 −2 −1)Figure 5: Image ltered by the Sobel convolutionlter3. Then, the wall and oor pixels areselected from the image. These pixels arethe ones whose H and S values are withina range. This range is calibrated o-linein order to classify the pixel right:Hmin ≤ Hpixel ≤ HmaxSmin ≤ Spixel ≤ SmaxThe V pixel components is not usedbecause this components is the onerelated to the environment lightingconditions. These conditions are notcontrolled.4. The continuous pixels with the sameinclination are grouped getting smallsegments. These segments are groupedagain depending on their inclinationresemblance for getting just one segment.This segment represents the border limitbetween the oor and the wall (Figure 6)The door detection is also performed insimilar way as we do in the border line case.The door pixels are the ones whose H andS values, in HSV color space, are withina precalibrated range. Figure 7 shows anoriginal door image. Figure 8 shows the doordetection (blue pixels) by color segmentation.Figure 6: Stimulus extracted form the imageFigure 7: Original image2.2 Control schemaThe control schema will send its output to theAIBO actuators (its legs) in order to follow thewall without colliding with it.The follow wall behavior can be activated intwo dierent modes:• Following a wall. The follow wallmode will make the robot patrol theenvironment following the walls. Figure9 shows the trajectory of the robot in thisroom.• Following a given path. When activatedin this mode, the robot follows apredetermined path. This path hasto be previously given by the schemathat activates this schema. In theexperiments we have used a GUI, thatwill be described in section 4 for pathspecication. We have developed a basicFigure 8: HSV door segmentationFigure 9: Follow-wall behavior for patrollingway to specify the path. Paths arespecied according to the elements thatthe robot can detect in the environment.For instance, as the robot is able to nddoors, the path could be specied as asequence of doors. Furthermore, If therobot is able to detect corners, the pathwill be specied as a doors and cornerssequence. In gure 10 a path is speciedas a door and corner sequence where therobot must detect two doors and a cornerbefore entering in the third door.3 Follow person behaviorThe follow person behavior will be alsoimplemented by two schemas. Obviously, therelevant stimulus in this case are the personand its relative position to the robot. Therst schema is a perceptive one, that mustdetect the person and its position. The secondFigure 10: Path followed by the robotschema is a control one, that sends its outputto the robotic platform in order to follow theperson at a predetermined distance.Liveness is the essential requirement in thisbehavior. The perception schema must getimages, analyze them and make the processedinformation available to the control schema.If the person moves quickly the robot mustbe able to follow him. If the robot cannotdetermine the person position, it will spin untilit nds again the person.3.1 Perception schemaThe stimulus identied by this schema are theperson and the distance to it. We have usedthe images from the on-board camera and theinfrared sensor to implement it. The robotwill detect the person using the color of herclothes. Infrared sensor is used to obtain thedistance to the person. The robot will try tomantain this distance within a certain value.Color segmentation of the clothes is madeby the robot following these steps:1. Converting images from RGB (Red,Green, Blue) to HSV (Hue, saturation,Value) color space.2. The pixels selected in the original imagep are the ones whose H and S values arewithin a range. This range is representthe color clothes we are selecting:Hmin ≤ Hpixel ≤ HmaxSmin ≤ Spixel ≤ Smax3. A new image p′ is obtained. This imagecontains only the ltered pixels. Themass center C for image p′ is calculatedas follows:C = (Cx, Cy)where,Cx =∑ni=0xinCy =∑ni=0yin}∀(x, y) ∈ p′3.2 Control schemaThis schema uses the stimulus developed bythe perception schema and adjust the velocityof the robot motors. The robot is continuouslytracking the person trying to keep the masscenter C of the person in the center of thecamera. The robot moves its neck for carryingout this tracking.Figure 11: The robot stops if the distance is lessthan 1 meterDepending on the distance to the personand its angle with respect the robot, thisschema sends dierent outputs to the robotactuators:infrared sensor < 1 m The robot stops itslegs, but it continues tracking the personusing its neck (gure 11).infrared sensor >= 1 m The robot movesdepending on the neck angle with respectits body:less than 10o In this case, the robotassumes the person is front ofit and it keeps moving straightforward(gure 12),voutputlin = vlinmaxvoutputrot = 0Figure 12: The robot goes forward if the distanceis more than 1 meter and the angle is less than 10obetween 10o and 50o The person ismoving lightly with respect therobot and it moves and turnsimultaneously (gure 13),voutputlin = vlinmidvoutputrot = vrotmidmore than 50o The person is about tobe lost from tracking and the robotmust spin quickly (gure 14),voutputlin = 0voutputrot = vrotmaxFigure 13: The robot spin and goes forward if thedistance is more than 1 meter and the angle ifbetween 10o and 50oFigure 14: The robot spin if the distance is morethan 1 meter and the angle if bigger than 50o4 ExperimentsThe experiments were oriented to test thebehaviors developed in this work. For an easyanalysis we have developed a GUI for each ofthe behaviors. With these GUIs we can testeach of the behaviors independently beforeintegrating them in the DSH architecture.These GUIs let us also to activate and totune these behaviors. GUIs are executed in aLinux based PC that communicates with therobot wireless using TCP. The user can sendcommands to the robot using these GUIs.The follow-wall behavior GUI (Figure 15)must let the user select between the followingwalls or following path modes. If followingpath is selected, the user can introduce thepath. In this GUI we can also visualize theimage from the camera and the informationextraction results.Figure 15: follow-wall GUIThe follow-person behavior GUI providesan easy method for selecting the colors tofollow. The dierent colors to follow aredened as channels. The user can congure1-8 channels. They dene HSV ranges forcolors. The channels are congured in the GUIby selecting a channel and clicking in a imagepixel. Then, this pixel will be incorporated tothat channel, modifying its HSV ranges.In gure 16 the user has congured twochannels. By selecting the rst channel andclicking in the person t-shirt the user modiesthe HSV lter range for the rst channel. Theltered pixels are shown black in the real timeimage. If the user makes a mistake whenclicking and deteriorate the HSV ranges fora channel, he can observe the results in theimage and he can undo the last selection.This method let us obtain good results insegmentation, as well as a fast calibrationmechanism.The experimentation process are shown insome videos in our site2. These behaviors arecurrently working in our oce environments.The follow-person behavior has been testedwith a big number of colors and underdierent lighting conditions. The follow-wallbehavior has been tested patrolling our oce2http://gsyc.escet.urjc.es/robotica/pfc/pfc-siguepersona-aibo.htmlFigure 16: follow-person GUIenvironments and it has been setup with manydierent paths.The paths were followed rightly underdierent lighting conditions as well as underdierent congurations of opened and closeddoors.5 Conclusions and further workTwo behaviors, follow-wall and follow-personfor a legged robot have been described.They have been successfully implementedcomplying the schema distribution of DSH.Perception and control have been partitionedinto dierent schemas has been validated.These two behaviors have been implementedusing a perceptual schema, which extractedthe relevant stimulus for the behavior, andcontrol one, which use this information todecide the correct output to the locomotionmoduleThe local behaviors developed have used anactive on-board camera. The behaviors areliveness, responding quickly to changes in thestimulus perceived by the perceptual schemas.HSV segmentation has been used in orderto improve the robust against dramaticillumination changes.Further works in this area will also focuson adding new behaviors to these robots andimproving existing implementations. Once thenavigation has been partially solved, we canfocus on the development of others behaviors,like mailman or security agent behavior, couldbe implemented.References[1] Michael A. Arbib and Jim-Shih Liaw.Sensorimotor transformations in theworlds of frogs and robots. ArticialIntelligence, 72(12):5379, 1995.[2] Juan Vicente Ruiz Carrillo.Comportamiento de seguimiento depared con el robot aibo, 2005.[3] J. Chamorro-Martínez, D. Sánchez, andB. Prados-Suarez. A fuzzy colour imagesegmentation applied to robot vision.In VII Online World Conference onSoft Computing in Industrial Applications,2002.[4] Vicente Matellán and Jose M. Cañas.Dynamic schema hierarchies for anautonomous robot, September 25 2002.[5] Pablo Montero Nóvoa. Sistema deseguimiento basado en visión para el robotaibo, 2004.[6] José M. Cañas Plaza. Jerarquía Dinámicade Esquemas para la generación decomportamiento autónomo. PhD thesis,Universidad Politécnica de Madrid,december 2003.[7] Francisco Martín Rico. Open-r: unenfoque práctico. Technical report,Robotic Lab, GSyC, 2005.

Vision based behaviors for a legged robot

https://hispana.mcu.es/es/registros/registro.do?tipoRegistro=MTD&idBib=11057623

Vision based behaviors for a legged robot

Abstract

Similar works

Full text

Available Versions

Leon University (Spain)