Traditional sources of information for image-based computer vision algorithms have been points, lines, corners, and recently SIFT features (Lowe, 2004), which seem to represent at present the state of the art in feature definition. Alternatively, the present work explores the possibility of using tracked contours as informative features, especially in applications no

B. Tordoff

E. Martínez

G. Alenyà

J. Koenderink

J.A. Fayman

L.S. Shapiro

M. Li

English

Traditional sources of information for image-based computer vision algorithms have been points, lines, corners, and recently SIFT features (Lowe, 2004), which seem to represent at present the state of the art in feature definition. Alternatively, the present work explores the possibility of using tracked contours as informative features.Peer Reviewe

Alenyà Ribas, Guillem

Torras, Carme

UPCommons. Portal del coneixement obert de la UPC

Robot egomotion from the deformation of active contours

Guillem Alenya

Carme Torras

Crossref

Robot Egomotion from the Deformation of Active Contours

CiteSeerX

IntechOpen

The final publication is available at link.springer.comZoom control has not received the attention one would expect in view of how it enriches the competences of a vision system. The possibility of changing the size of object projections not only permits analysing objects at a higher resolution, but it also may improve tracking and, therefore, subsequent 3D motion estimation and reconstruction results. Of further interest to us, zoom control enables much larger camera motions, while fixating on the same target, than it would be possible with fixed focal length cameras.This work is partially funded by the EU PACO-PLUS project FP6-2004-IST- 4-27657. The authors thank Gabriel Pi for their contribution in preparing the experiments.Peer ReviewedPostprint (author's final draft

Zoom control to compensate cameratranslation within a robot egomotionestimation approachGuillem Alenyà and Carme TorrasInstitut de Robòtica i Informàtica Industrial (CSIC-UPC), Llorens i Artigas4-6, 08028 Barcelona {galenya,torras}@iri.upc.eduSummary. We previously proposed a method to estimate robot egomotionfrom the deformation of a contour in the images acquired by a robot-mountedcamera [2, 1]. The fact that the contour should always be viewed under weak-perspective conditions limits the applicability of the method. In this paper,we overcome this limitation by controlling the zoom so as to compensate forrobot translation along the optic axis. Our control entails minimizing an errorsignal derived directly from image measurements, without requiring any 3Dinformation. Moreover, contrarily to other 2D control approaches, no pointcorrespondences are needed, since a parametric measure of contour deforma-tion suffices. As a further advantage, the error signal is obtained as a byprod-uct of egomotion estimation and, therefore, it does not introduce any burdenin the computation. Experimental results validate this zooming extension tothe method. Moreover, robot translations are correctly computed, includingthose along the optic axis.1 IntroductionZoom control has not received the attention one would expect in viewof how it enriches the competences of a vision system. The possibilityof changing the size of object projections not only permits analysingobjects at a higher resolution, but it also may improve tracking and,therefore, subsequent 3D motion estimation and reconstruction results.Of further interest to us, zoom control enables much larger cameramotions, while fixating on the same target, than it would be possiblewith fixed focal length cameras.Automating zoom control is, thus, a very promising option for visionsystems in general, and robotic applications in particular. One suchapplication, container transfer within a warehouse, where the trajectory2 Guillem Alenyà and Carme Torrasof a mobile robot needs to be traced with low precision demands butwithout any presetting of the environment, has motivated our work [1].We devised a method to estimate robot egomotion from the image flowcaptured by an on-board camera. Following the works of Blake [3] andMart́ınez and Torras [9, 10], instead of using point correspondences, wecodify the contour deformation of a selected target in the image withan affine shape vector.The two main limitations of the method are that, all along the robottrajectory, the target must be kept visible and it should be viewed un-der weak-perspective conditions (i.e., the depth variation of the targetshould be small compared to its distance to the camera). Note that,for a robot vehicle such as that of the warehouse, this reduces the setof possible motions almost to just looming and receding. The formerlimitation can be overcome by mounting the camera on a pan-and-tiltdevice, while the latter calls for automating zoom control to compen-sate translation along the optic axis, as addressed in this work.There are a few papers presenting different strategies for zoom con-trol. Fayman et. al. [4] consider a planar target and robot translationsonly along the optic axis. In order to keep a constant-sized image pro-jection of the target, they propose a technique, named ”zoom tracking”,aimed at preserving the ratio f/Z. A thick-lens camera model and fullcalibration is assumed. Tordoff and Murray [12] address also the prob-lem of fixating the target size in the image, but considering generalrobot motion, and perspective and affine camera models. With theperspective model, only the case of pure rotating cameras is tackled, asthe algorithm needs continuous auto-calibration This algorithm reliesalso on preserving the ratio f/Z. The authors report some problemsfor planar targets, far ones, and in situations where perspective effectsare not present or discrete, as common in broadcast or surveillance.We have been investigating the potential of the affine shape repre-sentation of the deformation induced by camera motion on an activecontour in the image plane [1]. From this representation, egomotioncan be recovered, even in the presence of zooming, as will be presentedin Sect. 2. In Sect. 3 and 4 our method to recover affine scale and gen-erate zoom demands is introduced. Experimental results are presentedin Sect. 5 and finally some conclusions are collected in Sect. 6.2 Mapping Contour Deformations to Camera MotionsThe motion of a robot carrying a camera induces changes in the imagedue to changes in viewpoint. Under weak-perspective conditions, everyZoom control to compensate camera translation... 33D motion of the camera results in an affine deformation within theimage of the target in the scene. The affinity relating two views isusually computed from a set of point matches [7, 11]. Unfortunately,point matching can be computationally very costly, it being still one ofthe key bottlenecks in computer vision. Instead, in this work we explorethe possibility of using an active contour [3] fitted to a target object.The contour, coded as a B-spline [5], deforms between views leading tochanges in the location of the control points.It has been formerly demonstrated [3, 9, 10] that the difference interms of control points Q′ − Q that quantifies the deformation of thecontour can be written as a linear combination of six vectors. Usingmatrix notationQ′ − Q = WS (1)whereW =([10],[01],[Qx0],[0Qy],[0Qx],[Qy0])(2)and S is a vector with the six coefficients of the linear combination.This so-called shape vectorS = [tx, ty, M1,1 − 1, M2,2 − 1, M2,1, M1,2] (3)encodes the affinity between two views d′(s) and d(s) of the planarcontour:d′(s) = Md(s) + t, (4)where M = [Mi,j ] and t = (tx, ty) are, respectively, the matrix andvector defining the affinity in the plane.The contour is tracked along the image sequence with a Kalmanfilter [3] and, for each frame, the shape vector and its associated covari-ance matrix are updated. Considering a camera that possibly changesthe focal length, the affinity coded by the shape vector relates to the3D camera motion in the following way [10]:M =fif0Z0Z0 + Tz[R11 R21R21 R22], (5)t =fiZ0 + Tz[TxTy]+[u0 − uiv0 − vi], (6)where Rij are the elements of the 3D rotation matrix R, Ti are theelements of the 3D translation vector T, Z0 is the distance from theviewed object to the camera in the initial position, f0 is the focal lengthat the initial frame, fi is the current focal length, (u0, v0) is the principal4 Guillem Alenyà and Carme Torraspoint position at the initial frame and (u1, v1) is its current position.Using (5) and (6) the deformation of the contour parameterized as aplanar affinity permits deriving the camera motion in 3D space. Inparticular, the scaled translation in direction Z is calculated as [10]TzZ0=fif01√λ1− 1, (7)where λ1 is the largest eigenvalue of the matrix MMT . Note that, tosimplify the derivation, the reference system has been assumed to becentered on the object.3 Generating Zoom DemandsImage-based 2D methods rely solely on image measurements. The effectof zooming by a factor fi/f0 is to translate the image point u along aline going from the principal point u0 to the point x′ = fif0u+(1− fif0)u0.At practical effects, this can be explained as multiplying the calibrationmatrix corresponding to the first frame by the factor fi/f0. Assuming aunit aspect ratio, the scale s of the affinity that relates two views can berecovered from the affine fundamental matrix FA [6]. Traditionally, ithas been estimated from image point correspondences, as the singularvector N = (a, b, c, d)T corresponding to the smallest singular value ofa matrix constructed with the normalized point correspondences. Atleast 4 non-coplanar point correspondences are needed.Instead, with the affinity representation we have introduced, we canestimate the current scale of the affine deformation in relation to theinitial contour as a function of the largest singular value λ1 in the SVDdecomposition of MMT . We propose to usee =1√λ1− 1. (8)as error function in the zoom control algorithm. It is not directly theaffine scale but it is linearly dependent on the related homothecy. Ob-serve that, in the estimation of robot egomotion1 this error value isalready computed, and so, no overcost is added to our process. Butwe have now the possibility of taking advantage of a zooming camera.1 Compared with [12], just motion is recovered (coded as an affine shape deforma-tion), not structure or 3D reprojection, and no foregroung/background extractionis performed, as comes by the definition of the active contour in the tracking al-gorithm.Zoom control to compensate camera translation... 5Note also that e corresponds to the scaled translation (7) with the ratiovalue between focal lengths equal to 1. This is effectively a 2D mea-sure. Changing the focal length values with this error function neithercalibration is needed nor estimation of the initial distance Z0. Further-more, as noticed by Tordoff and Murray [12], the idea of recovering aproperty of the overall projection instead of a property of the individ-ual points is an advantadge in noisy conditions. This is just what wereach with the weak-perspective camera model and the introduction ofthe affine shape space.4 Control and Egomotion AlgorithmA zoom control algorithm has been designed to drive the zoom lenseswith the proposed error function. In our tests, the velocity-controlledalgorithm didn’t provide any advantage, as the Visca protocol imple-mented in the camera only provides a few possible velocities, and thisintroduces instabilities in the control algorithm. As the precision re-quirements in terms of translation compensation are not very strictin our system, a simple proportional position controller proved to beenough. We tuned the controller with a very simple process at the be-ginning of the sequence. After the active contour is initialized, a pulse isinduced in the zoom position controller obtaining the relation betweenthe zoom change and the error computed by the error function. Notethat no camera calibration and no scene information is needed.As we are using an uncalibrated camera, the focal length is unknownand so is the ratio of focal lengths fi/f0. However, we can use theratio of zoom positions, as demanded to the zoom mechanism of thecamera. We assume that a linear function relates focal length and zoomdemand. This is a good approximation when zoom positions are not inthe maximum focal length zone [8]. As a consequence, we will not useextreme zoom positions.When zooming is incorporated to the egomotion algorithm, changesin the sequence of images are obviously produced. As zooming is con-tinuously correcting camera translations, depths in the acquired imagesare always very similar (this was just our objective!). Compared with afixed focal length set-up, the difference between the initial contour usedas template and the current contour after the deformations induced bycamera motion, i.e., the shape vector, are smaller. Ideally, with a veryprecise and very quick zoom control, the parameters of the affinity cod-ifying the depth translations should be nearly zero. From the zoomingfactor fif0introduced, the Tz translation can be recovered.6 Guillem Alenyà and Carme Torras5 Experimental Results(a)(b)(c)(d)(e)(f)Fig. 1. Images illustrating the performed experiment. (a) Robot at the initialposition and (b) robot at the final position, after performing a translation.(c) Detailed image of the target, a common trash can. (d) The initial imageof the trash can with an active contour fitted to it. (e) Image acquired at thefinal position after the zoom has been changed with the proposed zoom controlalgorithm. (f) Image acquired at the final position without zoom changesThe experiment is performed with a Pioneer AT robot (Fig. 1(a)).It has been equipped with a EVI-D31 pan, tilt and zoom camera. Forthis experiment, the pan and tilt are kept fixed at a constant valueand only zoom is controlled. A linear trajectory is performed with therobot approaching the target. The target used is a common cylindricaltrash can (Fig 1(c)). Lines normal to the contour (Fig.1(d)) are searchlines along which the tracking algorithm searches peak gradients, withwhich the shape vector is calculated. While the robot is moving, foreach acquired image the tracking algorithm estimates the affine shapedeformation of the current contour with respect to the initial one, andcomputes the egomotion. At frame rate, in our current implementationat 20 fps, the system is capable to generate a zoom demand to cancelin the image the robot translation. Figure 1(e) shows that the zoomZoom control to compensate camera translation... 70 500 1000 1500−600−400−2000200400600frameTz tracker(a)0 500 1000 1500200300400500600framezoom(b)0 500 1000 1500−2000−1500−1000−5000500frameTz(c)Fig. 2. Results of the experiment entailing zoom position control. (a) Errorfunction in the center, and variance values up and down. (b) Zoom position asdemanded by the algorithm. (c) Reconstructed Tz translation in the center,and variance values up and downcontrol has effectively cancelled the approaching motion. Figure 1(f)shows the resulting image if the zoom control is deactivated. As can beseen, target projection is much bigger, and after a small approachingtranslation the target would project out of the image plane.Figure 2(a) shows the computed error function. Observe that it iscentered at 0 and the errors keep always small. In Fig. 2(b) the zoompositions resulting from the zoom control algorithm are plotted. As atrajectory approaching the target is performed, the camera zooms out,leading to lower zoom values. The recovered scaled translation is plottedin Fig. 2(c). Here the initial distance was set to 3500 mm. The recoveredtranslation is scaled, as typical in monocular vision. If we would liketo obtain metric information the focal length of the camera should beknown. As we are using a zooming camera the relation between thezoom position and the corresponding focal length should be computed.6 Conclusions and Future WorksBased on the deformation of an active contour fitted to a target, wehave shown how to generate zoom control demands that compensate forrobot translation along the optic axis, thus keeping the virtual distancefrom the camera to the target approximately constant. This widens therange of applicability of an algorithm we previously proposed for robotegomotion estimation, in that it permits longer robot translations alongthe optic axis while preserving weak-perspective conditions.We use a measure of the scale of the affinity, leading to an algorithmwhich, as shown in additional experiments not included due to lengthlimitations, is robust to rotations of the robot. No overcost is added8 Guillem Alenyà and Carme Torrasto the image manipulation algorithm, as the proposed error measure isalready computed as a partial result in order to extract the egomotion.We are currently working on incorporating pan-and-tilt control tothe egomotion recovery algorithm. Preliminary results are promisingwhen we use also the shape vector estimation as error function in thepan-and-tilt control algorithm, as done here for the zoom. This willfurther lengthen the robot trajectories that our egomotion estimationalgorithm could handle.Acknowledgement. This work is partially funded by the EU PACO-PLUSproject FP6-2004-IST-4-27657. The authors thank Gabriel Pi for their contri-bution in preparing the experiments.References1. G. Alenyà, J. Escoda, A.B.Mart́ınez, and C. Torras. Using laser and visionto locate a robot in an industrial environment: A practical experience. InProc. IEEE Int. Conf. Robot. Automat., pages 3539–3544, Apr. 2005.2. G. Alenyà, E. Mart́ınez, and C. Torras. Fusing visual and inertial sensingto recover robot egomotion. Journal of Robotic Systems, 21:23–32, 2004.3. A. Blake and M. Isard. Active contours. Springer, 1998.4. Jeffrey A. Fayman, Oded Sudarsky, Ehud Rivlin, and Michael Rudzsky.Zoom tracking and its applications. Machine Vision and Applications,13(1):25 – 37, 2001.5. J. Foley, A. van Dam, S. Feiner, and F. Hughes. Computer Graphics.Principles and Practice. Addison-Wesley Publishing Company, 1996.6. R. Hartley and A. Zisserman. Multiple View Geometry in Computer Vi-sion. Cambridge University Press, 2 edition, 2004.7. J. Koenderink and A. J. van Doorn. Affine structure from motion. J. Opt.Soc. Am. A, 8(2):377–385, 1991.8. M. Li and J.-M. Lavest. Some aspects of zoom-lens camera calibration.IEEE Trans. Pattern Anal. Machine Intell., 18(11):1105–1110, 1996.9. E. Mart́ınez and C. Torras. Qualitative vision for the guidance of leggedrobots in unstructured environments. Pattern Recognition, 34:1585–1599,2001.10. E. Mart́ınez and C. Torras. Contour-based 3d motion recovery whilezooming. Robotics and Autonomous Systems, 44:219–227, 2003.11. L. S. Shapiro, A. Zisserman, and M. Brady. 3D motion recovery via affineepipolar geometry. Int. J. Comput. Vision, 16(2):147–182, 1995.12. Ben Tordoff and David Murray. Reactive control of zoom while fixatingusing perspective and affine cameras. IEEE Trans. Pattern Anal. MachineIntell., 26(1):98–112, January 2004.

Zoom control to compensate camera translation within a robot egomotion estimation approach

Traditional sources of information for image-based computer vision algorithms have been points, lines, corners, and recently SIFT features (Lowe, 2004), which seem to represent at present the state of the art in feature definition. Alternatively, the present work explores the possibility of using tracked contours as informative features.This work was supported by the project 'Perception, action & cognition through learning of object-action complexes.' (4915).This work is partially funded by the EU PACO-PLUS project FP6-2004-IST-4-27657.Peer Reviewe

Alenyà, Guillem

Digital.CSIC

We previously proposed a method to estimate robot egomotion from the deformation of a contour in the images acquired by a robot-mounted camera [2, 1]. The fact that the contour should always be viewed under weak-perspective conditions limits the applicability of the method. In this paper, we overcome this limitation by controlling the zoom so as to compensate for robot translation along the optic axis. Our control entails minimizing an error signal derived directly from image measurements, without requiring any 3D information. Moreover, contrarily to other 2D control approaches, no point correspondences are needed, since a parametric measure of contour deformation suffices. As a further advantage, the error signal is obtained as a byproduct of egomotion estimation and, therefore, it does not introduce any burden in the computation. Experimental results validate this zooming extension to the method. Moreover, robot translations are correctly computed, including those along the optic axis.Peer Reviewe

Zoom control to compensate cameratranslation within a robot egomotionestimation approachGuillem Alenya` and Carme TorrasInstitut de Robo`tica i Informa`tica Industrial (CSIC-UPC), Llorens i Artigas4-6, 08028 Barcelona {galenya,torras}@iri.upc.eduSummary. We previously proposed a method to estimate robot egomotionfrom the deformation of a contour in the images acquired by a robot-mountedcamera [2, 1]. The fact that the contour should always be viewed under weak-perspective conditions limits the applicability of the method. In this paper,we overcome this limitation by controlling the zoom so as to compensate forrobot translation along the optic axis. Our control entails minimizing an errorsignal derived directly from image measurements, without requiring any 3Dinformation. Moreover, contrarily to other 2D control approaches, no pointcorrespondences are needed, since a parametric measure of contour deforma-tion suffices. As a further advantage, the error signal is obtained as a byprod-uct of egomotion estimation and, therefore, it does not introduce any burdenin the computation. Experimental results validate this zooming extension tothe method. Moreover, robot translations are correctly computed, includingthose along the optic axis.1 IntroductionZoom control has not received the attention one would expect in viewof how it enriches the competences of a vision system. The possibilityof changing the size of object projections not only permits analysingobjects at a higher resolution, but it also may improve tracking and,therefore, subsequent 3D motion estimation and reconstruction results.Of further interest to us, zoom control enables much larger cameramotions, while fixating on the same target, than it would be possiblewith fixed focal length cameras.Automating zoom control is, thus, a very promising option for visionsystems in general, and robotic applications in particular. One suchapplication, container transfer within a warehouse, where the trajectory2 Guillem Alenya` and Carme Torrasof a mobile robot needs to be traced with low precision demands butwithout any presetting of the environment, has motivated our work [1].We devised a method to estimate robot egomotion from the image flowcaptured by an on-board camera. Following the works of Blake [3] andMart´ınez and Torras [9, 10], instead of using point correspondences, wecodify the contour deformation of a selected target in the image withan affine shape vector.The two main limitations of the method are that, all along the robottrajectory, the target must be kept visible and it should be viewed un-der weak-perspective conditions (i.e., the depth variation of the targetshould be small compared to its distance to the camera). Note that,for a robot vehicle such as that of the warehouse, this reduces the setof possible motions almost to just looming and receding. The formerlimitation can be overcome by mounting the camera on a pan-and-tiltdevice, while the latter calls for automating zoom control to compen-sate translation along the optic axis, as addressed in this work.There are a few papers presenting different strategies for zoom con-trol. Fayman et. al. [4] consider a planar target and robot translationsonly along the optic axis. In order to keep a constant-sized image pro-jection of the target, they propose a technique, named ”zoom tracking”,aimed at preserving the ratio f/Z. A thick-lens camera model and fullcalibration is assumed. Tordoff and Murray [12] address also the prob-lem of fixating the target size in the image, but considering generalrobot motion, and perspective and affine camera models. With theperspective model, only the case of pure rotating cameras is tackled, asthe algorithm needs continuous auto-calibration This algorithm reliesalso on preserving the ratio f/Z. The authors report some problemsfor planar targets, far ones, and in situations where perspective effectsare not present or discrete, as common in broadcast or surveillance.We have been investigating the potential of the affine shape repre-sentation of the deformation induced by camera motion on an activecontour in the image plane [1]. From this representation, egomotioncan be recovered, even in the presence of zooming, as will be presentedin Sect. 2. In Sect. 3 and 4 our method to recover affine scale and gen-erate zoom demands is introduced. Experimental results are presentedin Sect. 5 and finally some conclusions are collected in Sect. 6.2 Mapping Contour Deformations to Camera MotionsThe motion of a robot carrying a camera induces changes in the imagedue to changes in viewpoint. Under weak-perspective conditions, everyZoom control to compensate camera translation... 33D motion of the camera results in an affine deformation within theimage of the target in the scene. The affinity relating two views isusually computed from a set of point matches [7, 11]. Unfortunately,point matching can be computationally very costly, it being still one ofthe key bottlenecks in computer vision. Instead, in this work we explorethe possibility of using an active contour [3] fitted to a target object.The contour, coded as a B-spline [5], deforms between views leading tochanges in the location of the control points.It has been formerly demonstrated [3, 9, 10] that the difference interms of control points Q′ −Q that quantifies the deformation of thecontour can be written as a linear combination of six vectors. Usingmatrix notationQ′ −Q = WS (1)whereW =([10],[01],[Qx0],[0Qy],[0Qx],[Qy0])(2)and S is a vector with the six coefficients of the linear combination.This so-called shape vectorS = [tx, ty,M1,1 − 1,M2,2 − 1,M2,1,M1,2] (3)encodes the affinity between two views d′(s) and d(s) of the planarcontour:d′(s) = Md(s) + t, (4)where M = [Mi,j ] and t = (tx, ty) are, respectively, the matrix andvector defining the affinity in the plane.The contour is tracked along the image sequence with a Kalmanfilter [3] and, for each frame, the shape vector and its associated covari-ance matrix are updated. Considering a camera that possibly changesthe focal length, the affinity coded by the shape vector relates to the3D camera motion in the following way [10]:M =fif0Z0Z0 + Tz[R11 R21R21 R22], (5)t =fiZ0 + Tz[TxTy]+[u0 − uiv0 − vi], (6)where Rij are the elements of the 3D rotation matrix R, Ti are theelements of the 3D translation vector T, Z0 is the distance from theviewed object to the camera in the initial position, f0 is the focal lengthat the initial frame, fi is the current focal length, (u0, v0) is the principal4 Guillem Alenya` and Carme Torraspoint position at the initial frame and (u1, v1) is its current position.Using (5) and (6) the deformation of the contour parameterized as aplanar affinity permits deriving the camera motion in 3D space. Inparticular, the scaled translation in direction Z is calculated as [10]TzZ0=fif01√λ1− 1, (7)where λ1 is the largest eigenvalue of the matrix MMT . Note that, tosimplify the derivation, the reference system has been assumed to becentered on the object.3 Generating Zoom DemandsImage-based 2D methods rely solely on image measurements. The effectof zooming by a factor fi/f0 is to translate the image point u along aline going from the principal point u0 to the point x′ = fif0u+(1− fif0)u0.At practical effects, this can be explained as multiplying the calibrationmatrix corresponding to the first frame by the factor fi/f0. Assuming aunit aspect ratio, the scale s of the affinity that relates two views can berecovered from the affine fundamental matrix FA [6]. Traditionally, ithas been estimated from image point correspondences, as the singularvector N = (a, b, c, d)T corresponding to the smallest singular value ofa matrix constructed with the normalized point correspondences. Atleast 4 non-coplanar point correspondences are needed.Instead, with the affinity representation we have introduced, we canestimate the current scale of the affine deformation in relation to theinitial contour as a function of the largest singular value λ1 in the SVDdecomposition of MMT . We propose to usee =1√λ1− 1. (8)as error function in the zoom control algorithm. It is not directly theaffine scale but it is linearly dependent on the related homothecy. Ob-serve that, in the estimation of robot egomotion1 this error value isalready computed, and so, no overcost is added to our process. Butwe have now the possibility of taking advantage of a zooming camera.1 Compared with [12], just motion is recovered (coded as an affine shape deforma-tion), not structure or 3D reprojection, and no foregroung/background extractionis performed, as comes by the definition of the active contour in the tracking al-gorithm.Zoom control to compensate camera translation... 5Note also that e corresponds to the scaled translation (7) with the ratiovalue between focal lengths equal to 1. This is effectively a 2D mea-sure. Changing the focal length values with this error function neithercalibration is needed nor estimation of the initial distance Z0. Further-more, as noticed by Tordoff and Murray [12], the idea of recovering aproperty of the overall projection instead of a property of the individ-ual points is an advantadge in noisy conditions. This is just what wereach with the weak-perspective camera model and the introduction ofthe affine shape space.4 Control and Egomotion AlgorithmA zoom control algorithm has been designed to drive the zoom lenseswith the proposed error function. In our tests, the velocity-controlledalgorithm didn’t provide any advantage, as the Visca protocol imple-mented in the camera only provides a few possible velocities, and thisintroduces instabilities in the control algorithm. As the precision re-quirements in terms of translation compensation are not very strictin our system, a simple proportional position controller proved to beenough. We tuned the controller with a very simple process at the be-ginning of the sequence. After the active contour is initialized, a pulse isinduced in the zoom position controller obtaining the relation betweenthe zoom change and the error computed by the error function. Notethat no camera calibration and no scene information is needed.As we are using an uncalibrated camera, the focal length is unknownand so is the ratio of focal lengths fi/f0. However, we can use theratio of zoom positions, as demanded to the zoom mechanism of thecamera. We assume that a linear function relates focal length and zoomdemand. This is a good approximation when zoom positions are not inthe maximum focal length zone [8]. As a consequence, we will not useextreme zoom positions.When zooming is incorporated to the egomotion algorithm, changesin the sequence of images are obviously produced. As zooming is con-tinuously correcting camera translations, depths in the acquired imagesare always very similar (this was just our objective!). Compared with afixed focal length set-up, the difference between the initial contour usedas template and the current contour after the deformations induced bycamera motion, i.e., the shape vector, are smaller. Ideally, with a veryprecise and very quick zoom control, the parameters of the affinity cod-ifying the depth translations should be nearly zero. From the zoomingfactor fif0introduced, the Tz translation can be recovered.6 Guillem Alenya` and Carme Torras5 Experimental Results(a)(b)(c)(d)(e)(f)Fig. 1. Images illustrating the performed experiment. (a) Robot at the initialposition and (b) robot at the final position, after performing a translation.(c) Detailed image of the target, a common trash can. (d) The initial imageof the trash can with an active contour fitted to it. (e) Image acquired at thefinal position after the zoom has been changed with the proposed zoom controlalgorithm. (f) Image acquired at the final position without zoom changesThe experiment is performed with a Pioneer AT robot (Fig. 1(a)).It has been equipped with a EVI-D31 pan, tilt and zoom camera. Forthis experiment, the pan and tilt are kept fixed at a constant valueand only zoom is controlled. A linear trajectory is performed with therobot approaching the target. The target used is a common cylindricaltrash can (Fig 1(c)). Lines normal to the contour (Fig.1(d)) are searchlines along which the tracking algorithm searches peak gradients, withwhich the shape vector is calculated. While the robot is moving, foreach acquired image the tracking algorithm estimates the affine shapedeformation of the current contour with respect to the initial one, andcomputes the egomotion. At frame rate, in our current implementationat 20 fps, the system is capable to generate a zoom demand to cancelin the image the robot translation. Figure 1(e) shows that the zoomZoom control to compensate camera translation... 70 500 1000 1500−600−400−2000200400600frameTz tracker(a)0 500 1000 1500200300400500600framezoom(b)0 500 1000 1500−2000−1500−1000−5000500frameTz(c)Fig. 2. Results of the experiment entailing zoom position control. (a) Errorfunction in the center, and variance values up and down. (b) Zoom position asdemanded by the algorithm. (c) Reconstructed Tz translation in the center,and variance values up and downcontrol has effectively cancelled the approaching motion. Figure 1(f)shows the resulting image if the zoom control is deactivated. As can beseen, target projection is much bigger, and after a small approachingtranslation the target would project out of the image plane.Figure 2(a) shows the computed error function. Observe that it iscentered at 0 and the errors keep always small. In Fig. 2(b) the zoompositions resulting from the zoom control algorithm are plotted. As atrajectory approaching the target is performed, the camera zooms out,leading to lower zoom values. The recovered scaled translation is plottedin Fig. 2(c). Here the initial distance was set to 3500 mm. The recoveredtranslation is scaled, as typical in monocular vision. If we would liketo obtain metric information the focal length of the camera should beknown. As we are using a zooming camera the relation between thezoom position and the corresponding focal length should be computed.6 Conclusions and Future WorksBased on the deformation of an active contour fitted to a target, wehave shown how to generate zoom control demands that compensate forrobot translation along the optic axis, thus keeping the virtual distancefrom the camera to the target approximately constant. This widens therange of applicability of an algorithm we previously proposed for robotegomotion estimation, in that it permits longer robot translations alongthe optic axis while preserving weak-perspective conditions.We use a measure of the scale of the affinity, leading to an algorithmwhich, as shown in additional experiments not included due to lengthlimitations, is robust to rotations of the robot. No overcost is added8 Guillem Alenya` and Carme Torrasto the image manipulation algorithm, as the proposed error measure isalready computed as a partial result in order to extract the egomotion.We are currently working on incorporating pan-and-tilt control tothe egomotion recovery algorithm. Preliminary results are promisingwhen we use also the shape vector estimation as error function in thepan-and-tilt control algorithm, as done here for the zoom. This willfurther lengthen the robot trajectories that our egomotion estimationalgorithm could handle.Acknowledgement. This work is partially funded by the EU PACO-PLUSproject FP6-2004-IST-4-27657. The authors thank Gabriel Pi for their contri-bution in preparing the experiments.References1. G. Alenya`, J. Escoda, A.B.Mart´ınez, and C. Torras. Using laser and visionto locate a robot in an industrial environment: A practical experience. InProc. IEEE Int. Conf. Robot. Automat., pages 3539–3544, Apr. 2005.2. G. Alenya`, E. Mart´ınez, and C. Torras. Fusing visual and inertial sensingto recover robot egomotion. Journal of Robotic Systems, 21:23–32, 2004.3. A. Blake and M. Isard. Active contours. Springer, 1998.4. Jeffrey A. Fayman, Oded Sudarsky, Ehud Rivlin, and Michael Rudzsky.Zoom tracking and its applications. Machine Vision and Applications,13(1):25 – 37, 2001.5. J. Foley, A. van Dam, S. Feiner, and F. Hughes. Computer Graphics.Principles and Practice. Addison-Wesley Publishing Company, 1996.6. R. Hartley and A. Zisserman. Multiple View Geometry in Computer Vi-sion. Cambridge University Press, 2 edition, 2004.7. J. Koenderink and A. J. van Doorn. Affine structure from motion. J. Opt.Soc. Am. A, 8(2):377–385, 1991.8. M. Li and J.-M. Lavest. Some aspects of zoom-lens camera calibration.IEEE Trans. Pattern Anal. Machine Intell., 18(11):1105–1110, 1996.9. E. Mart´ınez and C. Torras. Qualitative vision for the guidance of leggedrobots in unstructured environments. Pattern Recognition, 34:1585–1599,2001.10. E. Mart´ınez and C. Torras. Contour-based 3d motion recovery whilezooming. Robotics and Autonomous Systems, 44:219–227, 2003.11. L. S. Shapiro, A. Zisserman, and M. Brady. 3D motion recovery via affineepipolar geometry. Int. J. Comput. Vision, 16(2):147–182, 1995.12. Ben Tordoff and David Murray. Reactive control of zoom while fixatingusing perspective and affine cameras. IEEE Trans. Pattern Anal. MachineIntell., 26(1):98–112, January 2004.

Presentado al 6th RoMoCo celebrado en 2007 en Polonia.We previously proposed a method to estimate robot egomotion from the deformation of a contour in the images acquired by a robot-mounted camera [2, 1]. The fact that the contour should always be viewed under weak-perspective conditions limits the applicability of the method. In this paper, we overcome this limitation by controlling the zoom so as to compensate for robot translation along the optic axis. Our control entails minimizing an error signal derived directly from image measurements, without requiring any 3D information. Moreover, contrarily to other 2D control approaches, no point correspondences are needed, since a parametric measure of contour deformation suffices. As a further advantage, the error signal is obtained as a byproduct of egomotion estimation and, therefore, it does not introduce any burden in the computation. Experimental results validate this zooming extension to the method. Moreover, robot translations are correctly computed, including those along the optic axis.This work was supported by the project 'Perception, action & cognition through learning of object-action complexes.' (4915).This work is partially funded by the EU PACO-PLUS
project FP6-2004-IST-4-27657.Peer Reviewe

UPCommons

http://www.intechopen.com/chapter/pdf-download/129

Robot Egomotion from the Deformation of Active Contours

Abstract

Similar works

Full text

Available Versions

UPCommons. Portal del coneixement obert de la UPC

Crossref

CiteSeerX

IntechOpen

UPCommons. Portal del coneixement obert de la UPC

Digital.CSIC

UPCommons. Portal del coneixement obert de la UPC

Digital.CSIC

UPCommons