Summary. Time-to-collision can be directly measured from a spatio-temporal image sequence obtained from an uncalibrated camera. This it would appear to offer a simple, elegant measurement for use in obstacle avoidance. However, previous techniques for computing time to collision from an optical flow have proven impractical for real applications. This paper present a new approach for computing time to collision (TTC) based on the idea of measuring the rate of change of the &amp;quot;intrinsic scale&amp;quot;. Intrinsic scale is a geometric invariant that is valid at most points in an image, and can be rapidly determined using a multi-resolution pyramid. In this paper we develop the approach and demonstrate its feasibility by comparing the results with range measurements obtained from a laser ranging device on a moving vehicle. Experimental results show that this is a simple method to obtain reliable TTC with a low computational cost. 

Amaury Nègre

Christian Laugier

Christophe Braillon

James L. Crowley

voir basilic : http://emotion.inrialpes.fr/bibemotion/2006/NBCL06/ address: Rio de Janeiro (BR)Time-to-collision can be directly measured from a spatiotemporal image sequence obtained from an uncalibrated camera. This it would appear to offer a simple, elegant measurement for use in obstacle avoidance. However, previous techniques for computing time to collision from an optical flow have proven impractical for real applications. This paper present a new approach for computing time to collision (TTC) based on the idea of measuring the rate of change of the "intrinsic scale". Intrinsic scale is a geometric invariant that is valid at most points in an image, and can be rapidly determined using a multiresolution pyramid. In this paper we develop the approach and demonstrate its feasibility by comparing the results with range measurements obtained from a laser ranging device on a moving vehicle. Experimental results show that this is a simple method to obtain reliable TTC with a low computational cost

Nègre, Amaury

Braillon, Christophe

Crowley, Jim

Laugier, Christian

Hal - Université Grenoble Alpes

Real-time Time-To-Collision from variation of Intrinsic Scale

CiteSeerX

INRIA a CCSD electronic archive server

HAL Id: inria-00182028https://hal.inria.fr/inria-00182028Submitted on 24 Oct 2007HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.Real-time Time-To-Collision from variation of IntrinsicScaleAmaury Nègre, Christophe Braillon, Jim Crowley, Christian LaugierTo cite this version:Amaury Nègre, Christophe Braillon, Jim Crowley, Christian Laugier. Real-time Time-To-Collisionfrom variation of Intrinsic Scale. Proc. of the Int. Symp. on Experimental Robotics, Jul 2006, Riode Janeiro (BR), France. ￿inria-00182028￿Real-time Time-To-Collision from variation ofIntrinsic ScaleAmaury Nègre1, Christophe Braillon1, James L. Crowley1, and Christian Laugier1INRIA, Grenoble, Francefirstname.lastname@inrialpes.frSummary. Time-to-collision can be directly measured from a spatio-temporal image sequenceobtained from an uncalibrated camera. This it would appear to offer a simple, elegant mea-surement for use in obstacle avoidance. However, previous techniques for computing time tocollision from an optical flow have proven impractical for real applications.This paper present a new approach for computing time to collisi n (TTC) based on the ideaof measuring the rate of change of the "intrinsic scale". Intrinsic scale is a geometric invariantthat is valid at most points in an image, and can be rapidly determined using a multi-resolutionpyramid.In this paper we develop the approach and demonstrate its feasibility by comparing theresults with range measurements obtained from a laser ranging device on a moving vehicle.Experimental results show that this is a simple method to obtain reliable TTC with a lowcomputational cost.1 IntroductionThis article takes place in the obstacle avoidance context for autonomous vehicle indynamic environment. In this domain, the estimation of time-to-collision (TTC) iscrucial. This variableτ has been defined by Lee [11] as the ratio of the distancezof the object and the relative speed of approach between the robot and an obstacle.Estimation of TTC has also been proposed as a biological model f r insect navigation[7], bird flight [8] as well as the human "looming" reflex [17].Most of vision based methods to compute TTC use the optical flow. In a firstmethod [3], a focus of expansion (FOE) is estimated andτ is computed from thepixel velocity and the FOE. But this method cannot be appliedif many objects moveindependently (several FOE). It has been also shown thatτ c n be approximated bythe inverse of the optical flow divergence [16], and in [18] the authors show thatτ canbe directly computed from the optical flow and its derivatives when using a log-polarmapping. Unfortunately, those methods require a first step of reliable dense opticalflow estimation, which has proved difficult.In this paper, we propose a simple new approach to direct measur ment of time tocollision. Our approach is based on the observation that thelocal "size" of features in2 Amaury Ǹegre, Christophe Braillon, James L. Crowley, Christian Laugieran image may be directly measured from the scale of extremal points in a Laplacianscale space. This invariant provide a simple, robust measurment of local size thatcan be obtained at most points in an image.A key problem when using intrinsic scale for time to collision is image registra-tion. We describe such a registration for natural interest points as defined in [13], aswell as for natural interest lines [9]. We experimentally measure the precision of TTCobtained from an image sequence taken from a moving vehicle using the distance totargets as provided by a laser scanner.2 Computation of TTC2.1 TTC from variation of sizeTime-to-collision is easily derived from geometrical considerations : let P representsan object in front of a camera such thatP travels toward the camera. We noteF thefocal length of the camera,S the real size ofP , s the size ofP in the image andZthe distance of the object. (see fig. 1).Fig. 1. Model of the camera projection. The objectPo with the sizeS at the distanceDisprojected in the focal planeF to obtainPi with the sizesWe can show that1s=ZS · FSo, the TTC can be expressed only withs and its derivatives :τ = −ZŻ=sṡThe main difficulty is then to compute the object size in the image. Our approachconsists in approximating the size by computing the intrinsic scale from a Laplacianpyramid.Real-time Time-To-Collision from variation of Intrinsic Scale 32.2 Intrinsic-scale-based size estimationTo estimate a characteristic size of an object in the image, we used a Scale Space rep-resentation, introduced by Witkin [19] and Koenderink [10]as a continuous extensionof pyramid-based multi-resolution descriptions of images[4], [2].The scale-space representation of an image,I(x y), is a continuous space ofimagesL(x, y, σ) obtained by convolution with a variable-scale Gaussian,G(σ):L(x, y, σ) = G(σ) ∗ I(x, y)whereG(σ) =12πσ2e−(x2+y2)/2σ2The Laplacian of an image is a circularly symmetric second derivative computedfrom the convolution of derivatives in orthogonal directions. The Laplacian of animage can be computed by convolution with sampled Gaussian derivatives.Lap(x, y, σ) = σ2(∂2G∂x2∗ I(x, y, σ) +∂2G∂y2∗ I(x, y, σ)) (1)As with the Gaussian function, the Laplacian can be used to define a continuousLaplacian scale space by varying scale (sigma) over a range of values. Extremalpoints in a Laplacian scale space have long been known to provide scale invariantfeature points [5] for matching or registering images. Suchpoints, known as "naturalinterest points", have recently become popular for use as key-points for matching andrecognition [13]. Experimental comparison with other key-points has demonstratedthat natural interest points provide reliable scale invariant key-points for registrationand recognition [15].Except degenerate points, one or more intrinsic scales may be measured at anyposition within an image from the values of sigma at which theLaplacian exhibitsa local extrema [12]. This property has been used to define image indexing andrecognition processes that are invariant to camera position [6].For any image position, the intrinsic scale is proportionalof the characteristic sizeof the local image feature. This suggests that Time-To-Collisi n can be estimated withan intrinsic scale. However, for such an approach to work, images must be registered.Such registration can be obtained by detecting and trackinginterest points and lines.2.3 Object detection and tracking2.4 Natural interest pointsAs described in the previous section, extremal points in theLaplacian scale spacecan be used for matching. To detect such points, the method consists in computing aLaplacian sampled scale space. Then, for each level of scale, each pixel is comparedwith its eight space neighbors and its two scale neighbors. Apoint is accepted ifit the Laplacian is greater or less than all of its neighbors.To compute an accurate4 Amaury Ǹegre, Christophe Braillon, James L. Crowley, Christian Laugierestimation of the intrinsic scale, we can approximate the Laplacian profile over ascale with a cubic spline. The zero of the derivate gives us the intrinsic scale. Itis interesting to characterize significance in order to filter such points. To do so,it is possible to consider the Laplacian value and the curvate of the Laplacianprofile. The Laplacian value describes the contrast of the points in the image, and thecurvature of the Laplacian profile characterizes the contrast over scales.Fig. 2. Interest point detection. The size of the circles correspond t the intrinsic scale. Circularobjects as the walker’s body are well detected whereas stretched objects give unstable results.The figure 2 shows the interest points detected in an urban environment. We cansee that interest points are obtained for circular objects as he walker and some partof cars. Nevertheless, stretched objects are not adequately described. The pole at theright is not detected whereas many points are detected on thewall. To resolve thisproblem, Lowe [14] proposes to eliminate all points that correspond to boundaries. Inour case, this solution is not acceptable because most of objects in urban environmentare stretched and would not be detected.2.5 Extension to linesAs seen in the previous section, natural interest points arelocal extrema of theLaplacian over all directions (x, y and scale). If we relax one spatial constraint, it ispossible to detect elongated feature.Three steps are needed to detect such feature and to compute their characteristics.The first step consists in detecting ridge points in the image. Next, a connexityanalysis is computed to separate each feature. The last stepcom utes the first andsecond order moments.Real-time Time-To-Collision from variation of Intrinsic Scale 5Ridge detectionTo extract ridge points at each level, a method has been developed in [9]. For eachpixel of the Gaussian image, the principal curvature direction is computed, and if thepixel is an extremum of the Laplacian image in this direction, this is a ridge.The principal curvatures and their directions are respectiv ly the eigen values andeigen vectors of the Hessian matrix :H =(∂2f∂x2∂2f∂xy∂2f∂xy∂2f∂y2)Where∂2f∂x2 ,∂2f∂xy ,∂2f∂y2 are the second derivates of the Image.The principal curvature direction correspond to the eigen vector associated to thebiggest eigen value. If the Laplacian assumes a local extremum in this direction, thisis a ridge point.(a) (b)Fig. 3. Gaussian image 3(a) and detected ridges 3(b) at the scale leve σ = 4. As we can see,the elongated shape, as the legs or the ground lines are well det cted.The result of this step is a binary image for each scale level,which represents theridges (cf. fig.3).Connexity analysisRidge structure is obtained by connected component analysis. Two neighbor points(over scale and space) have the same label if they are identified as ridge and if thedifference of two local ridge directions at these two pointss inferior to a threshold.Computing momentsTo obtain the mean position, the mean scale and the orientatio of ridge features, wecompute first and second order moments.6 Amaury Ǹegre, Christophe Braillon, James L. Crowley, Christian LaugierLet N be the number of pixels of the feature andXn=1..N =xnynsn the pointswhich form the feature, wheresn is the intrinsic scale.The gravity center is :µ =1NN∑n=1Xn (2)To give more weight to hight contrast points, we have to weight the position bythe absolute value of the Laplacian at this position. So the formula becomes :µ =∑Nn=1 |lap(Xn)|Xn∑Nn=1 |lap(Xn)|(3)Because we work with a pyramid, each stagek of this pyramid is down-sampled,so a ridge point in the stagek is equivalent to two points at the stagek − 1. Theformula becomes :µ =∑Nn=1 2kn |lap(Xn)|Xn∑Nn=1 2kn |lap(Xn)|(4)The second order moments are needed to evaluate the size and the orientation ofthe ridge. The covariance matrixC is :Cij =∑Nn=1 2kn |lap(Xn)|(Xni − µi)(Xnj − µj)∑Nn=1 2kn |lap(Xn)|(5)2.6 Tracking interest linesOur method for tracking object in video is inspired by [1]. Itbegins with a detectionprocess based on ridge detector explained previously, which gives a set of trackedobjects.Then for each frame, a correlation process is needed to associ te new detectedfeature with tracked objects.Ridge similarity• Spacial distance:distance between two gravity centersSprox =‖XA − XB‖sA(6)• Scale distance :distance between two average scalesSscale = | log(sAsB)| (7)Real-time Time-To-Collision from variation of Intrinsic Scale 7• Significance:compare the average Laplacian valuesSlap = | log(lapAlapB)| (8)• Second order moments :Smom =√∑i,j((CA)ij − (CB)ij)2∑i,j(CA)2ij(9)The combined similarity is the weighted sum of (6), (7), (8) and (9)Scomb = cposSpos + cscaleSscale + clapSlap + cmomSmom (10)Wherecpos, cscale, clap andcmom are negative.Fig. 4.Test sequence for ridge tracking : the licence plate of the car is tracked with our method.The rectangle represents the region of interest and the segment at center represents the positionand the direction of the tracked ridgeFor each tracked object, a region of interest is computed using the position andthe size of the object. A similarity score is then computed for each detected object8 Amaury Ǹegre, Christophe Braillon, James L. Crowley, Christian Laugierin the new frame, the object which obtains the highest score is associated with thetracked object and it is updated with the new properties (position, scale, etc.). In thefigure 4, we track the licence plate of the car. We can notice that both the camera andthe car move.3 Experimental resultsTo verify the precision of the TTC obtained by our method, we propose to captureimages from a moving vehicle equipped with a laser range finder and a camera. Inorder to make the comparison possible, the camera is put on the laser.In the test case (cf. fig.5(a)), the target is a white reflectivmark placed in thecamera axe. This mark can be detected in the laser scan using the intensity of thereflected signal. The laser provides precise distances to the bstacle and the Time-To-Collision can be easily computed with the formula :τ(t) = − z(t)∂z∂t(t)To detect the mark in the image (of size 640x480 pixels), we use o r ridge trackerdescribed in the previous section. The tracker is manually initialized, selecting theregion of interest. At each frame, the object is tracked and its properties (position,orientation, scale) are computed. We use a Kalman Filter to improve these values.The time-to-collision is obtained with a windowed linear regr ssion. The globalcomputation time, including Laplacian scale space calculation, ridge extraction andtracking and the computation of time-to-collision is about50 milliseconds in a 3GHzPentium 4 processor.In the test, the approach speed is constant and the distance to th obstacle,z(t),is linear (cf. fig.5(b)). The inverse of the object size in theimage is proportional tothe distance so that the function1s(t) is linear (cf. fig.5(c)). The figure 5(d) shows thedifference between the real TTC obtained by the laser and theTTC computed by ourmethod. This experiment shows that, after a short time of instability due to the lowersize of the seen object in the image, the TTC is well estimated.4 ConclusionThe method described in this paper is a new visual approach toompute the time tocontact from variation of intrinsic scale. To solve the problem of image registrationwe use a method based on natural interest points and an extension for oblong objectswhich requires an extraction and a tracking of ridges. Such an approach gives goodresults on contrasted features. In the shown test, only one object is tracked for moreclarity, but several objects can be tracked at the same time.This approach is interestingas it uses advantages of cameras (low cost, wide field of view,etc.) and offers lowcomputation time, thanks to a pyramidal algorithm.One possible direction to further research will be to generalize the computation ofTTC on more image points in order to obtain more information about the environment,an ideal case would be to compute a continuous field of TTC. Another point in ourReal-time Time-To-Collision from variation of Intrinsic Scale 9(a) Video sequence used to evaluate the precision. 100 200 300 400 500 600 700 800 900 1000 1100 0  5  10  15  20  25  30  35distance (cm)time (s)z(t)(b) Distance measured by the laser rangefinder 0 0.05 0.1 0.15 0.2 0.25 0  5  10  15  20  25  30  351/scaletime (s)1/s(c) Inverse of the scale evaluated by in-trinsic scale 0 10 20 30 40 50 0  5  10  15  20  25  30  35ttc (s)time (s)real ttcmeasured ttc(d) TTC obtained by the laser rangefinder and computed with TTC from in-trinsic scale.Fig. 5.Test with a constant speed. The distance (5(b)) and the inverse size (5(c)) of the obstacleare linear. The time-to collision is shown on (5(d)), after aperiod of initialization, the twocurve are fairly near.approach is the hypothesis that objects move with constant speed. This work usesan approximation with a piecewise linear model, future workwill explore morecomplexes models.References1. L. Bretzner and T. Lindeberg. Feature tracking with automatic selection of spatial scales.Computer Vision and Image Understanding, 71(3):385–392, 1998.10 Amaury Ǹegre, Christophe Braillon, James L. Crowley, Christian Laugier2. P. Burt and E. H Adelson. The laplacian pyramid as a compactimage code. IEEETransactions on Communications, 9(4):532–540, 1983.3. T.A. Camus.Real-time optical flow. PhD thesis, Brown university, Providence, RI 02912,USA, September 1994.4. J. L. Crowley. A representation for visual information. PhD thesis, Carnegie-MellonUniversity, 1981.5. J. L. Crowley and A. C. Parker. A representation for shape based on peaks and ridges inthe difference of low-pass transform.IEEE Transactions on Pattern Analysis and MachineIntelligence, 6(2):156–170, 1984.6. J.L. Crowley D. Hall and V. Colin de Verdière. View invariant object recognition usingcoloured receptive fields.Machine GRAPHICS and VISION, 9(2):341–352, 2000.7. N. Franceschini, J. M. Pichon, and C. Blanes. From insect vision to robot vision.Philo-sophical Transactions : Biological Sciences, 337:283–294, 1992.8. J.J. Gibson.The ecological approach to visual perception.Houghton Mifflin, Boston,1979.9. TRAN Thi Thanh Hai and Augustin Lux. Extraction de caractéristiques locales : Crêteset pics. InRIVF, pages 203–211, 2003.10. Jan J. Koenderink. The structure of images.Biological Cybernetics, 50:363–396, 1984.11. D.N. Lee. A theory of visual control of braking base on information about time-to-collision. Perception, 5:437–459, 1976.12. T. Lindeberg. Feature detection with automatic selection of spatial scales. InIJCV,volume 30, pages 79–116, 1998.13. D. G. Lowe. Object recognition from local scale-invariant feature. InInternationalConference on Computer Vision, pages 1150–1157, 1999.14. D. G. Lowe. Distinctive image features from scale-invariant keypoints. InIJCV, vol-ume 60, pages 91–110, 2004.15. K. Mikolajezyk and C. Schmid. Indexing based on scale invariant interest points. InProceedind of the International Conference on Computer Vision, pages 525–531, 2001.16. R. C. Nelson and J. Aloimonos. Obstacle avoidance using flow field divergence.IEEETrans. Pattern Anal. Mach. Intell., 11(10):1102–1106, 1989.17. W. Schiff, JA. Caviness, and JJ. Gibson. Persistent fearresponses in rhesus monkeys tothe optical stimulus of "looming".Science, 136:982–3, 1962.18. M. Tistarelli and G. Sandini. On the advantages of polar and log-polar mapping for directestimation of time-to-impact from optical flow.IEEE Trans. Pattern Anal. Mach. Intell.,15(4):401–410, 1993.19. Andrew P. Witkin. Scale-space filtering.IJCAI, pages 1019–1023, 1983.

https://hal.inria.fr/inria-00182028/document

Real-time Time-To-Collision from variation of Intrinsic Scale

Abstract

Similar works

Full text

Available Versions

Hal - Université Grenoble Alpes

CiteSeerX

INRIA a CCSD electronic archive server