We present a comparison of several local image descriptors in the context of visual

Simultaneous Localization and Mapping (SLAM). In visual SLAM a set of points in the

environment are extracted from images and used as landmarks. The points are represented

by local descriptors used to resolve the association between landmarks. In this paper, we

study the class separability of several descriptors under changes in viewpoint and scale.

Several experiments were carried out using sequences of images in 2D and 3D scenes

Ballesta, Monica

Gil, Arturo

Martinez Mozos, Oscar

Reinoso, Oscar

English

University of Lincoln Institutional Repository

Local Descriptors for Visual SLAMMo´nica Ballesta∗ Arturo Gil∗ O´scar Mart´ınez Mozos† O´scar Reinoso∗AbstractWe present a comparison of several local image descriptors in the context of visualSimultaneous Localization and Mapping (SLAM). In visual SLAM a set of points in theenvironment are extracted from images and used as landmarks. The points are representedby local descriptors used to resolve the association between landmarks. In this paper, westudy the class separability of several descriptors under changes in viewpoint and scale.Several experiments were carried out using sequences of images in 2D and 3D scenes.1 IntroductionBuilding a map of the environment is a fundamental skill for a mobile robot, since maps arerequired for a series of high level tasks. Typical approaches use range sensors to build mapsin two or three dimensions (e.g. [5, 6] [3, 14]).Recently, the interest on using cameras as the main sensors to build the map has increasedsignificantly. Such approach is denoted as visual SLAM. Typically, approaches using visionapply a feature-based SLAM (e.g, [2, 4, 8]), in which significant points in the environmentare used as landmarks. Two steps can be distinguished in the utilization of visual landmarks:The detection of interest points and the description of the selected points. The first stepinvolves the selection of suitable points in the images that can be used as landmarks. Thepoints should be detected at different distances and viewing angles, since they will be observedby the robot from different poses. In a second step the landmarks are described by a featurevector which is computed using local image information. The descriptor is used to solve thedata association problem: when the robot observes a landmark in the environment, it mustdecide whether the observation corresponds to a previously seen landmark or to a new one.The data association is a fundamental part of the SLAM process, since wrong associations willproduce incorrect maps. In practice, however, the interest points detected in the images arenot very stable, and the matching between different views becomes difficult. In consequence,∗Miguel Herna´ndez University. E-mail:{m.ballesta|arturo.gil|o.reinoso}@umh.es. Supported by theSpanish Government under projects DPI2004-07433-C02-01 and PCT-G54016977-2005.†University of Freiburg. E-mail:omartine@informatik.uni-freiburg.de. Supported by the EU underproject CoSy FP6-004250-IP.1the problem of selecting a suitable interest point detector and descriptor for visual SLAM isstill open. In a previous work [11], we evaluated some interest point detectors to be used aslandmarks in visual SLAM. The Harris corner detector was found to be the most suitable forvisual SLAM applications. In this paper we present a comparison of different interest pointdescriptors using Harris corner detector as point detector.In [10], Mikolajczyk and Schmid evaluated a set of local descriptors using a criterion basedon the number of correct and false matches between pairs of images. Instead, in this workwe concentrate on the variation of the descriptor when viewed from different angles and dis-tances. We apply a pattern recognition approach using validity clustering measurements [13]to estimate how well the descriptors representing the same landmark along a sequence aregrouped in the different descriptor spaces. This measurements will indicate which descriptorhas better separability properties, facilitating the data association. Several experiments havebeen carried out using sequences of real indoor environment images. We believe that theseresults would help the selection of visual landmarks for SLAM applications.2 Visual DescriptorsNext, we list the set of different descriptors that have been evaluated in this study. For all ofthem we compute the descriptors at the local neighborhood of the points detected by Harris.SIFT: The Scale-Invariant Feature Transform (SIFT) detects distinctive key points in imagesand computes a descriptor for them. The algorithm, developed by Lowe, was initiallyused for object recognition tasks [9]. SIFT features are located at maxima and minimaof a difference of Gaussian functions applied in scale space. Next, the descriptors arecomputed based on orientation histograms at a 4x4 subregion around the interest point,resulting in a 128 dimensional vector.SURF: Speeded Up Robust Features (SURF) is a scale and rotation invariant descriptorpresented by Bay et al. [1]. The detection process is based on the Hessian matrix.SURF descriptors are based on sums of 2D Haar wavelet responses, calculated in a 4x4subregion around each interest point. The standard SURF descriptor has a dimensionof 64 and the Extended version (e-SURF) of 128. The u-SURF version is not invariantto rotation and has a dimension of 64.Gray level patch: This method describes each landmark using the gray level values at asubregion around the interest point. This method has been used in [2] as descriptor ofHarris points in a visual SLAM framework.Orientation Histograms: The orientation histograms are computed from the gradient im-age, which represents the gray value variations in the x and y direction. In [7] orientationhistograms are applied for navigation tasks.20 4 6 11 15 200 3 5 8 10 12Figure 1: The upper sequence shows a planar object (a poster) under different viewpoints.The bottom sequence depicts a 3D scene under different scale changes.Zernike Moments: The moment formulation of the Zernike polynomials [15] appears tobe one of the most popular in terms of noise resilience, information redundancy andreconstruction capability. They are are constructed using a set of complex polynomialswhich form a complete orthogonal basis set defined on the unit disc.3 Descriptor EvaluationTo evaluate the stability of the different interest point descriptors under changes in viewpointand scale we track each interest point along different images in a sequence. Examples of se-quences are shown in Fig. 1. The interest points are extracted using Harris corner detector asshown in [11]. To track the points along the different images we have implemented two differ-ent algorithms for 2D and 3D images respectively. In the first case, we used the homographymatrix as in [12]. In the case of 3D images, we have implemented a method that is based onthe fundamental matrix. This method is divided in two steps. First, seven correspondencesbetween each pair of images are selected, which allows to compute a fundamental matrixF . Using the fundamental matrix F we find a set of preliminary correspondences that areused as input for the computation of a second fundamental matrix F ′. In this second step,the fundamental matrix is computed using a RANSAC approach, which results in a moreaccurate matrix F ′, that permits to find the correspondences with more precision.For each tracked interest point p in a sequence of images S = {i1, . . . , iN}, we obtain aset Dp of descriptor vectors Dp = {dp1 , . . . , dpN }. Each descriptor dpn represents the interest3Table 1: J ′3values computed in the viewpoint changing sequencesSequence SIFT SURF e-SURF u-SURF Patch Histogram Zernike2D sequences1 22.90 36.87 34.18 126.63 15.53 2.48 6.392 15.89 39.45 34.00 119.58 9.03 1.83 2.933 10.18 30.49 25.81 118.64 6.06 1.85 2.904 27.24 68.32 57.81 184.06 15.78 2.13 6.545 23.75 27.60 28.32 55.94 13.59 2.02 5.876 13.38 29.45 23.47 67.36 6.83 1.77 3.683D sequences7 5.71 10.70 10.70 35.93 2.59 1.46 2.138 17.62 16.45 18.96 73.23 5.99 1.51 4.339 7.11 7.83 7.65 25.17 3.33 1.72 2.3510 16.44 14.47 16.60 50.58 7.37 1.54 5.5411 6.22 9.60 9.41 30.33 2.76 1.78 2.2512 10.26 9.63 11.13 41.09 4.00 1.43 3.43point p in the image in. The set Dp forms a cluster in the vector space representing theinterest point p in the images along the sequence.In this work, we use the J3 separability criterion [13] to measure the separability of theclusters representing the interest points. This measure is based on two scatter matrices: Swand Sb. Sw is called within-class scatter matrix, and measures the compactness of the clusters.The between-class scatter matrix Sb measures the separability between vectors belonging todifferent clusters. In our case, Sw measures the invariance of the descriptor to viewpointand scale changes, whereas Sb measures the distinctiveness of the points described. The J3criterion is defined as:J3 = trace(S−1w Sm), (1)where Sm is the mixture scatter matrix and is computed as Sm = Sw + Sb. A good descrip-tor has a low value of Sw, since the variability of the vectors describing the same class issmall. Furthermore, it is desirable that vectors describing different points are as distinctiveas possible, resulting in a high value of Sb. In consequence, a suitable descriptor would havea high value of J3. This descriptor would have good results in terms of the data associationproblem, despite of changes in the imaging conditions, such as viewpoint and scale changes.To compare descriptors with different length we use a normalized version J ′3= J3N, where Nis the descriptor length.4 ExperimentsTables 1 and 2 show the results of applying the J ′3criterion to different sequences of 2Dand 3D scenes. The u-SURF descriptor achieves the highest value of separability in 96% ofthe sequences However u-SURF is not rotational invariant. When comparing only rotational4Table 2: J ′3values computed in the scale changing sequencesSequence SIFT SURF e-SURF u-SURF Patch Histogram Zernike2D sequences1 7.10 3.29 2.87 8.82 2.32 1.78 2.152 7.97 6.27 5.89 13.67 2.59 1.51 2.453 9.42 4.47 4.50 13.03 3.45 1.92 2.814 14.09 7.00 9.05 26.89 4.22 1.94 2.705 103.36 17.58 38.58 131.54 27.73 0.87 14.206 4.24 3.51 3.22 8.56 2.81 1.12 2.327 7.34 4.03 4.90 12.71 4.87 1.77 2.738 26.49 5.99 10.62 22.65 12.34 2.89 9.053D sequences9 7.06 10.12 10.24 28.01 4.47 1.70 3.1010 14.48 10.39 14.97 47.48 5.98 1.67 4.5411 8.76 9.18 10.02 24.72 3.47 2.48 3.9512 22.22 15.53 23.09 67.38 8.50 2.15 5.6113 6.28 8.84 10.00 25.56 3.56 1.94 3.0614 17.45 11.10 16.86 42.37 7.37 2.10 5.88invariant descriptors, SURF and e-SURF present similar results. In this case, the computa-tional cost of computing the extended version of SURF is not worthy, since the results are notimproved substantially. E-SURF always outperforms SIFT in changes in viewpoint. However,in scale changes it is only better in 43% of the cases (2D sequences).Taking into account the results of Tables 1 and 2 together with the results of our previouswork [11], we believe that the u-SURF descriptor in combination with the Harris cornerdetector is suitable for the common situation in which a robot explores the environment witha camera that only rotates around the vertical axis.5 ConclusionsWe have performed an evaluation of visual local descriptors to be applied for SLAM tasks.For this purpose, we analyzed each descriptor according to its separability. The results of theexperiments showed the behavior of seven different descriptors under changes in viewpointand scale. We believe that this information will be useful when selecting an interest pointdescriptor as visual landmark for SLAM.References[1] Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. SURF: Speeded up robust features.In European Conference on Computer Vision, 2006.[2] Andrew J. Davison and David W. Murray. Simultaneous localisation and map-buildingusing active vision. IEEE Transactions on Pattern Analysis and Machine Intelligence,2002.5[3] R. Eustice, H. Singh, and J.J. Leonard. Exactly sparse delayed-state filters. In IEEEInt. Conf. on Robotics & Automation, 2005.[4] A. Gil, O. Reinoso, W. Burgard, C. Stachniss, and O. Mart´ınez Mozos. Improving dataassociation in rao-blackwellized visual SLAM. In IEEE/RSJ Int. Conf. on IntelligentRobots & Systems, 2006.[5] G. Grisetti, C. Stachniss, and W. Burgard. Improved techniques for grid mapping withrao-blackwellized particle filters. IEEE Transactions on Robotics, 23(1), 2007.[6] D. Ha¨hnel, W. Burgard, D. Fox, and S. Thrun. An efficient FastSLAM algorithm forgenerating maps of large-scale cyclic environments from raw laser range measurements.In IEEE/RSJ Int. Conf. on Intelligent Robots & Systems, Las Vegas, NV, USA, 2003.[7] J. Kosecka, L. Zhou, P. Barber, and Z. Duric. Qualitative image based localizationin indoor environments. In Proc. of the IEEE Conf. on Computer Vision and PatternRecognition, 2003.[8] J. Little, S. Se, and D.G. Lowe. Global localization using distinctive visual features. InIEEE/RSJ Int. Conf. on Intelligent Robots & Systems, 2002.[9] D.G. Lowe. Object recognition from local scale-invariant features. In Int. Conf. onComputer Vision, 1999.[10] K. Mikolajczyk and C Schmid. A performance evaluation of local descriptors. IEEETransactions on Pattern Analysis and Machine Intelligence, 27(10), 2005.[11] O. Mart´ınez Mozos, A. Gil, M.Ballesta, and O. Reinoso. Interest point detectors for visualslam. In Proc. of the Conference of the Spanish Association for Artificial Intelligence(CAEPIA), 2007.[12] C. Schmid, R. Mohr, and C. Bauckhage. Evaluaton of interest point detectors. Interna-tional Journal of computer Vision, 37(2), 2000.[13] S. Theodoridis and K. Koutroumbas. Pattern Recognition. Academic Press, third edition,2006.[14] R. Triebel and W. Burgard. Improving simultaneous mapping and localization in 3dusing global constraints. In National Conference on Artificial Intelligence (AAAI), 2005.[15] F. Zernike. Diffraction theory of the cut procedure and its improved form, the phasecontrast method. Physica, 1:689–704, 1934.6

Local descriptors for visual SLAM

Abstract

Similar works

Full text

Available Versions

University of Lincoln Institutional Repository