The rank and census are two filters based on order statistics which have been applied to the image matching problem for stereo pairs. Advantages of these filters include their robustness to radiometric distortion and small amounts of random noise, and their amenability to hardware implementation. In this paper, a new matching algorithm is presented, which provides an overall framework for matching, and is used to compare the rank and census techniques with standard matching metrics. The algorithm was tested using both real stereo pairs and a synthetic pair with ground truth. The rank and census filters were shown to significantly improve performance in the case of radiometric distortion. In all cases, the results obtained were comparable to, if not better than, those obtained using standard matching metrics. Furthermore, the rank and census have the additional advantage that their computational overhead is less than these metrics. For all techniques tested, the difference between the results obtained for the synthetic stereo pair, and the ground truth results was small

Banks, Jasmine

English

Queensland University of Technology ePrints Archive

This is the author’s version of a work that was submitted/accepted for pub-lication in the following source:Banks, Jasmine (2001) Order statistic filters for image matching. In Barner,Kenneth & Arce , Gonzalo (Eds.) Proceedings 2001 IEEE-EURASIPWork-shop on Nonlinear Signal and Image Proocessing, University of Delaware,Baltimore, Maryland, USA., Baltimore, Maryland, USA.This file was downloaded from: http://eprints.qut.edu.au/58769/c© Copyright 2001 Please consult the author.Notice: Changes introduced as a result of publishing processes such ascopy-editing and formatting may not be reflected in this document. For adefinitive version of this work, please refer to the published source:ORDER STATISTIC FILTERS FOR IMAGE MATCHINGJasmine E. BanksFraunhofer Institute for Computer Graphis,Rundeturmstr. 6, Darmstadt 64283, Germany.Email: jbanksieee.orgABSTRACTThe rank and ensus are two lters based on orderstatistis whih have been applied to the image math-ing problem for stereo pairs. Advantages of these ltersinlude their robustness to radiometri distortion andsmall amounts of random noise, and their amenabil-ity to hardware implementation. In this paper, a newmathing algorithm is presented, whih provides anoverall framework for mathing, and is used to omparethe rank and ensus tehniques with standard math-ing metris. The algorithm was tested using both realstereo pairs and a syntheti pair with ground truth.The rank and ensus lters were shown to signif-iantly improve performane in the ase of radiomet-ri distortion. In all ases, the results obtained wereomparable to, if not better than, those obtained us-ing standard mathing metris. Furthermore, the rankand ensus have the additional advantage that theiromputational overhead is less than these metris. Forall tehniques tested, the dierene between the resultsobtained for the syntheti stereo pair, and the groundtruth results was small.1. INTRODUCTIONStereo vision is one tehnique for pereption of the 3Denvironment, in whih two or more images of a seneare taken from dierent perspetives, and depth in-formation is obtained by triangulating orrespondingpoints in the images. A fundamental problem faed bystereo vision algorithms is that of loating orrespond-ing points in the images. This is known as the imagemathing or orrespondene problem.The rank and ensus are two lters based on or-der statistis whih have been applied to the mathingproblem. Advantages of these lters inlude their ro-bustness to radiometri distortion and small amountsof random noise, and their amenability to hardwareimplementation. This paper presents a new math-ing algorithm, whih provides an overall framework formathing, and is used to ompare standard mathingmetris with rank and ensus based tehniques.This paper is organised as follows. Setion 2 de-sribes the rank and ensus lters. The new mathingalgorithm is then outlined in Setion 3. Results ob-tained for a number of test images, using both stan-dard mathing metris, and the rank and ensus, arepresented in Setion 4. In Setion 5, results obtainedfor a syntheti stereo pair are ompared with a knownground truth. Setion 6 provides some disussion ofthese results and onludes.2. RANK AND CENSUS FILTERSThe rank and ensus lters were rst proposed for thestereo mathing problem in [8℄. Their reliane on theordering of pixel values, rather than pixel values them-selves, means that they are robust to radiometri dis-tortion. This is signiant beause radiometri distor-tion is a problem whih an often arise in the eld,partiularly where low ost ameras are used[2℄. Fur-thermore their low omputational omplexity lends it-self to hardware implementation, therefore they havepotential for real-time appliations.2.1. Rank FilterA window of size M N is passed over the image. Ateah loation in the image, the number of pixels lessthan the entre pixel are ounted. This beomes thevalue of the rank image at that loation. Two rankltered image regions may be ompared using the SAD(Sum of Absolute Dierenes) metri[1℄.2.2. Census FilterAgain, a window of sizeMN is passed over the image.At eah loation in the image, the pixel neighbourhoodis mapped to a bit string. If a pixel is less than theentre pixel, the orresponding position in the bit stringis set to 1, otherwise it is set to 0. Eah loation in theensus transformed image therefore onsists of a bitComputation ofEpipolar GeometryResults: Disparity Map              Match DiagnosisIdentify PotentialMatchesMatching performed forall window sizes?Combine resultsfor all window sizesCompute MatchingScoresRemove InconsistentmatchesRemoveAnomalousmatchesSelect mostprobablematchesUpdate probabilitiesbased on neighboursHighest resolutionin image pyramid?Propagate matchesto next pyramid levelInput: imagesnoyesnonoyesyesComputation of Interest ScoresReset invalidmatches tounmatchedAll points matched?Maximum number ofpasses reached?Select next N most"interesting" pointsSparse MatchingnoyesDense MatchingFigure 1: Overall mathing algorithm.string, whih haraterises the pixel neighbourhood atthat image loation. Regions in ensus ltered imagesmay be ompared by ounting the number of bits whihdier in the bit strings | eetively an XOR operation| and summing over the region.Two hardware implementations of mathing usingthe ensus lter are desribed in [3, 7℄.3. MATCHING ALGORITHMFigure 1 depits the overall mathing algorithm. Thevarious omponents of this algorithm are desribed asfollows:3.1. Sparse Mathing and Epipolar GeometryThe input to the algorithm onsists of an image pair,and the fundamental matrix, whih enapsulates thegeometri relationship between the two images whihomprise the image pair. For ompleteness, Figure 1shows that the main algorithm is preeded by a sparsemathing stage, whih automatially detets a set ofrobust mathes in the images, and a stage whih om-putes the epipolar geometry (in this ase performedusing the INRIA FMatrix software[4℄). The sparsemathing algorithm ould itself be deomposed into ablok diagram and desribed in detail, however this isbeyond the sope of this paper.3.2. Computation of Interest SoresAn interest sore is omputed for every loation in theimages. Points whose interest sore is below a thresholdare agged as \low interest".3.3. Seletion of PointsThe N most interesting points are seleted for math-ing, where N is typially set to one tenth the size of theimage. The algorithm progressively mathes the nextN lesser important points, until all points are mathed.3.4. Computation of Mathing SoresA template window entered on eah point in the rstimage is shifted in integer inrements along the epipolarline in the seond image. The value of the math soreis omputed at eah andidate position, using mathmetris[1℄, or the rank and ensus tehniques. This re-sults in an array of math sores, in whih potentialmathes are identied as loal maxima or minima, de-pending on the metri used. The math sores are usedto estimate an initial probability for eah math.(a) (b)Figure 2: Test image pairs (a) road (b) hmmwv2.3.5. Image PyramidThe algorithm allows for the use of an image pyramid,where math sores are omputed from multiple reso-lution images. If the highest resolution level has notbeen reahed, the potential mathes are propagated tothe next level.3.6. Update of Math ProbabilitiesThe math probabilities are inreased proportional tothe number of neighbouring points already mathed,whih have a similar disparity.3.7. Removal of Invalid MathesFor eah point, the math having the highest probabil-ity is seleted. The math sore array is interpolated todetermine the disparity to sub-pixel auray. A num-ber of tehniques are then used to ull mathes likelyto be invalid. Loally anomalous mathes, whih dierfrom their neighbours more than a given threshold, areagged as \anomalous". Mathing is then performedin reverse, that is, the mathed loation in the se-ond image is itself mathed bak to the rst image. Ifthe mathed loation mathes bak to a loation otherthan the original point, the math is agged as \inon-sistent".3.8. Multiple algorithm passesIt is possible to run the algorithm more than one |on eah \pass", all points agged as having invalidmathes are set to unmathed, and it is attempted tomath these points again.3.9. Multiple Math Window SizesThe mathing proess may be repeated for a number ofdierent window sizes. The disparity results obtainedfor eah window size are then ombined into a singledisparity result. This is done by initially setting the(a) (b)() (d)Figure 3: Disparity results for \road" stereo pair (a)SAD (b) NCC () RANK (d) CENSUS.global disparity to that obtained using the largest win-dow size. The global disparity is updated in turn by thedisparity results of suessively smaller window sizes. Itis updated depending on the existene of enough neigh-bours in the global disparity whih are within a spe-ied threshold of the smaller window disparity value.This strategy assumes that the disparity results fromlarger window sizes are the most reliable, and ensuresthat obviously inorret disparity values from smallerwindow sizes do not inuene the global disparity re-sult. The aim of this tehnique is to improve the au-ray of the disparity result and remove the \smoothing"eet introdued by using a window of pixel values formathing.3.10. Output of the AlgorithmThe outputs of the mathing algorithm onsist of x,y and absolute value of disparity images, as well asan image of \math ags". Eah point in the \mathags" image is one of the following values:mathed - point suessfully mathed.low interest - interest sore below threshold.not found - there exists no loal maxima or minima ofthe math sores array.inonsistent - math failed the onsisteny test.anomalous - math was loally anomalous.border - loation was a border region.(a) (b)() (d)Figure 4: Disparity results for \hmmwv2" stereo pair(a) SAD (b) NCC () RANK (d) CENSUS.4. RESULTS FOR TEST IMAGESFigure 2 shows the left image of two dierent test pairs,obtained from [5℄. Both image pairs suer from radio-metri distortion, for the \road" pair, the right image isapproximately 30% brighter than the left, while for the\hmmwv2" pair, the left image is 15% brighter thanthe right.Disparity results for the test pairs, using the SAD(Sum of Absolute Dierenes) and NCC (NormalisedCross Correlation) metris [1℄, and the rank and en-sus tehniques, are shown in Figures 3 and 4. In eahase, the results were produed using the simplest ase| one pyramid level, one pass of the algorithm, andonly one math window size (in this ase 11). Thiswas in order to fous on the omparison between stan-dard mathing metris and rank and ensus based teh-niques, and to avoid the ombinatorial explosion of al-ternative ombinations of math algorithm input pa-rameters.Table 1 shows, for eah test image pair, the pro-portion of points mathed, and the number of mathesreturned for eah value of math ag. The omputa-tion of the proportion mathed does not inlude pointsagged as \border" or \low interest", for whih math-ing was not attempted.5. COMPARISON WITH GROUND TRUTHRESULTSComparison with ground truth, ie, \orret" results,provides a method of testing the auray of the re-(a) (b)Figure 5: Corridor images (a) left image (b) groundtruth disparity.(a) (b)() (d)Figure 6: Disparity results for orridor images (a) SAD(b) NCC () RANK (d) CENSUS.sults returned by the mathing algorithm. However,dense ground truth data is generally not available forreal stereo pairs. The \orridor" stereo pair[6℄ is onesyntheti stereo pair for whih ground truth disparitydata is known. Figure 5 shows the left image of thisstereo pair, and the ground truth disparity image.The mathing algorithm was run with the parame-ters as for the test pairs in Setion 4, exept that thethreshold \interest sore" was set to zero. This meantthat mathing would be attempted for all image points(apart from border areas). Figure 6 shows the dispar-ity results obtained using the SAD and NCC metris,and the rank and ensus tehniques. The proportionof points mathed, and the number of points returnedwith eah math ag, are given in Table 1. For theorridor images, Table 1 also shows the omputed rmsdierene been the disparity of the mathed pixels, andthe ground truth disparity.imagemathmethodprop.mathedmathedlowinterestnot found inonsistent anomalous border rms di.road SAD 0.0208 4372 20050 7497 191374 6851 15616 -NCC 0.6942 145852 20050 3674 54473 6095 15616 -RANK 0.7221 150451 17971 3297 43769 10832 19440 -CENSUS 0.8662 180477 17971 2880 20782 4210 19440 -hmmwv2 SAD 0.1849 43585 10335 22559 138294 31243 16128 -NCC 0.7845 184903 10335 21518 20945 8315 16128 -RANK 0.7915 185005 8325 21933 21104 5697 20080 -CENSUS 0.8010 187217 8325 22537 18310 5675 20080 -orridor SAD 0.8192 47182 0 1773 5795 2848 7936 0.3990NCC 0.7559 43539 0 3101 8044 2916 7936 0.5449RANK 0.8157 45431 0 4019 4796 1450 9840 0.4148CENSUS 0.8262 46016 0 4139 4189 1352 9840 0.4100Table 1: Results of mathing for test images.6. DISCUSSIONFrom Figures 3 and 4 it an be learly seen that theSAD is not robust to radiometri distortion. Othermetris, suh as the NCC, are robust to this type of dis-tortion, however introdue more omputational over-head. It an be seen from Table 1, and from resultsfrom of other test pairs not shown in this paper, thatthe performane of the rank and ensus is generallyomparable to, if not better than, metris suh as theNCC. Furthermore, they have the advantage that theiromputational overhead is muh less than these met-ris.The mathing metris and the rank and ensus havealso been ompared using a known ground truth pair.In Table 1 shows this is the only pair for whih theSAD metri has out-performed all other tehniques. Itis supposed that this is due to the \perfet" nature of asyntheti stereo pair | in that it does not suer fromnoise or radiometri distortion. However, it an be seenfrom Table 1 that the rank and ensus have performedomparable to the SAD, and that for all mathing teh-niques, the dierene between the omputed disparityand ground truth disparity is small.Further work would involve more extensive test-ing of the presented mathing framework | for ex-ample, using multiple \passes" of the algorithm to re-math points agged as \not found", \inonsistent"or \anomalous", and ombining results obtained frommultiple window sizes.7. ACKNOWLEDGEMENTSThis work was arried out with the support of the Eu-ropean Union through the CAMERA (ERB FMRX-CT97-0127) researh network.8. REFERENCES[1℄ P. Ashwanden and W. Guggenbuhl. Experimentalresults from a omparative study on orrelation-type registration algorithms. In Forstner andRuwiedel, editors, Robust Computer Vision, pages268{289. Wikmann, 1993.[2℄ J. Banks, M. Bennamoun, P. Corke, and K. Ku-bik. Suitability of non-parametri stereo mathingtehniques for mining automation. Australian Jour-nal of Intelligent Information Proessing Systems,5(2):111{119, 1998.[3℄ P. Dunn and P. Corke. Real-time stereopsis usingFPGAs. In Proeedings 7th International Work-shop on Field-Programmable Logi and Applia-tions, pages 400{409, London, September 1997.[4℄ Softwares FMatrix & Fdi. http://www-sop.inria.fr/robotvis/personnel/zzhang/software-fmatrix.html.[5℄ Vision and Autonomous Systems Center's ImageDatabase. http://www.vas.ri.mu.edu/idb/.[6℄ Stereo Images with Ground Truth Dispar-ity and Olusion. http://www-dbv.s.uni-bonn.de/ ft/stereo.html.[7℄ J. Woodll and B. Herzen. Real-time stereo vi-sion on the PARTS reongurable omputer. InProeedings IEEEWorkshop in FPGAs for CustomComputing Mahines, pages 242{250, 1993.[8℄ R. Zabih and J. Woodll. Non-parametri loaltransforms for omputing visual orrespondene. InProeedings ECCV'94, pages 151{158, Stokholm,May 1994.

Order statistic filters for image matching

The rank and census are two filters based on order statistics which have been applied to the image matching problem for stereo pairs. Advantages of these filters include their robustness to radiometric distortion and small amounts of random noise, and their amenability to hardware implementation.  In this paper, a new matching algorithm is presented, which provides an overall framework for matching, and is used to compare the rank and census techniques with standard matching metrics.  The algorithm was tested using both real stereo pairs and a synthetic pair with ground truth.\ud
\ud
The rank and census filters were shown to significantly improve performance in the case of radiometric distortion.\ud
In all cases, the results obtained were comparable to, if not better than, those obtained using standard matching metrics. Furthermore, the rank and census have the additional advantage that their computational overhead is less than these metrics.  For all techniques tested, the difference between the results obtained for the synthetic stereo pair, and the ground truth results was small

Order statistic filters for image matching

Abstract

Similar works

Full text

Available Versions

Queensland University of Technology ePrints Archive

Name not available