This paper details a novel approach to automatically selecting images which improve camera calibration results. An algorithm is presented which identiﬁes calibration images that inherently improve camera parameter estimates based on their geometric conﬁguration or image network geometry. Analysing images in a more intuitive geometric framework allows image networks to be formed based on the relationship between their world to image homographies. Geometrically, it is equivalent to enforcing maximum independence between calibration images, this ensures accuracy and stability when solving the planar calibration equations. A webcam application using the proposed strategy is presented. This demonstrates that careful consideration of image network geometry, which has largely been neglected within the community, can yield more accurate parameter estimates with less images

Byrne, Brendan P.

Mallon, John

Whelan, Paul F.

Irish Universities

Efficient planar camera calibration via automatic image selection

Name not available

EFFICIENT PLANAR CAMERA CALIBRATION VIA AUTOMATICIMAGE SELECTIONBrendan P. Byrne, John Mallon, Paul F. WhelanVision System Group, Dublin City University, Dublin 9, Irelandbrendan.byrne@eeng.dcu.ieKeywords: Planar Camera Calibration, Image Network, Automatic Image Selection.Abstract: This paper details a novel approach to automatically selecting images which improve camera calibration re-sults. An algorithm is presented which identifies calibration images that inherently improve camera parameterestimates based on their geometric configuration or image network geometry. Analysing images in a moreintuitive geometric framework allows image networks to be formed based on the relationship between theirworld to image homographies. Geometrically, it is equivalent to enforcing maximum independence betweencalibration images, this ensures accuracy and stability when solving the planar calibration equations. A we-bcam application using the proposed strategy is presented. This demonstrates that careful consideration ofimage network geometry, which has largely been neglected within the community, can yield more accurateparameter estimates with less images.1 INTRODUCTIONCamera calibration from planar targets is the domi-nant configuration when working with conventionalperspective cameras. Its foundations were proposedindependently by Sturm and Zhang (Sturm and May-bank, 1999; Zhang, 1999) and are well understoodwithin the community today. Compared with tradi-tional calibration techniques (Faugeras, 1993; Tsai,1987; Wei and Ma, 1994), plane based calibration sig-nificantly reduces input data requirements. This cou-pled with readily available implementations, (Bouget,2001; OpenCV, 2001), has contributed to its popular-ity.Planar calibration simply requires the user to cap-ture at least two images with varying planar targetpose. We call these images the image network (IN).Typically many more images are used to improve cal-ibration accuracy. The stability of planar calibrationmethods has been well studied in the literature (Gon-zalez et al., 2005; Salvi et al., 2002; Sun and Cooper-stock, 2007). Generally the stability is characterisedbased on the reprojection error which is the Euclideandistance between the 3D world points, back projectedonto the image plane, and the measured 2D imagepoints. In all cases findings suggest that larger INsincrease parameter estimation accuracy.Although large INs yield more accurate results,practical constraints such as capturing a large dataset, feature extraction, and calibration takes a consid-erable amount of time and expertise. To this end itwould be desirable to achieve good calibration accu-racy within these practical constraints. Thus the focusshould be on forming smaller INs capable of produc-ing accurate results. This paper outlines how this isachieved via automatic image selection (or a SelectedImage Network (SIN)).The possibility of degeneracies increases whenconsidering a smaller image set. The degeneracieseffect the conditioning of the planar calibration equa-tions and lead to poor parameter estimates. Criticalconfigurations have been identified and well studiedby (Sturm and Maybank, 1999) and (Zhang, 1999)and shown to consist of pure rotation and translation,but images close to critical, which are difficult to iden-tify and have received less attention in the community,also give poor results.In (Rupp and Elter, 2007), an image selectionstrategy is employed to improve calibration results.However, the algorithm selects images based on theirreprojection error which requires the camera to be cal-ibrated in the first place. This defeats the purpose ofan efficient calibration via image selection and addsconsiderable time and effort in doing so. Also, by us-ing the reprojection error as a quality measure, imageand camera noise may effect results as images withfavorable geometry could be omitted. By analysingthe planar calibration problem in an alternative geo-metric framework as proposed by (Gurdjos, 2001) itis possible to form a geometric criterion for selectinggood calibration images which does not require a pre-calibration stage.The main contribution of this work is that it pro-vides an image selection algorithm, based on an al-ternative geometric interpretation, which forms imagenetworks (SINs) that contain less images and give ef-ficient calibration results. An application using a we-bcam is presented which demonstrates that efficientcalibration results can be achieved with little effort.2 METHODThe proposed algorithm selects images based on char-acteristics from the alternative geometric approachproposed by (Gurdjos, 2001). In Figure 1, as theworld plane rotates about its intersection with the im-age plane it remains in homographic correspondence.Thus a single planar homography matrix H representsthe transformation. The associated camera centres, ofeach rotating world to image plane, forms a locus ofpossible camera centres which in turn projects to theimage plane in the form of a line, the Calibration Line(CL). By using the well known (Sturm and Maybank,Figure 1: Poncelet’s Theorem (Gurdjos, 2001)1999; Zhang, 1999) planar calibration equations con-straints (1), and following some algebraic manipula-tions, an expression for the CL can be derived (2).hT1 ωh1−hT2 ωh2 = 0, hT1 ωh2 = 0 (1)where hi is the ith column of H.v0 = Γu0+Λ (2)where Γ is the slope of the CL given by,Γ =−h11h332+h12h331−h11h231h32+h12h31h232h22h31h232−h21h231h32−h21h332+h22h331(3)and Λ is the y-intercept given by,Λ =h21h31+h22h32h231+h232− h11h31+h12h32h231+h232Γ (4)In (Wang and Liu, 2006), the authors propose amethod of linear calibration using the intersectionpoint of CLs. In our method we use the CLs as a guideto choosing calibration images for the SINs. Suit-able images are selected based on the orientation oftheir CLs. Since the CL encapsulates geometric infor-mation about the planar grid in the world, enforcingmaximum independence between CLs ensures eachimage contributes independently to solving the sys-tem of planar calibration equations. Independence be-tween images is evaluated based on the relative anglebetween image CLs. The angle between CLs, (θ), isdependant on the number of images in the IN (N).θ =180◦N(5)Therefore if a four IN is required from a data set theangle between image CLs should be 45◦. In practice,a tolerance of ±1◦ is enforced on θ for the selectionprocess. Based on the homographies from multipleimages, we employ an algorithm to select the mostsuitable images which will provide efficient calibra-tion results.3 IMAGE SELECTIONAutomatic image selection is implemented in twostages. The initial step requires each image CL to beformed (as in section 2) with the angle of each CL rel-ative to the x-axis calculated via the slope (arctan(Γ)).CLs that have absolute angle near 0◦ or 90◦ are re-jected for consideration as they correspond to unsta-ble IN geometry (fronto-parallel). Once N is chosenthe optimal θ can be calculated (5) which is used toform the SINs.The underlying search method of the proposedimage selection strategy is a binary search approach(Knuth, 1998). In Figure 2 the proposed strategy ispresented. Each node represents an image numberwhile the number adjacent to each node is its CL ori-entation. In a real situation all nodes are connectedto each other where the connecting lines represent theangle between image CLs. To aid explanation all pos-sible connecting lines are not shown, instead the validsearch paths are shown i.e each line is in fact equal tothe binary search key which is |θ|±1In this example, N, the number of images requiredin the SIN, is set to four therefore θ is 45◦. Thesearch begins with the seed node 1. When a routecorresponding to the search key is identified (node3) the search continues with node 3 as the seed. Inthis case there are two possible routes, node 12 and17. Since a binary search exploits only one route at atime, the proposed strategy implements the search ina cascaded fashion. Therefore the search will returnto this point and follow other routes in the next cy-cle. If node 12 is chosen there are three candidates toform the SIN - 22, 31 and 64. When one possibilityis chosen the SIN is formed and the node is deleted.This will allow the binary search to find the next validIN with subsequent searches. When all possible SINsalong the (1,3,12) route have been formed, all imagenodes are reintroduced but node 12 is deleted. Thisenables the algorithm to return to the point (1,3) andfollow route 17. The search continues in this fashionuntil all valid routes have been identified.1 31217223164-25 206465-70-69-71Tier 1Tier 2Tier 3 Tier 4Figure 2: Image Search Space3.1 Search Algorithm ImplementationThe implementation of the search algorithm is pre-sented in Figure 3. The algorithm is initialised withthe seed node which is stored in the IN matrix. Thebinary search is implemented and if it’s successful,the current IN size, NT , is incremented and the imagefound, imt, is stored in IN. A check is performed to seeif NT is equal to N the desired IN size. If true the cur-rent imt is deleted from the search space and the algo-rithm loops back to run the binary search again. Thisis a similar situation as in Figure 2 when tier 4 wasreached, the node was deleted so other nodes couldbe found in subsequent searches. In the case whereNT 6= N, the algortihm simply loops back to find thenext image in the network via the binary search.The most significant branch point in the algorithmis the binary search junction. When a search fails it in-dicates that there are no more routes in a particular tierof the search space. A check is performed on NT tosee if the search has failed in tier 2 (i.e. if NT = 1). Ifit has failed the overall search is finished and all validINs are stored in the IN matrix. On the other hand, ifNT 6= 1, all nodes are reintroduced to the search spaceand the Nth node of the current IN is deleted from thelist. This is a similar situation to the example givenin Figure 2 where the IN was (1,3,12). When all tier4 nodes were found and subsequently deleted, node12 was removed which allowed the binary search tofollow the (1,3,17) route. The search continues untilNT = 1 which indicates that all possible routes havebeen explored. The algorithm is implemented in thisfashion for each node in the search space. Further re-finement can be applied to the algorithm if duplicateINs are stored.InitialiseIN = [IN im]BS fail?N   = N   + 1 IN=[IN imt];T TN  = N?TN  = 1?Load nodes,delete N   from listand current INthdelete imtnoyesyesnoFinishStore INnoyes                imt : image returned (BS)      BS : Binary search         IN : Image Network    im : Initial image          N : Desired IN size         N  : Current IN sizeTTFigure 3: Algorithm Flowchart4 APPLICATIONA webcam calibration application was developedto demonstrate the benefits of the image selectionscheme. A labtec “web” webcam capturing 15fps at320× 240 pixels was used. The application capturesimages for 20 seconds thus giving 300 images for thecalibration data set. Images are captured, by vary-ing the location of the camera, while also ensuringthe planar calibration target remains within the fieldof view of the webcam. OpenCV is used to extractthe planar pattern corners from each frame to subpixelaccuracy. The homographies are calculated for eachframe followed by implementation of the proposedimage selection algorithm to identify the SINs. Theplanar calibration method of (Zhang, 1999) is used toverify results.5 RESULTSValidation of the SINs is performed by comparisonwith randomly drawn INs from the data set. RandomINs (RIN) are used as there are no specific methodsto compare with currently in the literature. Three ex-periments are conducted to illustrate the calibrationaccuracy and overall benefit of using SINs. The firstexperiment examines the accuracy of the calibrationresult as the number of views increases. This is fol-lowed by a rigorous comparison of the IN configura-tions with less images. The final experiment validatesthe calibration accuracy of SINs by analysing the lensdistortion in the images. As there is no ground trutha calibration result is calculated using all 300 framesand used as an indicative optimal result for compari-son.5.1 Convergence of Image NetworksThis experiment examines the accuracy of the cali-bration result as the number of images in the IN in-creases. The results are given in terms of percentageerror for each calibration result. For each image net-work instance (2 to 25 INs) ten SINs are comparedwith ten RINs from the data set. From Figure 4, it isevident that the SINs produce more accurate and re-liable calibration results than the RINs, especially forthe smaller INs. As the number of views increasesthe stability of the SIN results clearly outperform theRINs. The most significant result here is that less im-ages are needed to achieve accurate calibration resultswhen SINs are used. The following experiment exam-ines this in more detail.2 6 10 14 18 22 25051015202530Convergence of SIN v RIN# ViewsCalibration Pixel Error (%) RINSIN Figure 4: Calibration accuracy as the number of views in-creases from 2 to 25 INs. Standard deviation is representedby bars at each IN instance.Image Network No. fx fy u vConsensus Images x¯ (σ) x¯ (σ) x¯ (σ) x¯ (σ)Random 2 23.3 20.7 23.7 23.5(41.1)(31.5)(44.6)(27.1)SIN 2 11.5 11.1 13.3 24.1(11.7)(11.7)(16.6)(26.5)Random 3 20.2 20.5 24.2 25.1(30.6)(30.7)(31.4)(31.5)SIN 3 3.8 3.6 5.3 9.9(3.9) (4.3) (7.5) (7.9)Random 4 8.9 8.9 10.8 11.1(21.1)(21.1)(22.9)(19.4)SIN 4 1.59 1.49 2.74 3.88(1.55)(1.69)(3.31)(3.32)Table 1: Image Network Configuration Comparison5.2 Reduced Number of ImagesThe number of images for each IN configuration isreduced for this set of experiments to validate the pre-vious results which identified that SINs can achievemore accurate results than the RINs with less images.All results in Table 1 are given in terms of percentageerror with respect to the indicative optimal calibrationresults. There are 50 two INs, 30 three and 25 fourINs in the SIN cases of Table 1. The RINs consist of2500 randomly drawn image combinations for eachIN size. This consisted of 50 trials of 50 IN instancesfor each test. This insured that a wide range of RINswere compared for the results.The results in Table 1 indicate that the SINs sig-nificantly outperform the RINs. This is due to thegeometry of the SINs, which avoid degenerate con-figurations, and also have favorable image geometryfor solving the calibration equations more efficiently.This is a significant result in terms of the input re-quirements for planar camera calibration, by exploit-ing geometric properties of the images, less input im-ages are needed to achieve accurate calibration re-sults.5.3 Distortion CorrectionThis experiment further justifies the use of the SINsby analysing distortion in the images. The benefitof un-distorting the images is that the canonical cali-bration plane is used as ground truth which gives thedistortion correction residuals. Calibration results us-ing two INs is used for this experiment since it isthe minimum case for solving planar calibration. Theplots in Figure 5 represent the error vectors of the un-distortion estimation (scaled by 20 for plots). TheSINs have removed most radial distortion from theimage (mean residual 0.2 pixels) whereas the randomnetwork still has significant distortion present (meanresidual 0.39 pixels).100 150 200 250 30050100150Random 2 Image Network100 150 200 25050100150Filtered 2 Image NetworkFigure 5: Residuals after un-distortion stage (in pixels)6 CONCLUSIONSThis paper has presented a new approach to choosingplanar calibration images which give accurate calibra-tion results. This is accomplished by considering ge-ometric aspects of images in the form of the CL andthus forming selected image networks (SINs). Unlikeprevious image selection methods it does not require apre-calibration stage, it simply takes the image set andcalibrates based on the image CLs. This ensures inde-pendence in the calibration images and avoids degen-erate configurations. Experimentation has shown thatless input images are needed to achieve efficient re-sults using SINs. The application presented also pro-vides a simple means of achieving accurate results fornon-expert practitioners, as the only input needed isvarying the camera location while capturing the im-ages.ACKNOWLEDGEMENTSThis research is funded by the Irish Research Councilfor Science, Engineering and Technology: funded bythe National Development Plan.REFERENCESBouget, J. (2001). www.vision.caltech.edu/bouguetj/calib.Faugeras, O. (1993). Three Dimensional Computer Vision:A geometric Viewpoint. MIT Press.Gonzalez, J. I., Gmez, J. C., Artal, C. G., and Cabrera, A. N.(2005). Comparative analysis of calibration methodsfor a static camera. WAF05.Gurdjos, P. (2001). Calibration of a moving camera using aplanar pattern: A centre line-based approach for vary-ing focal length. In BMVC.Knuth, D. E. (1998). The Art of Computer Programming,volume 3. Addison-Wesley, second edition.OpenCV (2001). intel.com/technology/computing/opencv.Rupp, S. and Elter, M. (2007). Robust camera calibration -ageneric, optimization-based approach. In VISAPP07.Salvi, J., Armangue, X., and Batlle, J. (2002). A compara-tive review of camera calibrating methods with accu-racy evaluation. Pattern Recognition, 35(7).Sturm, P. and Maybank, S. (1999). On plane-based cameracalibration: A general algorithm, singularities, appli-cations. In CVPR99.Sun, W. and Cooperstock, J. (2007). An empirical evalua-tion of factors influencing camera calibration accuracyusing three publicly available techniques. MVA.Tsai, R. (1987). A versatile camera calibration technique forhigh-accuracy 3d machine vision metrology using off-the-shelf tv cameras and lenses. RA, 3(4):323–344.Wang, J. and Liu, Y. (2006). Characteristic line of planarhomography matrix and its applications in camera cal-ibration. icpr, 1:147–150.Wei, G. and Ma, S. (1994). Implicit and explicit cameracalibration: Theory and experiments. PAMI, 16(5).Zhang, Z. (1999). Flexible camera calibration by viewing aplane from unknown orientations. In ICCV99.

DCU Online Research Access Service

http://doras.dcu.ie/18622/1/whelan_2009_48.pdf

Efficient planar camera calibration via automatic image selection

Abstract

Similar works

Full text

Available Versions

Irish Universities

Name not available

DCU Online Research Access Service