In this paper, we propose a new method to self-calibrate camera with constant internal parameters under circular motion. The basis of our approach is to make use of the conjugate epipoles which are related to camera positions with rotation angles satisfying the conjugate constraint. A novel circular projective reconstruction is developed for computing the conjugate epipoles robustly. It is shown that for a camera with zero skew, two turntable sequences with different camera orientations are needed, and for a general camera three sequences with different camera orientations are required. The performance of the algorithm is tested with real images.published_or_final_versio

Hung, YS

Zhong, H

HKU Scholars Hub

Title Conjugate epipole-based self-calibration of camera undercircular motionAuthor(s) Zhong, H; Hung, YSCitation The 10th IEEE Conference on Mechatronics and Machine Visionin Practice, Hong Kong, China, 9-11 December 2003, p. 8 pagesIssued Date 2003URL http://hdl.handle.net/10722/48464Rights©2003 IEEE. Personal use of this material is permitted. However,permission to reprint/republish this material for advertising orpromotional purposes or for creating new collective works forresale or redistribution to servers or lists, or to reuse anycopyrighted component of this work in other works must beobtained from the IEEE.     © The Copyright of this paper is Reserved.  Mechatronics and Machine Vision 2003: Future Trends, ed. J. Billingsley,  Research Studies Press Ltd, Baldock, UK,  ISBN 0863802907, 2003.  1   Conjugate Epipole-based Self-Calibration of Camera under Circular Motion  H. Zhong and Y. S. Hung Department of Electrical and Electronic Engineering, The University of Hong Kong   Abstract In this paper, we propose a new method to self-calibrate camera with constant internal parameters under circular motion. The basis of our approach is to make use of the conjugate epipoles which are related to camera positions with rotation angles satisfying the conjugate constraint. A novel circular projective reconstruction is developed for computing the conjugate epipoles robustly. It is shown that for a camera with zero skew, two turntable sequences with different camera orientations are needed, and for a general camera three sequences with different camera orientations are required. The performance of the algorithm is tested with real images.  Keywords: Self-calibration, conjugate epipole, circular motion.   1. INTRODUCTION Self-calibration means the recovery of the camera's intrinsic parameters by using information only contained in uncalibrated images. In this paper, it is assumed that the camera undergoes known circular motion. This differs from the traditional calibration where the calibration objects with known relative positions are used. Pioneer work on self-calibration can be found in [1, 2] where Faugeras et al. [1] showed that it was possible to calibrate a camera by solving the well-known Kruppa equations. More recently, many papers on self-calibration for camera undergoing pure rotational motion were proposed [3, 4, 5, 6]. However, all these methods assume the translation from the camera center to the rotation axis is zero. This condition is difficult to ensure and inevitably introduces errors. The error associated with this assumption is analyzed by Wang et al. in [7]. In this paper, we will consider the circular motion of a camera around a single axis, also referred to as turntable motion. The reconstruction problems arising from circular motion have been investigated by many researchers [8, 10, 11]. Fitzgibbon [10] showed that like camera under planar motion, 3D structure and camera calibration may be determined to within a two parameter family. A theoretical analysis of this degeneracy was detailed in [11]. In [10], it was suggested that 2 knowledge in camera aspect ratio or parallel scene lines could be used to remove the reconstruction ambiguity. However, no complete self-calibration method for circular motion is proposed so far. In this paper we present a conjugate epipole-based algorithm for calibrating a camera with constant internal parameters under circular motion. It is shown that each pair of conjugate epipoles places a linear constraint on the image of absolute conic (the IAC). If sufficient conjugate epipole pairs are determined, then the IAC can be identified. The computation of epipoles is known to be sensitive to image noise, especially when the epipole has large coordinates relative to the image size. To overcome this sensitivity, a novel method based on the projective reconstruction method of Tang and Hung [9] is developed to compute the epipoles. This is called circular projective reconstruction. The paper is organized as follows. Section 2 describes the geometry of circular motion. Section 3 introduces the definition of conjugate epipoles and the calibration algorithm. Section 4 presents how to obtain robust conjugate epipoles via circular projective reconstructions from turntable sequences. Experimental results are presented in section 5. Conclusion is given in section 6.  2. GEOMETRY OF CIRCULAR MOTION A camera rotating about a fixed axis around an object is mathematically equivalent to a static camera viewing an object rotating on a turntable. For the purpose of analysis, the rotating camera model is adopted Fig. 1(a) shows a camera rotating about a fixed axis, with the camera centre describing a circle in a horizontal plane, i.e. the motion plane. The motion is characterized by the rotation angle θ of the camera.  (a)    (b) Fig 1. (a) Geometry of circular motion. C1, C2, and C3 are three camera frames with centres on a circle. C1 is the initial camera. C2 and C3 are characterized by the rotation angle θ2 and θ3 respectively. (b) Conjugate epipoles in circular motion. e12 and e32 are the epipoles on C2 with respect to C1 and C3 respectively. They are conjugate epipoles because the rays corresponding to them are orthogonal.  Without loss of generality, we can choose the world coordinate system such that the Z axis is aligned with the rotation axis and camera C1 is at position r on the world X axis. Let the rotation of C1 about its centre relative to the world frame be R. Then, the projection matrix of C1 may be written as  θ2C2C1C3e12e32C1C2 C3θ2 θ3 XZ 3   [ ]1 |=P KR I t  (2.1)  where   00 0 1uvs uvαα   =      K  (2.2)  is the camera calibration matrix and 0 0Tr =   t . In (2.2), αu and αv are the scaling factors in the X and Y directions of the image respectively, s is the skew parameter and the principle point is (u, v). Referring to this camera, a second camera rotated by θi from C1 about the Z axis is simply given by   1i i=P PQ  (2.3)  where Qi is a rotation matrix about the Z axis in homogeneous coordinates. Eq. (2.3) gives a compact description of the geometry of circular motion. Projection matrices are related by rotation angles only. If P1 is known, then any other projection matrix can be determined from the rotation angle θi. Given a set of projection matrices, if they satisfy (2.3), then this set of projection matrices is said to be consistent with the circular constraint. The corresponding projective reconstruction is called a circular projective reconstruction.  3. CALIBRATION BASED ON CONJUGATE EPIPOLES We will now derive the conjugacy constraint on the IAC from the conjugate epipoles under circular motion.  3.1 Conjugate epipoles Given two image points x1 and x2 and the image of the absolute conic 1T− −=ω K K , if the rays back-projected from these two points are orthogonal, then they satisfy the following relationship   1 2 0T =ωx x . (3.1.1)  Image points satisfying (3.1.1) are conjugate with respect to ω. If two epipoles on an image satisfy this constraint, they are called conjugate epipoles. Geometrically, this means that the rays back-projected from these two epipoles are orthogonal. Thus, conjugate epipoles encapsulate orthogonality in Euclidean space.  3.2 Conjugate epipoles in circular motion A top view of the circular motion is shown in fig. 1(b). The circle is the circular 4 path of the rotating camera. C1 and C3 are two cameras on a diameter of the circle. C2 is a third camera which is apart from C1 by θ2. From analytic geometry, the lines 1 2C C  and 3 2C C  are orthogonal. These two lines intersect the image plane of C2 at two points, e12 and e32, respectively. In the context of epipolar geometry, e12 and e32, are two epipoles on camera C2 with respect to camera C1 and C3. Because the rays corresponding to them are orthogonal, they form a conjugacy relation with respect to ω as discussed in section 3.1. Therefore, they are a pair of conjugate epipoles. In fact, let θij be the rotation angle from camera i to camera j, and eij be the projection of the ith camera center on the jth view, it can be seen that eik and ejk are conjugate epipoles if   ik kjθ θ π+ = . (3.2.1)  This is the conjugate constraint for cameras i, j, and k to satisfy in order to make the epipoles eik and ejk conjugate epipoles.  3.3 Estimating the IAC Each conjugate epipoles places one linear constraint on ω as given in (3.1.1). Given many pairs of conjugate epipoles, ω can be solved by least-squares method. More specifically, since ω is a conic, its matrix form may be written as   a b db c ed e f   =      ω .  Substituting the ith pair of conjugate epipoles 1Ti i ix y =   e  and 1Ti i ix y =   ? ? ?e  into (3.1.1) yields   1 0i i i i i i i i i i i ix x x y y x y y x x y y + + + =  ? ? ? ? ? ? w  (3.3.1)  where Ta b c d e f =   w . By stacking equations obtained from n pairs of conjugate epipoles, we may establish a linear system   =Aw 0  (3.3.2)  where the ith row of A is of the form   1i i i i i i i i i i i ix x x y y x y y x x y y + + +  ? ? ? ? ? ? .  5  The number of independent pairs of conjugate epipoles required to solve for ω depends on the degrees of freedom in ω. In a maximum case where the calibration matrix K has five d.o.f., five independent pairs of conjugate epipoles are needed.  It has been shown in [10] that camera calibration under circular motion can be determined up to a two-parameter ambiguity. In this calibration algorithm, the same ambiguity exhibits. For one turntable sequence, only two pairs of conjugate epipoles are independent. Because all conjugate epipoles lie on the vanishing line of the motion plane, they provide only information along the direction of the vanishing line and consequently no information about the direction of rotation axis can be recovered. For two turntable sequences obtained with different camera orientations, there exist up to four independent pairs of conjugate epipoles, which is sufficient to determine a ω arising from a K with 4 degrees of freedom. Finally, to solve for a general ω with 5 degrees of freedom, three sequences are required.   4. ESTIMATION OF CONJUGATE EPIPOLES In previous sections, we have discussed how the calibration matrix K may be determined via conjugate epipoles. In this section, we will present a novel method to compute the conjugate epipoles for circular motion based on the circular projective reconstruction.  4.1 Computing circular projective reconstruction Consider a set of projection matrices iP  for i=1,…,m obtained by a projective reconstruction method, e.g. [9], for one turntable sequence. These projection matrices do not necessarily satisfy the circular constraint (2.3). From the projective reconstruction theorem [12 Hartley and Zisserman], we know that there must be a non-singular matrix H which satisfies   i i=P PH  for i=1,…,m (4.1.1)  where Pi is the set of projection matrices satisfying the circular constraint as defined in (2.3). Substituting (2.3) into (4.1.1) gives   1i i=PH PQ  for i=1,…,m. (4.1.2)  A linear system in H and P1 can be formed given iP  and Qi from (4.1.2). Each iP  and Qi gives rise to 12 constraints on H and P1, and they can be computed by using least-squares method if at least three projection matrices together with three corresponding rotation matrices are provided. The P1 computed is then used to obtain another projective reconstruction [9] under the constraint (2.3), i.e. only P1 will be iteratively estimated, other projection matrices are computed from (2.3). The resultant projective reconstruction obtained in this manner is consistent with the circular constraint, and thus provides consistent estimation of epipoles for cameras related by circular motion.  6 4.2 Computing conjugate epipoles The identification of P1 is equivalent to the determination of the epipolar geometry for the whole sequence. This is because fundamental matrix, and thus epipoles can be computed from P1 and the known Qi (see [12 Hartley and Zisserman]). Here we will use the following equations to compute epipoles associated with two cameras              i j j i′= =e PC e PC  (4.2.1)  where Pi and Pj are two camera matrices with PiCi=0 and PjCj=0. To compute the conjugate epipoles, we start with the circular projective reconstruction P1. Let C1 be the camera center of P1 so that P1C1=0. Then the camera matrix opposite to P1 is computed as 1 1 π=?P PQ . Furthermore, its camera center is given by 1 1Tπ=?C Q C  so that 1 1 =? ?PC 0 . Now let Pi be a camera between P1 and 1?P , then according to (2.3) and (4.2.1), the epipoles w.r.t. P1 and 1?P  on Pi are given by   1 1 1i i= =e PC PQ C  and 1 1 1Ti i π= =??e PC PQ Q C  (4.2.2)  As stated in subsection 3.2, it can be verified that e and ?e  are conjugate epipoles. For each sequence, many conjugate epipoles can be computed by choosing different Qi based on (4.2.2), but only two of them will be independent. It is worth noting that, because the conjugate epipoles are essentially computed from P1 for each sequence, the actual values of rotation angles used to compute them do not affect the ultimate calibration result. Similarly the same rotation angles can be used in each sequence without affecting the calibration result.  5 EXPERIMENTAL RESULTS WITH REAL DATA In our experiment with real images, as shown in fig. 2, a house and a book were placed on a turntable which was rotated by about 9.95 degrees each step. The camera focal length was known to be around 70mm. The image size measures 3072 x 2048 pixels. The first sequence was captured with a camera such that the image X axis is roughly parallel to the horizon (fig. 2(a)). The second sequence was captured with a camera rotated such that the angle between its image X axis and the motion plane is about 35 degrees (fig. 2(b)). The third sequence was captured with a camera such that the image Y axis is roughly parallel to the horizon (fig. 2(c)). A total of 35 correspondences were established manually with 16 points on the roof, 16 points on the wall and 3 points from the title on the book. Only those views to which all the matched points are visible were selected for the calibration. Circular projective reconstruction for individual sequence was carried out and three sets of circular projective reconstructions were obtained. Then the conjugate epipoles for each sequence were computed using the first projection matrix of the circular projective reconstruction just obtained. The reprojection errors, obtained after three individual circular projective 7  reconstructions for the three sequences, are 1.1557, 0.9794, and 1.1614 pixels respectively, which suggest that the noise level is about 1 pixel. All combinations of two out of the three computed circular projection matrices are used to compute the calibration matrix assuming zero skew. A full calibration is also estimated by using all the three circular projection matrices. Because no ground truth is known, an image of three squares on three different planes is also captured, and the simple calibration method presented in [12 Hartley and Zisserman] is used to compute a calibration result as a reference. This result is shown in the first row of table 1, suggesting the scaling factors in the X and Y axes are about 8500 and 8400 pixels respectively, and the principal point is near the image centre, at (1541.2, 941.7).     (a)   (b)   (c) Fig 2. Three real images from sequences captured with different camera orientations, (a) the X axis of image plane is roughly parallel to the motion plane, (b) the angle between the X axis of image plane and the motion plane is about 35 degrees, (c) the X axis of image plane is roughly parallel to the rotation axis.  Calibration results obtained from image sequences are shown in the rows from 2 to 5 in table 1. The first column indicates the sequences used to compute the calibration results (except the first row). For those using two sequences, the skew parameter is zero. Compared with the results obtained from the simple calibration method, the calibration results computed from three image sequences are reasonably good. However, in terms of generality the result obtained from three sequences is more convincing since the calibration does not assume zero skew.  Sequences used αu αv s u V Three squares 8508.1 8401.3 11.8 1541.2 941.7 S1S2 8556.3 8492.9 0 1794.0 846.0 S2S3 8478.0 8395.6 0 1608.9 1139.3S3S1 8499.9 8429.8 0 1798.6 1142.4S1S2S3 8509.3 8416.2 -4.9 1728.4 1100.7Table 1. Calibration results. The first row shows the calibration result obtained by using the simple calibration algorithm; SiSj indicates calibration results are computed from the ith and jth sequences; S1S2S3 indicates calibration results are computed from three sequences. All values are in pixels.  6 CONCLUSION In this paper, a novel self-calibration algorithm based on conjugate epipoles is developed. The conjugate constraint on rotation angle under circular motion is exploited. The algorithm gives accurate calibration result on real images. The 8 performance of this algorithm relies on how accurately the conjugate epipoles are estimated. The sensitivity problem of epipole estimation is overcome by using the circular projective reconstruction with minimization of reprojection error. Furthermore, the development of circular projective reconstruction makes the computation of conjugate epipoles more practical because the cameras are not necessarily rotated with angles satisfying the conjugate constraint.  Acknowledgement We like to thank W. K. Tang and Yan Li for useful discussion on projective reconstruction.  References 1. D. Faugeras, Q.-T. Luong, and S. J. Maybank, “Camera self-calibration: Theory and experiments”, Computer Vision-ECCV’92, vol. 588, LNCS –Series, Springer-Verlag, pp. 321-334, 1992. 2. S. J. Maybank and O. D. Faugeras, “A Theory of Self-calibration of a moving camera”, International Journal of Computer Vision, 8(2), pp. 123-151. 3. Songtal Dai and Qiang Ji, “A New Technique for Camera Self-Calibration”, Proc. of the 2001 IEEE Int. Conf. on Robotics & Automation, pp. 2165-2170, 2001. 4. L. Agapito, E. Hayman, and I. Reid, “Self-Calibration of Rotating and Zooming Cameras”, International Journal of Computer Vision, 45(2), pp. 107-127, 2001. 5. R. Hartley, “Self-Calibration of Stationary Cameras”, International Journal of Computer Vision, 22(1), pp. 5-23, 1997. 6. P. R. S. Mendoca and R. Cipolla, “A Simple Technique for Self-Calibration”, IEEE Conf. on Computer Vision and Pattern Recognition, pp. 500-505, 1999. 7. L. Wang, S. B. Kang, H. Y. Shum, and G. Xu, “Error Analysis of Pure Rotation-based Self-Calibration”, Proc. of the 8th Int. Conf. on Computer Vision, IEEE, pp. 464-701, 2001. 8. A. Zisserman, D. Liebowitz and M. Armstrong, “Resolving Ambiguities in Auto-Calibration”, Phil. Trans. R. Soc. Lnad. A 356, pp. 1193-1211, 1998. 9. W. K. Tang and Y. S. Hung, “A Factorization-based method for Projective Reconstruction with minimization of 2-D reprojection errors”, DAGM, pp. 387-394, 2002. 10. A. W. Fitzgibbon, G. Cross, and A. Zisserman, “Automatic 3D Model Construction for Turn-Table Sequences”, SMILE’98, LNCS 1506, pp. 155-170, 1998. 11. Y. Liu, H. T. Tsui, and C. K. Wu, “Resolving Ambiguities of Self-Calibration in Turntable Motion”, Proc. of the 15th Int. Conf. on Pattern Recognition, IEEE, pp. 865-868, 2000. 12. R. Hartley and A. Zisserman, “Multiple View Geometry in Computer Vision”, Cambridge Press, 2000. 

Conjugate epipole-based self-calibration of camera under circular motion

http://hub.hku.hk/bitstream/10722/48464/1/91701.pdf

Conjugate epipole-based self-calibration of camera under circular motion

Abstract

Similar works

Full text

Available Versions

HKU Scholars Hub