Multiview Differential Geometry of Curves

The field of multiple view geometry has seen tremendous progress in reconstruction and calibration due to methods for extracting reliable point features and key developments in projective geometry. Point features, however, are not available in certain applications and result in unstructured point cloud reconstructions. General image curves provide a complementary feature when keypoints are scarce, and result in 3D curve geometry, but face challenges not addressed by the usual projective geometry of points and algebraic curves. We address these challenges by laying the theoretical foundations of a framework based on the differential geometry of general curves, including stationary curves, occluding contours, and non-rigid curves, aiming at stereo correspondence, camera estimation (including calibration, pose, and multiview epipolar geometry), and 3D reconstruction given measured image curves. By gathering previous results into a cohesive theory, novel results were made possible, yielding three contributions. First we derive the differential geometry of an image curve (tangent, curvature, curvature derivative) from that of the underlying space curve (tangent, curvature, curvature derivative, torsion). Second, we derive the differential geometry of a space curve from that of two corresponding image curves. Third, the differential motion of an image curve is derived from camera motion and the differential geometry and motion of the space curve. The availability of such a theory enables novel curve-based multiview reconstruction and camera estimation systems to augment existing point-based approaches. This theory has been used to reconstruct a "3D curve sketch", to determine camera pose from local curve geometry, and tracking; other developments are underway.Comment: International Journal of Computer Vision Final Accepted version. International Journal of Computer Vision, 2016. The final publication is available at Springer via http://dx.doi.org/10.1007/s11263-016-0912-

Toward Recovering Shape and Motion of 3D Curves from Multi-View Image Sequences

We introduce a framework for recovering the 3D shape and motion of unknown, arbitrarily-moving curves from two or more image sequences acquired simultaneously from distinct points in space. We use this framework to (1) identify ambiguities in the multi-view recovery of (rigid or nonrigid) 3D motion for arbitrary curves, and (2) identify a novel spatio-temporal constraint that couples the problems of 3D shape and 3D motion recovery in the multi-view case. We show that this constraint leads to a simple hypothesizeand -test algorithm for estimating 3D curve shape and motion simultaneously. Experiments performed with synthetic data suggest that, in addition to recovering 3D curve motion, our approach yields shape estimates of higher accuracy than those obtained when stereo analysis alone is applied to a multi-view sequence. 1 Introduction A fundamental problem in computer vision is to recover the 3D shape and motion of unknown dynamic scenes from sequences of images. While this problem h..