Robust Camera Location Estimation by Convex Programming

$3$D structure recovery from a collection of $2$D images requires the estimation of the camera locations and orientations, i.e. the camera motion. For large, irregular collections of images, existing methods for the location estimation part, which can be formulated as the inverse problem of estimating $n$ locations $\mathbf{t}_1, \mathbf{t}_2, \ldots, \mathbf{t}_n$ in $\mathbb{R}^3$ from noisy measurements of a subset of the pairwise directions $\frac{\mathbf{t}_i - \mathbf{t}_j}{\|\mathbf{t}_i - \mathbf{t}_j\|}$, are sensitive to outliers in direction measurements. In this paper, we firstly provide a complete characterization of well-posed instances of the location estimation problem, by presenting its relation to the existing theory of parallel rigidity. For robust estimation of camera locations, we introduce a two-step approach, comprised of a pairwise direction estimation method robust to outliers in point correspondences between image pairs, and a convex program to maintain robustness to outlier directions. In the presence of partially corrupted measurements, we empirically demonstrate that our convex formulation can even recover the locations exactly. Lastly, we demonstrate the utility of our formulations through experiments on Internet photo collections.Comment: 10 pages, 6 figures, 3 table

Stable Camera Motion Estimation Using Convex Programming

We study the inverse problem of estimating n locations $t_1, ..., t_n$ (up to global scale, translation and negation) in $R^d$ from noisy measurements of a subset of the (unsigned) pairwise lines that connect them, that is, from noisy measurements of $\pm (t_i - t_j)/\|t_i - t_j\|$ for some pairs (i,j) (where the signs are unknown). This problem is at the core of the structure from motion (SfM) problem in computer vision, where the $t_i$'s represent camera locations in $R^3$. The noiseless version of the problem, with exact line measurements, has been considered previously under the general title of parallel rigidity theory, mainly in order to characterize the conditions for unique realization of locations. For noisy pairwise line measurements, current methods tend to produce spurious solutions that are clustered around a few locations. This sensitivity of the location estimates is a well-known problem in SfM, especially for large, irregular collections of images. In this paper we introduce a semidefinite programming (SDP) formulation, specially tailored to overcome the clustering phenomenon. We further identify the implications of parallel rigidity theory for the location estimation problem to be well-posed, and prove exact (in the noiseless case) and stable location recovery results. We also formulate an alternating direction method to solve the resulting semidefinite program, and provide a distributed version of our formulation for large numbers of locations. Specifically for the camera location estimation problem, we formulate a pairwise line estimation method based on robust camera orientation and subspace estimation. Lastly, we demonstrate the utility of our algorithm through experiments on real images.Comment: 40 pages, 12 figures, 6 tables; notation and some unclear parts updated, some typos correcte

Camera Motion Estimation by Convex Programming

This thesis studies two inverse problems closely related two each other: The first problem is the estimation of n locations t1, t2, . . . , tn (up to global scale, translation and negation) in Rd from noisy measurements of a subset of the pairwise lines that connect them, that is, from noisy measurements of ± (ti - tj)/||ti - tj|| for some pairs (i,j), where the signs are unknown. For the second problem, we assume the availability of (measurements of) the signs, i.e., we consider the estimation of the locations ti (up to global scale and translation) from measurements of the pairwise directions (ti - tj)/||ti - tj|| for some pairs (i,j). These inverse problems are at the core of the structure from motion (SfM) problem in computer vision, where the ti's represent camera locations in R3. After introducing the inverse problems and providing a discussion of previous related works in Chapter 1, we continue with the characterization of well-posed problem instances in Chapter 2. The contents of Chapter 2 are based on the existing results of parallel rigidity theory, the significance of which was not previously recognized in the context of the camera location estimation problem. Basically, parallel rigidity theory studies the conditions of unique realizability of locations from exact (i.e., noiseless) pairwise lines and directions. We reiterate these results to identify a complete com- binatorial characterization of well-posed instances for the two inverse problems, and provide efficient algorithms to decide in the well-posedness of a given instance. In the absence of these conditions, we discuss how to identify maximal subsets of the pairwise measurements inducing well-posed sub-problems. In Chapter 3, we study the inverse problem of location estimation from noisy pairwise line measurements. We firstly present a fundamental difficulty observed for the existing methods, that is, the tendency to produce spurious solutions that are clustered around a few locations. This is a well-known problem in SfM, especially for large, irregular collections of images. To overcome this difficulty, we introduce a semidefinite relaxation (SDR) method, specially tailored to exclude clustering solutions. For this formulation, we prove exact (in the noiseless case) and stable (in the presence of noisy lines) location recovery results. We also formulate an alternat- ing direction augmented Lagrangian method (ADM) to efficiently solve the resulting semidefinite program. Chapter 4 investigates the second inverse problem, i.e. estimation of locations from noisy pairwise directions. For current methods in the literature, existence of outliers among the direction measurements typically induces large errors in the location estimates (especially for large, unordered image sets). To reduce the effect of outliers, we introduce two efficient convex programs for robust estimation of locations. As we observe in Chapter 7, compared to the existing alternatives, these methods provide highly accurate location estimates in the presence of outlier direction measurements. Provided with partially corrupted measurements (with sufficiently many noiseless directions), we empirically demonstrate that these programs can even recover the locations exactly. In Chapter 4, we also provide iteratively reweighted least squares (IRLS) solvers in order to efficiently solve these robust convex formulations. To maintain the computational efficiency of the SDR formulation of Chapter 3 and the robust convex programs of Chapter 4 for large sets of locations, we introduce distributed formulations of these solvers in Chapter 5, based on spectral clustering and convex programming. We also show that these distributed methods induce well- posed distributed problem instances. In Chapter 6, we demonstrate how to formulate the camera location estimation problem in terms of the two inverse problems of location estimation (in R3) from pairwise lines and directions, and also introduce a convex programming-based method to robustly estimate pairwise lines and directions. Lastly, we demonstrate the utility of our formulations through experiments on synthetic data and real images, in Chapter 7

Synchronization over Cartan motion groups via contraction

Group contraction is an algebraic map that relates two classes of Lie groups by a limiting process. We utilize this notion for the compactification of the class of Cartan motion groups. The compactification process is then applied to reduce a non-compact synchronization problem to a problem where the solution can be obtained by means of a unitary, faithful representation. We describe this method of synchronization via contraction in detail and analyze several important aspects of this application. One important special case of Cartan motion groups is the group of rigid motions, also called the special Euclidean group. We thoroughly discuss the synchronization over this group and show numerically the advantages of our approach compared to some current state-of-the-art synchronization methods on both synthetic and real data

A survey of structure from motion

The structure from motion (SfM) problem in computer vision is to recover the three-dimensional (3D) structure of a stationary scene from a set of projective measurements, represented as a collection of two-dimensional (2D) images, via estimation of motion of the cameras corresponding to these images. In essence, SfM involves the three main stages of (i) extracting features in images (e.g. points of interest, lines, etc.) and matching these features between images, (ii) camera motion estimation (e.g. using relative pairwise camera positions estimated from the extracted features), and (iii) recovery of the 3D structure using the estimated motion and features (e.g. by minimizing the so-called reprojection error). This survey mainly focuses on relatively recent developments in the literature pertaining to stages (ii) and (iii). More specifically, after touching upon the early factorization-based techniques for motion and structure estimation, we provide a detailed account of some of the recent camera location estimation methods in the literature, followed by discussion of notable techniques for 3D structure recovery. We also cover the basics of the simultaneous localization and mapping (SLAM) problem, which can be viewed as a specific case of the SfM problem. Further, our survey includes a review of the fundamentals of feature extraction and matching (i.e. stage (i) above), various recent methods for handling ambiguities in 3D scenes, SfM techniques involving relatively uncommon camera models and image features, and popular sources of data and SfM software