Open problem: Tightness of maximum likelihood semidefinite relaxations

We have observed an interesting, yet unexplained, phenomenon: Semidefinite programming (SDP) based relaxations of maximum likelihood estimators (MLE) tend to be tight in recovery problems with noisy data, even when MLE cannot exactly recover the ground truth. Several results establish tightness of SDP based relaxations in the regime where exact recovery from MLE is possible. However, to the best of our knowledge, their tightness is not understood beyond this regime. As an illustrative example, we focus on the generalized Procrustes problem

Hierarchical structure-and-motion recovery from uncalibrated images

This paper addresses the structure-and-motion problem, that requires to find camera motion and 3D struc- ture from point matches. A new pipeline, dubbed Samantha, is presented, that departs from the prevailing sequential paradigm and embraces instead a hierarchical approach. This method has several advantages, like a provably lower computational complexity, which is necessary to achieve true scalability, and better error containment, leading to more stability and less drift. Moreover, a practical autocalibration procedure allows to process images without ancillary information. Experiments with real data assess the accuracy and the computational efficiency of the method.Comment: Accepted for publication in CVI

Bearing-based Network Localizability: A Unifying View

This paper provides a unifying view and offers new insights on bearing-based network localizability, that is the problem of establishing whether a set of directions between pairs of nodes uniquely determines (up to translation and scale) the position of the nodes in d-space. If nodes represent cameras then we are in the context of global structure from motion. The contribution of the paper is theoretical: first, we rewrite and link in a coherent structure several results that have been presented in different communities using disparate formalisms; second, we derive some new localizability results within the edge-based formulation

Synchronization Problems in Computer Vision

The goal of \u201csynchronization\u201d is to infer the unknown states of a network of nodes, where only the ratio (or difference) between pairs of states can be measured. Typically, states are represented by elements of a group, such as the Symmetric Group or the Special Euclidean Group. The former can represent local labels of a set of features, which refer to the multi-view matching application, whereas the latter can represent camera reference frames, in which case we are in the context of structure from motion, or local coordinates where 3D points are represented, in which case we are dealing with multiple point-set registration. A related problem is that of \u201cbearing-based network localization\u201d where each node is located at a fixed (unknown) position in 3-space and pairs of nodes can measure the direction of the line joining their locations. In this thesis we are interested in global techniques where all the measures are considered at once, as opposed to incremental approaches that grow a solution by adding pieces iteratively

Camera Motion Estimation by Convex Programming

This thesis studies two inverse problems closely related two each other: The first problem is the estimation of n locations t1, t2, . . . , tn (up to global scale, translation and negation) in Rd from noisy measurements of a subset of the pairwise lines that connect them, that is, from noisy measurements of ± (ti - tj)/||ti - tj|| for some pairs (i,j), where the signs are unknown. For the second problem, we assume the availability of (measurements of) the signs, i.e., we consider the estimation of the locations ti (up to global scale and translation) from measurements of the pairwise directions (ti - tj)/||ti - tj|| for some pairs (i,j). These inverse problems are at the core of the structure from motion (SfM) problem in computer vision, where the ti's represent camera locations in R3. After introducing the inverse problems and providing a discussion of previous related works in Chapter 1, we continue with the characterization of well-posed problem instances in Chapter 2. The contents of Chapter 2 are based on the existing results of parallel rigidity theory, the significance of which was not previously recognized in the context of the camera location estimation problem. Basically, parallel rigidity theory studies the conditions of unique realizability of locations from exact (i.e., noiseless) pairwise lines and directions. We reiterate these results to identify a complete com- binatorial characterization of well-posed instances for the two inverse problems, and provide efficient algorithms to decide in the well-posedness of a given instance. In the absence of these conditions, we discuss how to identify maximal subsets of the pairwise measurements inducing well-posed sub-problems. In Chapter 3, we study the inverse problem of location estimation from noisy pairwise line measurements. We firstly present a fundamental difficulty observed for the existing methods, that is, the tendency to produce spurious solutions that are clustered around a few locations. This is a well-known problem in SfM, especially for large, irregular collections of images. To overcome this difficulty, we introduce a semidefinite relaxation (SDR) method, specially tailored to exclude clustering solutions. For this formulation, we prove exact (in the noiseless case) and stable (in the presence of noisy lines) location recovery results. We also formulate an alternat- ing direction augmented Lagrangian method (ADM) to efficiently solve the resulting semidefinite program. Chapter 4 investigates the second inverse problem, i.e. estimation of locations from noisy pairwise directions. For current methods in the literature, existence of outliers among the direction measurements typically induces large errors in the location estimates (especially for large, unordered image sets). To reduce the effect of outliers, we introduce two efficient convex programs for robust estimation of locations. As we observe in Chapter 7, compared to the existing alternatives, these methods provide highly accurate location estimates in the presence of outlier direction measurements. Provided with partially corrupted measurements (with sufficiently many noiseless directions), we empirically demonstrate that these programs can even recover the locations exactly. In Chapter 4, we also provide iteratively reweighted least squares (IRLS) solvers in order to efficiently solve these robust convex formulations. To maintain the computational efficiency of the SDR formulation of Chapter 3 and the robust convex programs of Chapter 4 for large sets of locations, we introduce distributed formulations of these solvers in Chapter 5, based on spectral clustering and convex programming. We also show that these distributed methods induce well- posed distributed problem instances. In Chapter 6, we demonstrate how to formulate the camera location estimation problem in terms of the two inverse problems of location estimation (in R3) from pairwise lines and directions, and also introduce a convex programming-based method to robustly estimate pairwise lines and directions. Lastly, we demonstrate the utility of our formulations through experiments on synthetic data and real images, in Chapter 7