Spatially Coherent RANSAC for Multi-Model Fitting

RANSAC [15, 38, 1] is a reliable method for fitting parametric models to sparse data with many outliers. Originally designed for extracting a single model, RANSAC also has variants for fitting multiple models when supported by data. Our main insight is that, in practice, inliers for each model are often spatially coherent — all previous RANSAC-based methods ignore this. Our new method fits an unspecified number of models to data by combining ideas of random sampling and spatial regularization. As in basic RANSAC, we randomly sample data points to generate a set of proposed models (labels). We formulate model selection and inlier classification as a single problem — labeling of triangulated data points. Geometric fit errors and spatial coherence are combined in one MRF-based energy. In contrast to basic RANSAC, inlier classification does not depend on a fixed threshold. Moreover, our optimization framework allows iterative re-estimation of models/inliers with a clear stopping criteria and convergence guarantees. We show that our new method, SCO- RANSAC, can significantly improve results on synthetic and real data supporting multiple linear, affine, and homographic models

Energy Based Multi-Model Fitting and Matching Problems

Feature matching and model fitting are fundamental problems in multi-view geometry. They are chicken-&-egg problems: if models are known it is easier to find matches and vice versa. Standard multi-view geometry techniques sequentially solve feature matching and model fitting as two independent problems after making fairly restrictive assumptions. For example, matching methods rely on strong discriminative power of feature descriptors, which fail for stereo images with repetitive textures or wide baseline. Also, model fitting methods assume given feature matches, which are not known a priori. Moreover, when data supports multiple models the fitting problem becomes challenging even with known matches and current methods commonly use heuristics. One of the main contributions of this thesis is a joint formulation of fitting and matching problems. We are first to introduce an objective function combining both matching and multi-model estimation. We also propose an approximation algorithm for the corresponding NP-hard optimization problem using block-coordinate descent with respect to matching and model fitting variables. For fixed models, our method uses min-cost-max-flow based algorithm to solve a generalization of a linear assignment problem with label cost (sparsity constraint). Fixed matching case reduces to multi-model fitting subproblem, which is interesting in its own right. In contrast to standard heuristic approaches, we introduce global objective functions for multi-model fitting using various forms of regularization (spatial smoothness and sparsity) and propose a graph-cut based optimization algorithm, PEaRL. Experimental results show that our proposed mathematical formulations and optimization algorithms improve the accuracy and robustness of model estimation over the state-of-the-art in computer vision

Fast approximate energy minimization with label costs

The α-expansion algorithm [7] has had a significant impact in computer vision due to its generality, effectiveness, and speed. Thus far it can only minimize energies that involve unary, pairwise, and specialized higher-order terms. Our main contribution is to extend α-expansion so that it can simultaneously optimize “label costs ” as well. An energy with label costs can penalize a solution based on the set of labels that appear in it. The simplest special case is to penalize the number of labels in the solution. Our energy is quite general, and we prove optimality bounds for our algorithm. A natural application of label costs is multi-model fitting, and we demonstrate several such applications in vision: homography detection, motion segmentation, and unsupervised image segmentation. Our C++/MATLAB implementation is publicly available