Model-fitting in the presence of outliers

We study the problem of parametric model-fitting in a finite alphabet setting. We characterize the weak convergence of the goodness-of-fit statistic with respect to an exponential family when the observations are drawn from some alternate distribution. We then study the effects of outliers on the model-fitting procedure by specializing our results to $\epsilon$-contaminated versions of distributions from the exponential family. We characterize the sensitivity of various distributions from the exponential family to outliers, and provide guidelines for choosing thresholds for a goodness-of-fit test that is robust to outliers in the data

Approximate least trimmed sum of squares fitting and applications in image analysis

The least trimmed sum of squares (LTS) regression estimation criterion is a robust statistical method for model fitting in the presence of outliers. Compared with the classical least squares estimator, which uses the entire data set for regression and is consequently sensitive to outliers, LTS identifies the outliers and fits to the remaining data points for improved accuracy. Exactly solving an LTS problem is NP-hard, but as we show here, LTS can be formulated as a concave minimization problem. Since it is usually tractable to globally solve a convex minimization or concave maximization problem in polynomial time, inspired by [1], we instead solve LTS’ approximate complementary problem, which is convex minimization. We show that this complementary problem can be efficiently solved as a second order cone program. We thus propose an iterative procedure to approximately solve the original LTS problem. Our extensive experiments demonstrate that the proposed method is robust, efficient and scalable in dealing with problems where data are contaminated with outliers. We show several applications of our method in image analysis.Fumin Shen, Chunhua Shen, Anton van den Hengel and Zhenmin Tan

Improving RANSAC for Efficient and Precise Model Fitting with Statistical Analysis

RANSAC (random sample consensus) has been widely used as a benchmark algorithm for model fitting in the presence of outliers for more than thirty years. It is robust for outlier removal and rough model fitting, but neither reliable nor efficient enough for many applications where precision and time is critical. Many other algorithms have been proposed for the improvement of RANSAC. However, no much effort has been done to systematically tackle its limitations on model fitting repeatability, quality indication, iteration termination, and multi-model fitting.A new paradigm, named as SASAC (statistical analysis for sample consensus), is introduced in this paper to relinquish the limitations of RANSAC above. Unlike RANSAC that does not consider sampling noise, which is true in most sampling cases, a term named as ? rate is defined in SASAC. It is used both as an indicator for the quality of model fitting and as a criterion for terminating iterative model searching. Iterative least square is advisably integrated in SASAC for optimal model estimation, and a strategy is proposed to handle a multi-model situation.Experiment results for linear and quadratic function model fitting demonstrate that SASAC can significantly improve the quality and reliability of model fitting and largely reduce the number of iterations for model searching. Using the ? rate as an indicator for the quality of model fitting can effectively avoid wrongly estimated model. In addition, SASAC works very well to a multi-model dataset and can provide reliable estimations to all the models. SASAC can be combined with RANSAC and its variants to dramatically improve their performance.</jats:p

Phase wrap error correction by random sample consensus with application to synthetic aperture sonar micro-navigation

Accurate time delay estimation between signals is crucial for coherent imaging systems such as synthetic aperture sonar (SAS) and synthetic aperture radar (SAR). In such systems, time delay estimates resulting from the cross-correlation of complex signals are commonly used to generate navigation and scene height measurements. In the presence of noise, the time delay estimates can be ambiguous, containing errors corresponding to an integer number of phase wraps. These ambiguities cause navigation and bathymetry errors and reduce the quality of synthetic aperture imagery. In this article, an algorithm is introduced for the detection and correction of phase wrap errors. The random sample consensus (RANSAC) algorithm is used to fit 1-D and 2-D models to the ambiguous time delay estimates made in the time delay estimation step of redundant phase center (RPC) micronavigation. Phase wrap errors are then corrected by recalculating the phase wrap number using the best-fitting model. The approach is demonstrated using the data collected by the 270&amp;#x2013;330 kHz SAS of the NATO Centre for Maritime Research and Experimentation Minehunting unmanned underwater vehicle for Shallow water Covert Littoral Expeditions. Systems with lower fractional bandwidth were emulated by windowing the bandwidth of the signals to increase the occurrence of phase wrap errors. The time delay estimates were refined using both the RANSAC algorithms using 1-D and 2-D models and the commonly used branch-cuts method. Following qualitative assessment of the smoothness of the full-bandwidth time delay estimates after application of these three methods, the results from the 2-D RANSAC method were chosen as the reference time delay estimates. Comparison with the reference estimates shows that the 1-D and 2-D RANSAC methods outperform the branch-cuts method, with improvements of 29&amp;#x0025;&amp;#x2013;125&amp;#x0025; and 30&amp;#x0025;&amp;#x2013;150&amp;#x0025;, respectively, compared to 16&amp;#x0025;&amp;#x2013;134&amp;#x0025; for the branch-cuts method for this data set.</p

Phase wrap error correction by random sample consensus with application to synthetic aperture sonar micro-navigation

Accurate time delay estimation between signals is crucial for coherent imaging systems such as synthetic aperture sonar (SAS) and synthetic aperture radar (SAR). In such systems, time delay estimates resulting from the cross-correlation of complex signals are commonly used to generate navigation and scene height measurements. In the presence of noise, the time delay estimates can be ambiguous, containing errors corresponding to an integer number of phase wraps. These ambiguities cause navigation and bathymetry errors and reduce the quality of synthetic aperture imagery. In this article, an algorithm is introduced for the detection and correction of phase wrap errors. The random sample consensus (RANSAC) algorithm is used to fit 1-D and 2-D models to the ambiguous time delay estimates made in the time delay estimation step of redundant phase center (RPC) micronavigation. Phase wrap errors are then corrected by recalculating the phase wrap number using the best-fitting model. The approach is demonstrated using the data collected by the 270&amp;#x2013;330 kHz SAS of the NATO Centre for Maritime Research and Experimentation Minehunting unmanned underwater vehicle for Shallow water Covert Littoral Expeditions. Systems with lower fractional bandwidth were emulated by windowing the bandwidth of the signals to increase the occurrence of phase wrap errors. The time delay estimates were refined using both the RANSAC algorithms using 1-D and 2-D models and the commonly used branch-cuts method. Following qualitative assessment of the smoothness of the full-bandwidth time delay estimates after application of these three methods, the results from the 2-D RANSAC method were chosen as the reference time delay estimates. Comparison with the reference estimates shows that the 1-D and 2-D RANSAC methods outperform the branch-cuts method, with improvements of 29&amp;#x0025;&amp;#x2013;125&amp;#x0025; and 30&amp;#x0025;&amp;#x2013;150&amp;#x0025;, respectively, compared to 16&amp;#x0025;&amp;#x2013;134&amp;#x0025; for the branch-cuts method for this data set.</p

Efficient Algorithms for Robust Estimation

One of the most commonly encountered tasks in computer vision is the estimation of model parameters from image measurements. This scenario arises in a variety of applications -- for instance, in the estimation of geometric entities, such as camera pose parameters, from feature matches between images. The main challenge in this task is to handle the problem of outliers -- in other words, data points that do not conform to the model being estimated. It is well known that if these outliers are not properly accounted for, even a single outlier in the data can result in arbitrarily bad model estimates. Due to the widespread prevalence of problems of this nature, the field of robust estimation has been well studied over the years, both in the statistics community as well as in computer vision, leading to the development of popular algorithms like Random Sample Consensus (RANSAC). While recent years have seen exciting advances in this area, a number of important issues still remain open. In this dissertation, we aim to address some of these challenges. The main goal of this dissertation is to advance the state of the art in robust estimation techniques by developing algorithms capable of efficiently and accurately delivering model parameter estimates in the face of noise and outliers. To this end, the first contribution of this work is in the development of a coherent framework for the analysis of RANSAC-based robust estimators, which consolidates various improvements made over the years. In turn, this analysis leads naturally to the development of new techniques that combine the strengths of existing methods, and yields high-performance robust estimation algorithms, including for real-time applications. A second contribution of this dissertation is the development of an algorithm that explicitly characterizes the effects of estimation uncertainty in RANSAC. This uncertainty arises from small-scale measurement noise that affects the data points, and consequently, impacts the accuracy of model parameters. We show that knowledge of this measurement noise can be leveraged to develop an inlier classification scheme that is dependent on the model uncertainty, as opposed to a fixed inlier threshold, as in RANSAC. This has the advantage that, given a model with associated uncertainty, we can immediately identify a set of points that support this solution, which in turn leads to an improvement in computational efficiency. Finally, we have also developed an approach to addresses the issue of the inlier threshold, which is a user-supplied parameter that can vary depending on the estimation problem and the data being processed. Our technique is based on the intuition that the residual errors for good models are in some way consistent with each other, while bad models do not exhibit this consistency. In other words, looking at the relationship between \\subsets of models can reveal useful information about the validity of the models themselves. We show that it is possible to efficiently identify this consistent behaviour by exploiting residual ordering information coupled with simple non-parametric statistical tests, which leads to an effective algorithm for threshold-free robust estimation.Doctor of Philosoph