Real-time robust estimation of vanishing points through nonlinear optimization

Vanishing points are elements of great interest in the computer vision field, since they are the main source of information about the geometry of the scene and the projection process associated to the camera. They have been studied and applied during decades for plane rectification, 3D reconstruction, and mainly auto-calibration tasks. Nevertheless, the literature lacks accurate online solutions for multiple vanishing point estimation. Most strategies focalize on the accuracy, using highly computational demanding iterative procedures. We propose a novel strategy for multiple vanishing point estimation that finds a trade-off between accuracy and efficiency, being able to operate in real time for video sequences. This strategy takes advantage of the temporal coherence of the images of the sequences to reduce the computational load of the processing algorithms while keeping a high level of accuracy due to an optimization process. The key element of the approach is a robust scheme based on the MLESAC algorithm, which is used in a similar way to the EM algorithm. This approach ensures robust and accurate estimations, since we use the MLESAC in combination with a novel error function, based on the angular error between the vanishing point and the image features. To increase the speed of the MLESAC algorithm, the selection of the minimal sample sets is substituted by a random sampling step that takes into account temporal information to provide better initializations. Besides, for the sake of flexibility, the proposed error function has been designed to work using as image features indiscriminately gradient-pixels or line segments. Hence, we increase the range of applications in which our approach can be used, according to the type of information that is available. The results show a real-time system that delivers real-time accurate estimations of multiple vanishing points for online processing, tested in moving camera video sequences of structured scenarios, both indoors and outdoors, such as rooms, corridors, facades, roads, etc

Calibration of a wide angle stereoscopic system

This paper was published in OPTICS LETTERS and is made available as an electronic reprint with the permission of OSA. The paper can be found at the following URL on the OSA website: http://dx.doi.org/10.1364/OL.36.003064. Systematic or multiple reproduction or distribution to multiple locations via electronic or other means is prohibited and is subject to penalties under law.Inaccuracies in the calibration of a stereoscopic system appear with errors in point correspondences between both images and inexact points localization in each image. Errors increase if the stereoscopic system is composed of wide angle lens cameras. We propose a technique where detected points in both images are corrected before estimating the fundamental matrix and the lens distortion models. Since points are corrected first, errors in point correspondences and point localization are avoided. To correct point location in both images, geometrical and epipolar constraints are imposed in a nonlinear minimization problem. Geometrical constraints define the point localization in relation to its neighbors in the same image, and eipolar constraints represent the location of one point referred to its corresponding point in the other image. © 2011 Optical Society of America.Ricolfe Viala, C.; Sánchez Salmerón, AJ.; Martínez Berti, E. (2011). Calibration of a wide angle stereoscopic system. Optics Letters. 36(16):3064-3067. doi:10.1364/OL.36.003064S306430673616Zhang, Z., Ma, H., Guo, T., Zhang, S., & Chen, J. (2011). Simple, flexible calibration of phase calculation-based three-dimensional imaging system. Optics Letters, 36(7), 1257. doi:10.1364/ol.36.001257Longuet-Higgins, H. C. (1981). A computer algorithm for reconstructing a scene from two projections. Nature, 293(5828), 133-135. doi:10.1038/293133a0Ricolfe-Viala, C., & Sanchez-Salmeron, A.-J. (2010). Lens distortion models evaluation. Applied Optics, 49(30), 5914. doi:10.1364/ao.49.005914Armangué, X., & Salvi, J. (2003). Overall view regarding fundamental matrix estimation. Image and Vision Computing, 21(2), 205-220. doi:10.1016/s0262-8856(02)00154-3Devernay, F., & Faugeras, O. (2001). Straight lines have to be straight. Machine Vision and Applications, 13(1), 14-24. doi:10.1007/pl0001326

Bispectrum Inversion with Application to Multireference Alignment

We consider the problem of estimating a signal from noisy circularly-translated versions of itself, called multireference alignment (MRA). One natural approach to MRA could be to estimate the shifts of the observations first, and infer the signal by aligning and averaging the data. In contrast, we consider a method based on estimating the signal directly, using features of the signal that are invariant under translations. Specifically, we estimate the power spectrum and the bispectrum of the signal from the observations. Under mild assumptions, these invariant features contain enough information to infer the signal. In particular, the bispectrum can be used to estimate the Fourier phases. To this end, we propose and analyze a few algorithms. Our main methods consist of non-convex optimization over the smooth manifold of phases. Empirically, in the absence of noise, these non-convex algorithms appear to converge to the target signal with random initialization. The algorithms are also robust to noise. We then suggest three additional methods. These methods are based on frequency marching, semidefinite relaxation and integer programming. The first two methods provably recover the phases exactly in the absence of noise. In the high noise level regime, the invariant features approach for MRA results in stable estimation if the number of measurements scales like the cube of the noise variance, which is the information-theoretic rate. Additionally, it requires only one pass over the data which is important at low signal-to-noise ratio when the number of observations must be large

Rectification from Radially-Distorted Scales

This paper introduces the first minimal solvers that jointly estimate lens distortion and affine rectification from repetitions of rigidly transformed coplanar local features. The proposed solvers incorporate lens distortion into the camera model and extend accurate rectification to wide-angle images that contain nearly any type of coplanar repeated content. We demonstrate a principled approach to generating stable minimal solvers by the Grobner basis method, which is accomplished by sampling feasible monomial bases to maximize numerical stability. Synthetic and real-image experiments confirm that the solvers give accurate rectifications from noisy measurements when used in a RANSAC-based estimator. The proposed solvers demonstrate superior robustness to noise compared to the state-of-the-art. The solvers work on scenes without straight lines and, in general, relax the strong assumptions on scene content made by the state-of-the-art. Accurate rectifications on imagery that was taken with narrow focal length to near fish-eye lenses demonstrate the wide applicability of the proposed method. The method is fully automated, and the code is publicly available at https://github.com/prittjam/repeats.Comment: pre-prin

Robust Estimation and Wavelet Thresholding in Partial Linear Models

This paper is concerned with a semiparametric partially linear regression model with unknown regression coefficients, an unknown nonparametric function for the non-linear component, and unobservable Gaussian distributed random errors. We present a wavelet thresholding based estimation procedure to estimate the components of the partial linear model by establishing a connection between an $l_1$-penalty based wavelet estimator of the nonparametric component and Huber's M-estimation of a standard linear model with outliers. Some general results on the large sample properties of the estimates of both the parametric and the nonparametric part of the model are established. Simulations and a real example are used to illustrate the general results and to compare the proposed methodology with other methods available in the recent literature