Vanishing Point Estimation in Uncalibrated Images with Prior Gravity Direction

We tackle the problem of estimating a Manhattan frame, i.e. three orthogonal vanishing points, and the unknown focal length of the camera, leveraging a prior vertical direction. The direction can come from an Inertial Measurement Unit that is a standard component of recent consumer devices, e.g., smartphones. We provide an exhaustive analysis of minimal line configurations and derive two new 2-line solvers, one of which does not suffer from singularities affecting existing solvers. Additionally, we design a new non-minimal method, running on an arbitrary number of lines, to boost the performance in local optimization. Combining all solvers in a hybrid robust estimator, our method achieves increased accuracy even with a rough prior. Experiments on synthetic and real-world datasets demonstrate the superior accuracy of our method compared to the state of the art, while having comparable runtimes. We further demonstrate the applicability of our solvers for relative rotation estimation. The code is available at https://github.com/cvg/VP-Estimation-with-Prior-Gravity.Comment: Accepted at ICCV 202

Automatic Detection of Calibration Grids in Time-of-Flight Images

It is convenient to calibrate time-of-flight cameras by established methods, using images of a chequerboard pattern. The low resolution of the amplitude image, however, makes it difficult to detect the board reliably. Heuristic detection methods, based on connected image-components, perform very poorly on this data. An alternative, geometrically-principled method is introduced here, based on the Hough transform. The projection of a chequerboard is represented by two pencils of lines, which are identified as oriented clusters in the gradient-data of the image. A projective Hough transform is applied to each of the two clusters, in axis-aligned coordinates. The range of each transform is properly bounded, because the corresponding gradient vectors are approximately parallel. Each of the two transforms contains a series of collinear peaks; one for every line in the given pencil. This pattern is easily detected, by sweeping a dual line through the transform. The proposed Hough-based method is compared to the standard OpenCV detection routine, by application to several hundred time-of-flight images. It is shown that the new method detects significantly more calibration boards, over a greater variety of poses, without any overall loss of accuracy. This conclusion is based on an analysis of both geometric and photometric error.Comment: 11 pages, 11 figures, 1 tabl

Real-time robust estimation of vanishing points through nonlinear optimization

Vanishing points are elements of great interest in the computer vision field, since they are the main source of information about the geometry of the scene and the projection process associated to the camera. They have been studied and applied during decades for plane rectification, 3D reconstruction, and mainly auto-calibration tasks. Nevertheless, the literature lacks accurate online solutions for multiple vanishing point estimation. Most strategies focalize on the accuracy, using highly computational demanding iterative procedures. We propose a novel strategy for multiple vanishing point estimation that finds a trade-off between accuracy and efficiency, being able to operate in real time for video sequences. This strategy takes advantage of the temporal coherence of the images of the sequences to reduce the computational load of the processing algorithms while keeping a high level of accuracy due to an optimization process. The key element of the approach is a robust scheme based on the MLESAC algorithm, which is used in a similar way to the EM algorithm. This approach ensures robust and accurate estimations, since we use the MLESAC in combination with a novel error function, based on the angular error between the vanishing point and the image features. To increase the speed of the MLESAC algorithm, the selection of the minimal sample sets is substituted by a random sampling step that takes into account temporal information to provide better initializations. Besides, for the sake of flexibility, the proposed error function has been designed to work using as image features indiscriminately gradient-pixels or line segments. Hence, we increase the range of applications in which our approach can be used, according to the type of information that is available. The results show a real-time system that delivers real-time accurate estimations of multiple vanishing points for online processing, tested in moving camera video sequences of structured scenarios, both indoors and outdoors, such as rooms, corridors, facades, roads, etc

Minimal Solvers for Single-View Lens-Distorted Camera Auto-Calibration

This paper proposes minimal solvers that use combinations of imaged translational symmetries and parallel scene lines to jointly estimate lens undistortion with either affine rectification or focal length and absolute orientation. We use constraints provided by orthogonal scene planes to recover the focal length. We show that solvers using feature combinations can recover more accurate calibrations than solvers using only one feature type on scenes that have a balance of lines and texture. We also show that the proposed solvers are complementary and can be used together in a RANSAC-based estimator to improve auto-calibration accuracy. State-of-the-art performance is demonstrated on a standard dataset of lens-distorted urban images. The code is available at https://github.com/ylochman/single-view-autocalib

Plane extraction for indoor place recognition

In this paper, we present an image based plane extraction method well suited for real-time operations. Our approach exploits the assumption that the surrounding scene is mainly composed by planes disposed in known directions. Planes are detected from a single image exploiting a voting scheme that takes into account the vanishing lines. Then, candidate planes are validated and merged using a region grow- ing based approach to detect in real-time planes inside an unknown in- door environment. Using the related plane homographies is possible to remove the perspective distortion, enabling standard place recognition algorithms to work in an invariant point of view setup. Quantitative Ex- periments performed with real world images show the effectiveness of our approach compared with a very popular method

CONSAC: Robust Multi-Model Fitting by Conditional Sample Consensus

We present a robust estimator for fitting multiple parametric models of the same form to noisy measurements. Applications include finding multiple vanishing points in man-made scenes, fitting planes to architectural imagery, or estimating multiple rigid motions within the same sequence. In contrast to previous works, which resorted to hand-crafted search strategies for multiple model detection, we learn the search strategy from data. A neural network conditioned on previously detected models guides a RANSAC estimator to different subsets of all measurements, thereby finding model instances one after another. We train our method supervised as well as self-supervised. For supervised training of the search strategy, we contribute a new dataset for vanishing point estimation. Leveraging this dataset, the proposed algorithm is superior with respect to other robust estimators as well as to designated vanishing point estimation algorithms. For self-supervised learning of the search, we evaluate the proposed algorithm on multi-homography estimation and demonstrate an accuracy that is superior to state-of-the-art methods.Comment: CVPR 202