Line geometry and camera autocalibration

We provide a completely new rigorous matrix formulation of the absolute quadratic complex (AQC), given by the set of lines intersecting the absolute conic. The new results include closed-form expressions for the camera intrinsic parameters in terms of the AQC, an algorithm to obtain the dual absolute quadric from the AQC using straightforward matrix operations, and an equally direct computation of a Euclidean-upgrading homography from the AQC. We also completely characterize the 6×6 matrices acting on lines which are induced by a spatial homography. Several algorithmic possibilities arising from the AQC are systematically explored and analyzed in terms of efficiency and computational cost. Experiments include 3D reconstruction from real images

3D facial merging for virtual human reconstruction

There is an increasing need of easy and affordable technologies to automatically generate virtual 3D models from their real counterparts. In particular, 3D human reconstruction has driven the creation of many clever techniques, most of them based on the visual hull (VH) concept. Such techniques do not require expensive hardware; however, they tend to yield 3D humanoids with realistic bodies but mediocre faces, since VH cannot handle concavities. On the other hand, structured light projectors allow to capture very accurate depth data, and thus to reconstruct realistic faces, but they are too expensive to use several of them. We have developed a technique to merge a VH-based 3D mesh of a reconstructed humanoid and the depth data of its face, captured by a single structured light projector. By combining the advantages of both systems in a simple setting, we are able to reconstruct realistic 3D human models with believable faces

Aerial video geo-registration using terrain models from dense and coherent stereo matching

In the context of aerial imagery, one of the first steps toward a coherent processing of the information contained in multiple images is geo-registration, which consists in assigning geographic 3D coordinates to the pixels of the image. This enables accurate alignment and geo-positioning of multiple images, detection of moving objects and fusion of data acquired from multiple sensors. To solve this problem there are different approaches that require, in addition to a precise characterization of the camera sensor, high resolution referenced images or terrain elevation models, which are usually not publicly available or out of date. Building upon the idea of developing technology that does not need a reference terrain elevation model, we propose a geo-registration technique that applies variational methods to obtain a dense and coherent surface elevation model that is used to replace the reference model. The surface elevation model is built by interpolation of scattered 3D points, which are obtained in a two-step process following a classical stereo pipeline: first, coherent disparity maps between image pairs of a video sequence are estimated and then image point correspondences are back-projected. The proposed variational method enforces continuity of the disparity map not only along epipolar lines (as done by previous geo-registration techniques) but also across them, in the full 2D image domain. In the experiments, aerial images from synthetic video sequences have been used to validate the proposed technique

Autocalibration with the Minimum Number of Cameras with Known Pixel Shape

In 3D reconstruction, the recovery of the calibration parameters of the cameras is paramount since it provides metric information about the observed scene, e.g., measures of angles and ratios of distances. Autocalibration enables the estimation of the camera parameters without using a calibration device, but by enforcing simple constraints on the camera parameters. In the absence of information about the internal camera parameters such as the focal length and the principal point, the knowledge of the camera pixel shape is usually the only available constraint. Given a projective reconstruction of a rigid scene, we address the problem of the autocalibration of a minimal set of cameras with known pixel shape and otherwise arbitrarily varying intrinsic and extrinsic parameters. We propose an algorithm that only requires 5 cameras (the theoretical minimum), thus halving the number of cameras required by previous algorithms based on the same constraint. To this purpose, we introduce as our basic geometric tool the six-line conic variety (SLCV), consisting in the set of planes intersecting six given lines of 3D space in points of a conic. We show that the set of solutions of the Euclidean upgrading problem for three cameras with known pixel shape can be parameterized in a computationally efficient way. This parameterization is then used to solve autocalibration from five or more cameras, reducing the three-dimensional search space to a two-dimensional one. We provide experiments with real images showing the good performance of the technique.Comment: 19 pages, 14 figures, 7 tables, J. Math. Imaging Vi

Learning Depth With Very Sparse Supervision

Motivated by the astonishing capabilities of natural intelligent agents and inspired by theories from psychology, this paper explores the idea that perception gets coupled to 3D properties of the world via interaction with the environment. Existing works for depth estimation require either massive amounts of annotated training data or some form of hard-coded geometrical constraint. This paper explores a new approach to learning depth perception requiring neither of those. Specifically, we train a specialized global-local network architecture with what would be available to a robot interacting with the environment: from extremely sparse depth measurements down to even a single pixel per image. From a pair of consecutive images, our proposed network outputs a latent representation of the observer's motion between the images and a dense depth map. Experiments on several datasets show that, when ground truth is available even for just one of the image pixels, the proposed network can learn monocular dense depth estimation up to 22.5% more accurately than state-of-the-art approaches. We believe that this work, despite its scientific interest, lays the foundations to learn depth from extremely sparse supervision, which can be valuable to all robotic systems acting under severe bandwidth or sensing constraints.Comment: Accepted for Publication at the IEEE Robotics and Automation Letters (RA-L) 2020, and International Conference on Intelligent Robots and Systems (IROS) 202

Calibración euclídea a partir de longitudes de segmentos

We address the problem of the recovery of Euclidean structure of a projectively distorted n-dimensional space from the knowledge of the, possibly diverse, lenghts of a set of segments. This problem is relevant, in particular, for Euclidean reconstruction with uncalibrated cameras, extending previously known results in the affine setting. The key concept is the Quadric of Segments (QoS), defined in a higher-dimensional space by the set of segments of a fixed lenght, from which Euclidean structure can be obtained in closed form. We have intended to make a thorough study of the properties of the QoS, including the determination of the minimum number of segments of arbitrary length that determine it and its relationship with the standard geometric objects associated to the Euclidean structure of space. Explicit formulas are given to obtain the dual absolute quadric and the absolute quadratic complex from the QoS. Experiments with real and synthetic images evaluate the performance of the techniques

The design of a robust 3D Reconstruction system for video sequences in non controlled environments

Along this thesis, a novel and robust approach for obtaining 3D models from video sequences captured with hand-held cameras is adressed. This work defines a fully automatic pipeline that is able to deal with diferent types of sequences and acquiring devices. The designed and implemented system follows a divide and conquer approach. An smart frame decimation process reduces the temporal redundancy of the input video sequence and selects the best conditioned frames for the reconstruction step. Next, the video is split into overlapped clips with a fixed and small number of Key-frames. This allows to parallelize the Structure and Motion process which translates into a dramatic reduction in the computational complexity. The short length of the clips allows an intensive search for the best solution at each step of the reconstruction, which improves the overall system performance. The process of feature tracking is embedded within the reconstruction loop for each clip as a difference with other approaches. The last contribution of this thesis is a final registration step that merges all the processed clips to the same coordinate frame. This last step consists on a set of linear algorithms that combine information of the structure (3D points) and motion (cameras) shared by partial reconstructions of the same static scene to more accurately estimate their registration to the same coordinate system. The performance for the presented algorithm as well as for the global system is demonstrated in experiments with real data

Automatic Detection of a calibration pattern for automatic camera calibration

[EN] The number of applications that need to calibrate cameras is increasing. Present methods can calculate camera parameter in a semi-automatic way. Because of that full-automatic methods are being researched in order to save user's time and effort. The algorithm proposed in this article uses a pattern similar to a chessboard. It is automatically found in every image without previous information of the number of files or columns. To do this, a joint analysis of the line set that form the pattern is found through the Hough transform, corners and projective invariants. Some examples and a comparison with other methods are shown.[ES] Cada vez se requieren más aplicaciones en las que es necesaria la calibración de una cámara para poder realizar mediciones sobre las imágenes. En la actualidad se dispone de una serie de algoritmos capaces de obtener estos valores de forma semi-automática, por lo que se investigan métodos para calcularlos de la manera más automática posible ahorrando un gran tiempo a los usuarios. El método que se propone en este artículo utiliza un patrón similar a un tablero de ajedrez, que es encontrado en cada imagen de forma automática sin información previa del número de filas y columnas. Para ello encuentra los conjuntos de líneas que forman el patrón mediante un análisis conjunto de la transformada de Hough, esquinas e invariantes a la transformación de la perspectiva. Se muestran varios ejemplos y su comparación con otros métodos más tradicionales.Este trabajo ha sido realizado parcialmente gracias al apoyo del ministerio español de Ciencia e Innovación a través del CDTI y el proyecto CENIT-VISION 2007-1007.De La Escalera, A.; Armingol, JM.; Pech, JL.; Gómez, JJ. (2010). Detección Automática de un Patrón para la Calibración Automática de Cámaras. Revista Iberoamericana de Automática e Informática industrial. 7(4):83-94. https://doi.org/10.1016/S1697-7912(10)70063-7OJS839474Ahn S. J. Rauh W. Kim S. I. (2001). Circular coded target for automation of optical 3D-measurement and camera calibration. In: International Journal of Pattern Recognition and Artificial Intelligence, 15 (6), pp. 905–919.Bouguet J.Y. Camera calibration toolbox for Matlab. www.vision.caltech/bouguetj/calib_docDouskos V. Kalisperakis I. Karras G. (2007). Automatic calibration of digital cameras using planar chess-board patterns. In: Optical 3-D Measurement Techniques VIII, 1, pp. 132–140.Douskos V. Kalisperakis I. Karras G. Petsa E. (2008). Fully automatic camera calibration using regular planar patterns. In: International Archives of Photogrammetric, Remote Sensing and the Spatial Information Sciences, 37 (5), pp. 21–26.Ronda J. I. Valdés A. Gallego G. (2008). Line geometry and camera autocalibration. In: Journal of Mathematical Imaging and Vision, 32 (2), pp. 193–214.Shu C. Brunton A. and Fiala M. (2003). Automatic grid finding in calibration patterns using Delaunay triangulation. In: Technical Report NRC-46497/ERB-1104. National Research Council, Institute for Information Technology.Tsai R.Y. (1987). A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-theshelf TV cameras and lenses. In: Journal of Robotics and Automation, 3, pp. 323–344.VISIÓN: Comunicaciones de Vídeo de Nueva Generación www.cenit-vision.org/index.php.Wang Z. Wu W. Xu X. Xue D. (2007). Recognition and location of the internal corners of planar checkerboard calibration pattern image. In: Applied mathematics and Computation, 185 (2), pp: 894–906.Yu C. and Peng Q. (2006). Robust recognition of checkerboard pattern for camera calibration. In: Optical Engineering, 45 (9), 093201.Zhang Z. (2000). A fexible new technique for camera calibration. In: IEEE Transactions on Pattern Analysis and Machine Intelligence, 22 (11), pp. 1330–1334.Zhang Z. (2004) Camera Calibration With One-Dimensional Objects. In: IEEE Transactions on Pattern Analysis and Machine Intelligence, 26 (7), pp. 892–899.Zhaoxue C. and Pengfei S. (2004). Efficient method for camera calibration in traffic scenes. In: Electronics Letters, 40, pp: 368–369