Infrastructure-based Multi-Camera Calibration using Radial Projections

Multi-camera systems are an important sensor platform for intelligent systems such as self-driving cars. Pattern-based calibration techniques can be used to calibrate the intrinsics of the cameras individually. However, extrinsic calibration of systems with little to no visual overlap between the cameras is a challenge. Given the camera intrinsics, infrastucture-based calibration techniques are able to estimate the extrinsics using 3D maps pre-built via SLAM or Structure-from-Motion. In this paper, we propose to fully calibrate a multi-camera system from scratch using an infrastructure-based approach. Assuming that the distortion is mainly radial, we introduce a two-stage approach. We first estimate the camera-rig extrinsics up to a single unknown translation component per camera. Next, we solve for both the intrinsic parameters and the missing translation components. Extensive experiments on multiple indoor and outdoor scenes with multiple multi-camera systems show that our calibration method achieves high accuracy and robustness. In particular, our approach is more robust than the naive approach of first estimating intrinsic parameters and pose per camera before refining the extrinsic parameters of the system. The implementation is available at https://github.com/youkely/InfrasCal.Comment: ECCV 202

Camera Calibration with Non-Central Local Camera Models

Kamerakalibrierung ist eine wichtige Grundvoraussetzung für viele Computer-Vision-Algorithmen wie Stereo-Vision und visuelle Odometrie. Das Ziel der Kamerakalibrierung besteht darin, sowohl die örtliche Lage der Kameras als auch deren Abbildungsmodell zu bestimmen. Das Abbildungsmodell einer Kamera beschreibt den Zusammenhang zwischen der 3D-Welt und der Bildebene. Aktuell werden häufig einfache globale Kamera-Modelle in einem Kalibrierprozess geschätzt, welcher mit vergleichsweise geringem Aufwand und einer großen Fehlertoleranz durchgeführt werden kann. Um das resultierende Kameramodell zu bewerten, wird in der Regel der Rückprojektionsfehler als Maß herangezogen. Jedoch können auch einfache Kameramodelle, die das Abbildungsverhalten von optischen Systemen nicht präzise beschreiben können, niedrige Rückprojektionsfehler erzielen. Dies führt dazu, dass immer wieder schlecht kalibrierte Kameramodelle nicht als solche identifiziert werden. Um dem entgegen zu wirken, wird in dieser Arbeit ein neues kontinuierliches nicht-zentrales Kameramodell basierend auf B-Splines vorgeschlagen. Dieses Abbildungsmodell ermöglicht es, verschiedene Objektive und nicht-zentrale Verschiebungen, die zum Beispiel durch eine Platzierung der Kamera hinter einer Windschutzscheibe entstehen, akkurat abzubilden. Trotz der allgemeinen Modellierung kann dieses Kameramodell durch einen einfach zu verwendenden Schachbrett-Kalibrierprozess geschätzt werden. Um Kalibrierergebnisse zu bewerten, wird anstelle des mittleren Rückprojektionsfehlers ein Kalibrier-Benchmark vorgeschlagen. Die Grundwahrheit des Kameramodells wird durch ein diskretes Sichtstrahlen-basiertes Modell beschrieben. Um dieses Modell zu schätzen, wird ein Kalibrierprozess vorgestellt, welches ein aktives Display als Ziel verwendet. Dabei wird eine lokale Parametrisierung für die Sichtstrahlen vorgestellt und ein Weg aufgezeigt, die Oberfläche des Displays zusammen mit den intrinsischen Kameraparametern zu schätzen. Durch die Schätzung der Oberfläche wird der mittlere Punkt-zu-Linien-Abstand um einen Faktor von mehr als 20 reduziert. Erst dadurch kann das so geschätzte Kameramodell als Grundwahrheit dienen. Das vorgeschlagene Kameramodell und die dazugehörigen Kalibrierprozesse werden durch eine ausführliche Auswertung in Simulation und in der echten Welt mithilfe des neuen Kalibrier-Benchmarks bewertet. Es wird gezeigt, dass selbst in dem vereinfachten Fall einer ebenen Glasscheibe, die vor der Kamera platziert ist, das vorgeschlagene Modell sowohl einem zentralen als auch einem nicht-zentralen globalen Kameramodell überlegen ist. Am Ende wird die Praxistauglichkeit des vorgeschlagenen Modells bewiesen, indem ein automatisches Fahrzeug kalibriert wird, das mit sechs Kameras ausgestattet ist, welche in unterschiedliche Richtungen zeigen. Der mittlere Rückprojektionsfehler verringert sich durch das neue Modell bei allen Kameras um den Faktor zwei bis drei. Der Kalibrier-Benchmark ermöglicht es in Zukunft, die Ergebnisse verschiedener Kalibrierverfahren miteinander zu vergleichen und die Genauigkeit des geschätzten Kameramodells mithilfe der Grundwahrheit akkurat zu bestimmen. Die Verringerung des Kalibrierfehlers durch das neue vorgeschlagene Kameramodell hilft die Genauigkeit weiterführender Algorithmen wie Stereo-Vision, visuelle Odometrie oder 3D-Rekonstruktion zu erhöhen

Self-calibration of turntable sequences from silhouettes

This paper addresses the problem of recovering both the intrinsic and extrinsic parameters of a camera from the silhouettes of an object in a turntable sequence. Previous silhouette-based approaches have exploited correspondences induced by epipolar tangents to estimate the image invariants under turntable motion and achieved a weak calibration of the cameras. It is known that the fundamental matrix relating any two views in a turntable sequence can be expressed explicitly in terms of the image invariants, the rotation angle, and a fixed scalar. It will be shown that the imaged circular points for the turntable plane can also be formulated in terms of the same image invariants and fixed scalar. This allows the imaged circular points to be recovered directly from the estimated image invariants, and provide constraints for the estimation of the imaged absolute conic. The camera calibration matrix can thus be recovered. A robust method for estimating the fixed scalar from image triplets is introduced, and a method for recovering the rotation angles using the estimated imaged circular points and epipoles is presented. Using the estimated camera intrinsics and extrinsics, a Euclidean reconstruction can be obtained. Experimental results on real data sequences are presented, which demonstrate the high precision achieved by the proposed method. © 2009 IEEE.published_or_final_versio

Camera calibration from surfaces of revolution

This paper addresses the problem of calibrating a pinhole camera from images of a surface of revolution. Camera calibration is the process of determining the intrinsic or internal parameters (i.e., aspect ratio, focal length, and principal point) of a camera, and it is important for both motion estimation and metric reconstruction of 3D models. In this paper, a novel and simple calibration technique is introduced, which is based on exploiting the symmetry of images of surfaces of revolution. Traditional techniques for camera calibration involve taking images of some precisely machined calibration pattern (such as a calibration grid). The use of surfaces of revolution, which are commonly found in daily life (e.g., bowls and vases), makes the process easier as a result of the reduced cost and increased accessibility of the calibration objects. In this paper, it is shown that two images of a surface of revolution will provide enough information for determining the aspect ratio, focal length, and principal point of a camera with fixed intrinsic parameters. The algorithms presented in this paper have been implemented and tested with both synthetic and real data. Experimental results show that the camera calibration method presented here is both practical and accurate.published_or_final_versio

Determination of the Interior Orientation Parameters of a Non-metric Digital Camera for Terrestrial Photogrammetric Applications

AbstractHigh cost of metric photogrammetric cameras has given rise to the utilisation of non-metric digital cameras to generate photogrammetric products in traditional close range or terrestrial photogrammetric applications. For precision photogrammetric applications, the internal metric characteristics of the camera, customarily known as the Interior Orientation Parameters, need to be determined and analysed. The derivation of these parameters is usually achieved by implementing a bundle adjustment with self-calibration procedure. The stability of the Interior Orientation Parameters is an issue in terms of accuracy in digital cameras since they are not built with photogrammetric applications in mind. This study utilised two photogrammetric software (i.e. Photo Modeler and Australis) to calibrate a non-metric digital camera to determine its Interior Orientation Parameters. The camera parameters were obtained using the two software and the Root Mean Square Errors (RMSE) calculated. It was observed that Australis gave a RMSE of 0.2435 and Photo Modeler gave 0.2335, implying that, the calibrated non-metric digital camera is suitable for high precision terrestrial photogrammetric projects. Keywords: Camera Calibration, Interior Orientation Parameters, Non-Metric Digital Camer

Preliminary Study on the 3D Digitization of Millimeter Scale Products by Means of Photogrammetry

AbstractPhotogrammetry is a passive 3D digitization technique, mainly oriented to large sized objects, since its origins are in architectural and civil engineering. With the continuos development of digital imaging hardware and software, photogrammetric applications are involving smaller and smaller fields of view, with some critical aspects such as the depth of field getting narrower. In this conditions the lack of focus becomes important and affects heavily the possibility of accurately calibrate cameras. Bi-dimensional calibration patterns are affected by this problem when the camera principal axis has an angle with the pattern plane higher than a critical value. Moreover, the accuracy of the pattern, in terms of both shape and 3D positions of the targets, becomes critical decreasing the size of the pattern. In this paper the authors address these problems through a comparison of several calibration patterns included into the open source computer vision software library called OpenCV. 3D digitization of a small object is presented to test the best resulting calibration, using a consumer reflex camera equipped with macro lens and extension tube

Towards A Self-calibrating Video Camera Network For Content Analysis And Forensics

Due to growing security concerns, video surveillance and monitoring has received an immense attention from both federal agencies and private firms. The main concern is that a single camera, even if allowed to rotate or translate, is not sufficient to cover a large area for video surveillance. A more general solution with wide range of applications is to allow the deployed cameras to have a non-overlapping field of view (FoV) and to, if possible, allow these cameras to move freely in 3D space. This thesis addresses the issue of how cameras in such a network can be calibrated and how the network as a whole can be calibrated, such that each camera as a unit in the network is aware of its orientation with respect to all the other cameras in the network. Different types of cameras might be present in a multiple camera network and novel techniques are presented for efficient calibration of these cameras. Specifically: (i) For a stationary camera, we derive new constraints on the Image of the Absolute Conic (IAC). These new constraints are shown to be intrinsic to IAC; (ii) For a scene where object shadows are cast on a ground plane, we track the shadows on the ground plane cast by at least two unknown stationary points, and utilize the tracked shadow positions to compute the horizon line and hence compute the camera intrinsic and extrinsic parameters; (iii) A novel solution to a scenario where a camera is observing pedestrians is presented. The uniqueness of formulation lies in recognizing two harmonic homologies present in the geometry obtained by observing pedestrians; (iv) For a freely moving camera, a novel practical method is proposed for its self-calibration which even allows it to change its internal parameters by zooming; and (v) due to the increased application of the pan-tilt-zoom (PTZ) cameras, a technique is presented that uses only two images to estimate five camera parameters. For an automatically configurable multi-camera network, having non-overlapping field of view and possibly containing moving cameras, a practical framework is proposed that determines the geometry of such a dynamic camera network. It is shown that only one automatically computed vanishing point and a line lying on any plane orthogonal to the vertical direction is sufficient to infer the geometry of a dynamic network. Our method generalizes previous work which considers restricted camera motions. Using minimal assumptions, we are able to successfully demonstrate promising results on synthetic as well as on real data. Applications to path modeling, GPS coordinate estimation, and configuring mixed-reality environment are explored

A Full Scale Camera Calibration Technique with Automatic Model Selection – Extension and Validation

This thesis presents work on the testing and development of a complete camera calibration approach which can be applied to a wide range of cameras equipped with normal, wide-angle, fish-eye, or telephoto lenses. The full scale calibration approach estimates all of the intrinsic and extrinsic parameters. The calibration procedure is simple and does not require prior knowledge of any parameters. The method uses a simple planar calibration pattern. Closed-form estimates for the intrinsic and extrinsic parameters are computed followed by nonlinear optimization. Polynomial functions are used to describe the lens projection instead of the commonly used radial model. Statistical information criteria are used to automatically determine the complexity of the lens distortion model. In the first stage experiments were performed to verify and compare the performance of the calibration method. Experiments were performed on a wide range of lenses. Synthetic data was used to simulate real data and validate the performance. Synthetic data was also used to validate the performance of the distortion model selection which uses Information Theoretic Criterion (AIC) to automatically select the complexity of the distortion model. In the second stage work was done to develop an improved calibration procedure which addresses shortcomings of previously developed method. Experiments on the previous method revealed that the estimation of the principal point during calibration was erroneous for lenses with a large focal length. To address this issue the calibration method was modified to include additional methods to accurately estimate the principal point in the initial stages of the calibration procedure. The modified procedure can now be used to calibrate a wide spectrum of imaging systems including telephoto and verifocal lenses. Survey of current work revealed a vast amount of research concentrating on calibrating only the distortion of the camera. In these methods researchers propose methods to calibrate only the distortion parameters and suggest using other popular methods to find the remaining camera parameters. Using this proposed methodology we apply distortion calibration to our methods to separate the estimation of distortion parameters. We show and compare the results with the original method on a wide range of imaging systems

Experimental investigation on camera calibration for 3D photogrammetric scanning of micro-features for micrometric resolution

[EN] Recently, it has been demonstrated that photogrammetry can be used for the measurement of small objects with micro-features, with good results and lower cost, compared to other established techniques such as interferometry, conoscopic holography, and 3D microscopy. Calibration is a critical step in photogrammetry and the classical pinhole camera model has been tested for magnifications lower than 2×. At higher magnification levels, because of the reduction of the depth of field (DOF), images can lead to calibration data with low reprojection errors. However, this could lead to bad results in the 3D reconstruction. With the aim of verifying the possibility of applying the camera model to magnifications higher than 2×, experiments have been conducted using reflex cameras with 60 mm macro lens, equipped with the combination of three extension tubes, corresponding to 2.06, 2.23, and 2.4 magnification levels, respectively. Experiments consisted of repeating calibration five times for each configuration and testing each calibration model, measuring two artifacts with different geometrical complexity. The calibration results have shown good repeatability of a subset of the internal calibration parameters. Despite the differences in the calibration reprojection error (RE), the quality of the photogrammetric 3D models retrieved was stable and satisfying. The experiment demonstrated the possibilities of the photogrammetric system presented, equipped to very high magnification levels, to retrieve accurate 3D reconstruction of micro-features with uncertainties of few micrometers, comparable with industry s expensive state-of-the-art technologies.Percoco, G.; Guerra, MG.; Sánchez Salmerón, AJ.; Galantucci, LM. (2017). Experimental investigation on camera calibration for 3D photogrammetric scanning of micro-features for micrometric resolution. The International Journal of Advanced Manufacturing Technology. 91(9-12):2935-2947. https://doi.org/10.1007/s00170-016-9949-6S29352947919-12Uhlmann E, Mullany B, Biermann D, Rajurkar KP, Hausotte T, Brinksmeier E (2016) Process chains for high-precision components with micro-scale features. CIRP Ann - Manuf Technol 65:549–572. doi: 10.1016/j.cirp.2016.05.001Savio E, De Chiffre L, Schmitt R (2007) Metrology of freeform shaped parts. CIRP Ann - Manuf Technol 56:810–835. doi: 10.1016/j.cirp.2007.10.008Rodríguez-martín M, Lagüela S, González-aguilera D, Rodríguez-gonzálvez P (2015) Optics & Laser Technology Procedure for quality inspection of welds based on macro-photogrammetric three-dimensional reconstruction;73:54–62Xu Z, Toncich D, Stefani S (1999) Vision-based measurement of three-dimensional geometric workpiece properties. Int J Adv Manuf Technol 15:322–331. doi: 10.1007/s001700050074Galantucci LM, Lavecchia F, Percoco G (2013) Multistack close range photogrammetry for low cost submillimeter metrology. J Comput Inf Sci Eng 13:44501. doi: 10.1115/1.4024973Maté González, M.T., Yravedra, J., González-Aguilera, D., Palomeque-González, J.F., Domínguez-Rodrigo, M. Micro-photogrammetric characterization of cut marks on bones (2015) Journal of Archaeological Science, 62, pp. 128-142. doi: 10.1016/j.jas.2015.08.006Brown DC (1971) Close-range camera calibration. Photogramm Eng 37:855–866 doi:10.1.1.14.6358Tang R, Fritsch D (2013) Correlation analysis of camera self-calibration in close range photogrammetry. Photogramm Rec 28:86–95. doi: 10.1111/phor.12009Agisoft LLC (2011) Agisoft PhotoScan User Manual :37.Jcgm JCFGIM (2008) Evaluation of measurement data—guide to the expression of uncertainty in measurement- annex B "general metrological terms"- B.2.14. Int Organ Stand Geneva ISBN 50:134. doi: 10.1373/clinchem.2003.030528Yanagi, H., Chikatsu, H. Performance evaluation of macro lens in digital close range photogrammetry (2009) Proceedings of SPIE - The International Society for Optical Engineering, 7447, art. no. 74470J, doi: 10.1117/12.825817Galantucci LM, Pesce M, Lavecchia F (2015) A stereo photogrammetry scanning methodology, for precise and accurate 3D digitization of small parts with sub-millimeter sized features. CIRP Ann - Manuf Technol 64:507–510. doi: 10.1016/j.cirp.2015.04.016Galantucci LM, Pesce M, Lavecchia F (2015) A powerful scanning methodology for 3D measurements of small parts with complex surfaces and sub millimeter-sized features, based on close range photogrammetry. Precis Eng. doi: 10.1016/j.precisioneng.2015.07.010Percoco G, Sánchez Salmerón AJ (2015) Photogrammetric measurement of 3D freeform millimetre-sized objects with micro features: an experimental validation of the close-range camera calibration model for narrow angles of view. Meas Sci Technol 26:95203. doi: 10.1088/0957-0233/26/9/095203Gallo A, Muzzupappa M, Bruno F (2014) 3D reconstruction of small sized objects from a sequence of multi-focused images. J Cult Herit 15:173–182. doi: 10.1016/j.culher.2013.04.009Stamatopoulos C, Fraser CS, Cronk S (2010) On the self-calibration of long focal length lenses. Int Arch Photogramm Remote Sens Spat Inf Sci Newcastle upon Tyne, UK 2010 XXXVIII:560–564Atkinson KB (1996) Close range photogrammetry and machine vision. Whittles PublishingLuhmann T, Fraser C, Maas HG (2016) Sensor modelling and camera calibration for close-range photogrammetry. ISPRS J Photogramm Remote Sens 115:37–46. doi: 10.1016/j.isprsjprs.2015.10.006Tsai RY (1986) An efficient and accurate camera calibration technique for 3D machine vision. Proc IEEE Conf Comput Vis Pattern Recognition 1986Ricolfe-Viala C, Sanchez-Salmeron A-J (2011) Camera calibration under optimal conditions. Opt Express 19:10769–10775. doi: 10.1364/OE.19.010769Wang L, Wang W, Shen C, Duan F (2016) A convex relaxation optimization algorithm for multi-camera calibration with 1D objects. NeurocomputingRicolfe-Viala C, Sanchez-Salmeron A. Lens distortion models evaluation. Appl Opt 2010;49:5914–5928.Percoco G, Lavecchia F, Salmerón AJS (2015) Preliminary study on the 3D digitization of millimeter scale products by means of photogrammetry. Procedia CIRP 33:257–262. doi: 10.1016/j.procir.2015.06.046Bradski G (2000) The OpenCV Library. Dr Dobb’s J Softw ToolsRicolfe-Viala C, Sanchez-Salmeron A-J (2011) Optimal conditions for camera calibration using a planar template. 2011 18th IEEE. Int Conf Image Process, IEEE 2011:853–856. doi: 10.1109/ICIP.2011.6116691Lowe DG (1999) Object recognition from local scale-invariant features. Proc Seventh IEEE Int Conf Comput Vis 2:1150–1157. doi: 10.1109/ICCV.1999.790410Triggs B, Mclauchlan P, Hartley R, Fitzgibbon A, Triggs B, Mclauchlan P, et al. (2010) Bundle adjustment—a modern synthesis to cite this version: bundle adjustment—a modern synthesisCignoni P, Callieri M, Corsini M, Dellepiane M, Ganovelli F, Ranzuglia G (2008) Meshlab: an open-source mesh processing tool. Eurographics Ital Chapter Conf 2008:129–136Besl P, McKay N (1992) A method for registration of 3-D shapes. IEEE Trans Pattern Anal Mach Intell 14:239–256. doi: 10.1109/34.12179