Monocular Vision based Crowdsourced 3D Traffic Sign Positioning with Unknown Camera Intrinsics and Distortion Coefficients

Autonomous vehicles and driver assistance systems utilize maps of 3D semantic landmarks for improved decision making. However, scaling the mapping process as well as regularly updating such maps come with a huge cost. Crowdsourced mapping of these landmarks such as traffic sign positions provides an appealing alternative. The state-of-the-art approaches to crowdsourced mapping use ground truth camera parameters, which may not always be known or may change over time. In this work, we demonstrate an approach to computing 3D traffic sign positions without knowing the camera focal lengths, principal point, and distortion coefficients a priori. We validate our proposed approach on a public dataset of traffic signs in KITTI. Using only a monocular color camera and GPS, we achieve an average single journey relative and absolute positioning accuracy of 0.26 m and 1.38 m, respectively.Comment: Accepted at 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC

Hierarchical structure-and-motion recovery from uncalibrated images

This paper addresses the structure-and-motion problem, that requires to find camera motion and 3D struc- ture from point matches. A new pipeline, dubbed Samantha, is presented, that departs from the prevailing sequential paradigm and embraces instead a hierarchical approach. This method has several advantages, like a provably lower computational complexity, which is necessary to achieve true scalability, and better error containment, leading to more stability and less drift. Moreover, a practical autocalibration procedure allows to process images without ancillary information. Experiments with real data assess the accuracy and the computational efficiency of the method.Comment: Accepted for publication in CVI

Crowdsourced 3D Mapping: A Combined Multi-View Geometry and Self-Supervised Learning Approach

The ability to efficiently utilize crowdsourced visual data carries immense potential for the domains of large scale dynamic mapping and autonomous driving. However, state-of-the-art methods for crowdsourced 3D mapping assume prior knowledge of camera intrinsics. In this work, we propose a framework that estimates the 3D positions of semantically meaningful landmarks such as traffic signs without assuming known camera intrinsics, using only monocular color camera and GPS. We utilize multi-view geometry as well as deep learning based self-calibration, depth, and ego-motion estimation for traffic sign positioning, and show that combining their strengths is important for increasing the map coverage. To facilitate research on this task, we construct and make available a KITTI based 3D traffic sign ground truth positioning dataset. Using our proposed framework, we achieve an average single-journey relative and absolute positioning accuracy of 39cm and 1.26m respectively, on this dataset.Comment: Accepted at 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS

On Constant Focal Length Self-Calibration From Multiple Views

International audienc

Vision 3D multi-images : contribution à l’obtention de solutions globales par optimisation polynomiale et théorie des moments

L’objectif général de cette thèse est d’appliquer une méthode d’optimisation polynomiale basée sur la théorie des moments à certains problèmes de vision artificielle. Ces problèmes sont en général non convexes et classiquement résolus à l’aide de méthodes d’optimisation locales Ces techniques ne convergent généralement pas vers le minimum global et nécessitent de fournir une estimée initiale proche de la solution exacte. Les méthodes d’optimisation globale permettent d’éviter ces inconvénients. L’optimisation polynomiale basée sur la théorie des moments présente en outre l’avantage de prendre en compte des contraintes. Dans cette thèse nous étendrons cette méthode aux problèmes de minimisation d’une somme d’un grand nombre de fractions rationnelles. De plus, sous certaines hypothèses de "faible couplage" ou de "parcimonie" des variables du problème, nous montrerons qu’il est possible de considérer un nombre important de variables tout en conservant des temps de calcul raisonnables. Enfin nous appliquerons les méthodes proposées aux problèmes de vision par ordinateur suivants : minimisation des distorsions projectives induites par le processus de rectification d’images, estimation de la matrice fondamentale, reconstruction 3D multi-vues avec et sans distorsions radiales. ABSTRACT : The overall objective of this thesis is to apply a polynomial optimization method, based on moments theory, on some vision problems. These problems are often nonconvex and they are classically solved using local optimization methods. Without additional hypothesis, these techniques don’t converge to the global minimum and need to provide an initial estimate close to the exact solution. Global optimization methods overcome this drawback. Moreover, the polynomial optimization based on moments theory could take into account particular constraints. In this thesis, we extend this method to the problems of minimizing a sum of many rational functions. In addition, under particular assumptions of "sparsity", we show that it is possible to deal with a large number of variables while maintaining reasonable computation times. Finally, we apply these methods to particular computer vision problems: minimization of projective distortions due to image rectification process, Fundamental matrix estimation, and multi-view 3D reconstruction with and without radial distortions

Vision 3D multi-images (contribution à l'obtention de solutions globales par optimisation polynomiale et théorie des moments)

L objectif général de cette thèse est d appliquer une méthode d optimisation polynomiale basée sur la théorie des moments à certains problèmes de vision artificielle. Ces problèmes sont en général non convexes et classiquement résolus à l aide de méthodes d optimisation locales Ces techniques ne convergent généralement pas vers le minimum global et nécessitent de fournir une estimée initiale proche de la solution exacte. Les méthodes d optimisation globale permettent d éviter ces inconvénients. L optimisation polynomiale basée sur la théorie des moments présente en outre l avantage de prendre en compte des contraintes. Dans cette thèse nous étendrons cette méthode aux problèmes de minimisation d une somme d un grand nombre de fractions rationnelles. De plus, sous certaines hypothèses de "faible couplage" ou de "parcimonie" des variables du problème, nous montrerons qu il est possible de considérer un nombre important de variables tout en conservant des temps de calcul raisonnables. Enfin nous appliquerons les méthodes proposées aux problèmes de vision par ordinateur suivants : minimisation des distorsions projectives induites par le processus de rectification d images, estimation de la matrice fondamentale, reconstruction 3D multi-vues avec et sans distorsions radiales.The overall objective of this thesis is to apply a polynomial optimization method, based on moments theory, on some vision problems. These problems are often nonconvex and they are classically solved using local optimization methods. Without additional hypothesis, these techniques don t converge to the global minimum and need to provide an initial estimate close to the exact solution. Global optimization methods overcome this drawback. Moreover, the polynomial optimization based on moments theory could take into account particular constraints. In this thesis, we extend this method to the problems of minimizing a sum of many rational functions. In addition, under particular assumptions of "sparsity", we show that it is possible to deal with a large number of variables while maintaining reasonable computation times. Finally, we apply these methods to particular computer vision problems: minimization of projective distortions due to image rectification process, Fundamental matrix estimation, and multi-view 3D reconstruction with and without radial distortions.TOULOUSE-ENSIACET (315552325) / SudocSudocFranceF