Building an Omnidirectional 3D Color Laser Ranging System through a Novel Calibration Method

3D color laser ranging technology plays a crucial role in many applications. This paper develops a new omnidirectional 3D color laser ranging system. It consists of a 2D laser rangefinder (LRF), a color camera, and a rotating platform. Both the 2D LRF and the camera rotate with the rotating platform to collect line point clouds and images synchronously. The line point clouds and the images are then fused into a 3D color point cloud by a novel calibration method of a 2D LRF and a camera based on an improved checkerboard pattern with rectangle holes. In the calibration, boundary constraint and mean approximation are deployed to accurately compute the centers of rectangle holes from the raw sensor data based on data correction. Then, the data association between the 2D LRF and the camera is directly established to determine their geometric mapping relationship. These steps make the calibration process simple, accurate, and reliable. The experiments show that the proposed calibration method is accurate, robust to noise, and suitable for different geometric structures, and the developed 3D color laser ranging system has good performance for both indoor and outdoor scenes

Global Optimality via Tight Convex Relaxations for Pose Estimation in Geometric 3D Computer Vision

In this thesis, we address a set of fundamental problems whose core difficulty boils down to optimizing over 3D poses. This includes many geometric 3D registration problems, covering well-known problems with a long research history such as the Perspective-n-Point (PnP) problem and generalizations, extrinsic sensor calibration, or even the gold standard for Structure from Motion (SfM) pipelines: The relative pose problem from corresponding features. Likewise, this is also the case for a close relative of SLAM, Pose Graph Optimization (also commonly known as Motion Averaging in SfM). The crux of this thesis contribution revolves around the successful characterization and development of empirically tight (convex) semidefinite relaxations for many of the aforementioned core problems of 3D Computer Vision. Building upon these empirically tight relaxations, we are able to find and certify the globally optimal solution to these problems with algorithms whose performance ranges as of today from efficient, scalable approaches comparable to fast second-order local search techniques to polynomial time (worst case). So, to conclude, our research reveals that an important subset of core problems that has been historically regarded as hard and thus dealt with mostly in empirical ways, are indeed tractable with optimality guarantees.Artificial Intelligence (AI) drives a lot of services and products we use everyday. But for AI to bring its full potential into daily tasks, with technologies such as autonomous driving, augmented reality or mobile robots, AI needs to be not only intelligent but also perceptive. In particular, the ability to see and to construct an accurate model of the environment is an essential capability to build intelligent perceptive systems. The ideas developed in Computer Vision for the last decades in areas such as Multiple View Geometry or Optimization, put together to work into 3D reconstruction algorithms seem to be mature enough to nurture a range of emerging applications that already employ as of today 3D Computer Vision in the background. However, while there is a positive trend in the use of 3D reconstruction tools in real applications, there are also some fundamental limitations regarding reliability and performance guarantees that may hinder a wider adoption, e.g. in more critical applications involving people's safety such as autonomous navigation. State-of-the-art 3D reconstruction algorithms typically formulate the reconstruction problem as a Maximum Likelihood Estimation (MLE) instance, which entails solving a high-dimensional non-convex non-linear optimization problem. In practice, this is done via fast local optimization methods, that have enabled fast and scalable reconstruction pipelines, yet lack of guarantees on most of the building blocks leaving us with fundamentally brittle pipelines where no guarantees exist

Calibração multi-modal de sensores a bordo do ATLASCAR2

Complex robot systems have several sensors with different modalities. In order to estimate the pose of these various multi-modal sensors, some works propose sequential pairwise calibrations, which have some inherent problems. ATLASCAR2 is an intelligent vehicle with several sensors of different modalities. The main goal of this work is to calibrate all sensors on board the ATLASCAR2. A ROS based interactive and semi-automatic approach, that works for any robot system, even the most complex ones, was developed. After the step of identifying which geometric transformations, between all robot description, should be estimated and collecting the detected data from each sensor, a least-squares optimization occurs to enhance the position and orientation of each one of the robot sensors. Results show that the four sensors simultaneous calibration is as good as the pairwise procedures used with the standard calibration tools, such as the OpenCV ones. In that way, the proposed solution brings a novel and advantageous methodology, since it fits to any complex robot system and calibrates all sensors at the same time.Os mais complexos sistemas robóticos possuem vários sensors de diferentes modalidades. Com o objetivo de se estimar a posição e orientação destes vários sensors multi-modais, existem alguns trabalhos que propõem calibrações sequenciais par a par: calibrações essas com alguns problemas inerentes. ATLASCAR2 é um veículo inteligente com vários sensores de diferentes modalidades. O objetivo principal deste projeto é calibrar todos os sensors a bordo do ATLASCAR2. Foi desenvolvida uma abordagem interativa e semi-automática, que funciona para qualquer robô em ROS, mesmo os mais complexos. Depois da etapa de identificação de quais as transformações geométricas, de entre toda a descrição do robô, devem ser estimadas e da coleção da informação recolhida por cada sensor, optimiza-se, através do método dos mínimos quadrados, os parâmetros de posição e orientação de cada um dos sensores do robô. Os resultados mostram que a calibração simultânea dos quatro sensores é tão boa quanto os procedimentos par a par usados pelas ferramentas de calibração padrão, como as do OpenCV. Assim sendo, a solução proposta apresenta uma nova e vantajosa metodologia, uma vez que se adequa a qualquer sistema robótico complexo e que calibra todos os seus sensors ao mesmo tempo.Mestrado em Engenharia Mecânic

Impacto da calibração num LiDAR baseado em visão estereoscópica

Every year 1.3 million people die due to road accidents. Given that the main culprit is human error, autonomous driving is the path to avert and prevent these numbers. An autonomous vehicle must be able to perceive its surroundings, therefore requiring vision sensors. Of the many kinds of vision sensors available, the three main automotive vision sensors are cameras, RADAR and LiDAR. LiDARs have the unique capability of capturing a high-resolution point cloud, thus enabling 3D object detection. However, current LiDAR technology is still immature and expensive, which makes it unattractive to the automotive market. We propose an alternative LiDAR concept – the LiDART – that is able to generate a point cloud simply resorting to stereoscopic vision and dot projection. LiDART takes advantage of mass-produced components such as a dot pattern projector and a stereoscopic camera rig, thus inherently overcoming problems in cost and maturity. Nonetheless, LiDART has four key challenges: noise, correspondence, centroiding and calibration. This thesis focuses on the calibration aspects of LiDART and aims to investigate the systematic error introduced by standard calibration techniques. In this work, the quality of stereoscopic calibration was assessed both experimentally and numerically. The experimental validation consisted in assembling a prototype and calibrating it using standard calibration techniques for stereoscopic vision. Calibration quality was assessed by estimating the distance to a target. As for numerical assessment, a simulation tool was developed to cross-validate most experimental results. The obtained results show that standard calibration techniques result in a considerable systematic error, reaching 30% of the correct distance. Nonetheless, the estimated error depends monotonically on distance. Consequently, the systematic error can be significantly reduced if better calibration methods, specifically designed for the application at hand, are used in the future.Todos os anos 1.3 milhões de pessoas perdem a vida devido a acidentes de viação. Dado que a principal razão por detrás destes trágicos números é o erro humano, o caminho para prevenir perder tantas vidas passa pela condução autónoma. Um veículo autónomo deve ser capaz de observar o cenário envolvente. Para tal, são necessários sensores de visão. Dos vários sensores de visão disponiveis no mercado, os três principais sensores de visão automotivos são a câmara, o RADAR e o Li- DAR. O LiDAR tem a capacidade única de capturar uma nuvem de pontos com alta resolução, permitindo assim deteção de objetos em 3D. Contudo, a tecnologia por detrás de um LiDAR é atualmente dispendiosa e imatura, o que tem dificultado a adoção por parte de fabricantes de automóveis. Este trabalho propõe um conceito de LiDAR alternativo – o LiDART – capaz de gerar uma nuvem de pontos recorrendo simplesmente a visão estereoscópica e à projeção de pontos. O LiDART tem a vantagem de se basear em componentes produzidos em massa, tais como um projector de pontos e uma câmara estereoscópica, ultrapassando assim os problemas de custo e maturidade. Não obstante, o LiDART tem quatro desafios principais: ruído, correspondência, estimação de centróide e calibração. Esta tese foca-se nas características de calibração do LiDART, tendo como objectivo investigar o erro sistemático introduzido por técnicas de calibração comuns. A qualidade da calibração foi avaliada experimentalmente e numericamente. A validação experimental consistiu em montar um protótipo e calibrá-lo de várias maneiras. A qualidade da calibração foi então avaliada através da estimação da distância a um alvo. Relativamente à parte numérica, desenvolveu-se uma ferramenta de simulação para validar grande parte dos resultados experimentais. Os resultados obtidos mostram que técnicas de calibração comuns resultam num erro sistemático considerável, chegando a 30% da distância correta. Porém, o erro de estimação varia monotonicamente com a distância. Consequentemente, o erro sistemático pode ser reduzido significativamente se melhores métodos de calibração, especialmente pensados para a aplicação em questão, forem aplicados no futuro.Mestrado em Engenharia Eletrónica e Telecomunicaçõe

Convex Global 3D Registration with Lagrangian Duality

The registration of 3D models by a Euclidean transformation is a fundamental task at the core of many application in computer vision. This problem is non-convex due to the presence of rotational constraints, making traditional local optimization methods prone to getting stuck in local minima. This paper addresses finding the globally optimal transformation in various 3D registration problems by a unified formulation that integrates common geometric registration modalities (namely point-to-point, point-to-line and point-to-plane). This formulation renders the optimization problem independent of both the number and nature of the correspondences. The main novelty of our proposal is the introduction of a strengthened Lagrangian dual relaxation for this problem, which surpasses previous similar approaches [32] in effectiveness. In fact, even though with no theoretical guarantees, exhaustive empirical evaluation in both synthetic and real experiments always resulted on a tight relaxation that allowed to recover a guaranteed globally optimal solution by exploiting duality theory. Thus, our approach allows for effectively solving the 3D registration with global optimality guarantees while running at a fraction of the time for the state-of-the-art alternative [34], based on a more computationally intensive Branch and Bound method.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech