Building an Omnidirectional 3D Color Laser Ranging System through a Novel Calibration Method

3D color laser ranging technology plays a crucial role in many applications. This paper develops a new omnidirectional 3D color laser ranging system. It consists of a 2D laser rangefinder (LRF), a color camera, and a rotating platform. Both the 2D LRF and the camera rotate with the rotating platform to collect line point clouds and images synchronously. The line point clouds and the images are then fused into a 3D color point cloud by a novel calibration method of a 2D LRF and a camera based on an improved checkerboard pattern with rectangle holes. In the calibration, boundary constraint and mean approximation are deployed to accurately compute the centers of rectangle holes from the raw sensor data based on data correction. Then, the data association between the 2D LRF and the camera is directly established to determine their geometric mapping relationship. These steps make the calibration process simple, accurate, and reliable. The experiments show that the proposed calibration method is accurate, robust to noise, and suitable for different geometric structures, and the developed 3D color laser ranging system has good performance for both indoor and outdoor scenes

Automatic Extrinsic Self-Calibration of Mobile Mapping Systems Based on Geometric 3D Features

Mobile Mapping is an efficient technology to acquire spatial data of the environment. The spatial data is fundamental for applications in crisis management, civil engineering or autonomous driving. The extrinsic calibration of the Mobile Mapping System is a decisive factor that affects the quality of the spatial data. Many existing extrinsic calibration approaches require the use of artificial targets in a time-consuming calibration procedure. Moreover, they are usually designed for a specific combination of sensors and are, thus, not universally applicable. We introduce a novel extrinsic self-calibration algorithm, which is fully automatic and completely data-driven. The fundamental assumption of the self-calibration is that the calibration parameters are estimated the best when the derived point cloud represents the real physical circumstances the best. The cost function we use to evaluate this is based on geometric features which rely on the 3D structure tensor derived from the local neighborhood of each point. We compare different cost functions based on geometric features and a cost function based on the Rényi quadratic entropy to evaluate the suitability for the self-calibration. Furthermore, we perform tests of the self-calibration on synthetic and two different real datasets. The real datasets differ in terms of the environment, the scale and the utilized sensors. We show that the self-calibration is able to extrinsically calibrate Mobile Mapping Systems with different combinations of mapping and pose estimation sensors such as a 2D laser scanner to a Motion Capture System and a 3D laser scanner to a stereo camera and ORB-SLAM2. For the first dataset, the parameters estimated by our self-calibration lead to a more accurate point cloud than two comparative approaches. For the second dataset, which has been acquired via a vehicle-based mobile mapping, our self-calibration achieves comparable results to a manually refined reference calibration, while it is universally applicable and fully automated

External multi-modal imaging sensor calibration for sensor fusion: A review

Multi-modal data fusion has gained popularity due to its diverse applications, leading to an increased demand for external sensor calibration. Despite several proven calibration solutions, they fail to fully satisfy all the evaluation criteria, including accuracy, automation, and robustness. Thus, this review aims to contribute to this growing field by examining recent research on multi-modal imaging sensor calibration and proposing future research directions. The literature review comprehensively explains the various characteristics and conditions of different multi-modal external calibration methods, including traditional motion-based calibration and feature-based calibration. Target-based calibration and targetless calibration are two types of feature-based calibration, which are discussed in detail. Furthermore, the paper highlights systematic calibration as an emerging research direction. Finally, this review concludes crucial factors for evaluating calibration methods and provides a comprehensive discussion on their applications, with the aim of providing valuable insights to guide future research directions. Future research should focus primarily on the capability of online targetless calibration and systematic multi-modal sensor calibration.Ministerio de Ciencia, Innovación y Universidades | Ref. PID2019-108816RB-I0

Impact of different trajectories on extrinsic self-calibration for vehicle-based mobile laser scanning systems

The trend toward further integration of automotive electronic control units functionality into domain control units as well as the rise of computing-intensive driver assistance systems has led to a demand for high-performance automotive computation platforms. These platforms have to fulfill stringent safety requirements. One promising approach is the use of performance computation units in combination with safety controllers in a single control unit. Such systems require adequate communication links between the computation units. While Ethernet is widely used, a high-speed serial link communication protocol supported by an Infineon AURIX safety controller appears to be a promising alternative. In this paper, a high-speed serial link IP core is presented, which enables this type of high-speed serial link communication interface for field-programmable gate array–based computing units. In our test setup, the IP core was implemented in a high-performance Xilinx Zynq UltraScale+, which communicated with an Infineon AURIX via high-speed serial link and Ethernet. The first bandwidth measurements demonstrated that high-speed serial link is an interesting candidate for inter-chip communication, resulting in bandwidths reaching up to 127 Mbit/s using stream transmissions

Registration and Recognition in 3D

The simplest Computer Vision algorithm can tell you what color it sees when you point it at an object, but asking that computer what it is looking at is a much harder problem. Camera and LiDAR (Light Detection And Ranging) sensors generally provide streams pixel of values and sophisticated algorithms must be engineered to recognize objects or the environment. There has been significant effort expended by the computer vision community on recognizing objects in color images; however, LiDAR sensors, which sense depth values for pixels instead of color, have been studied less. Recently we have seen a renewed interest in depth data with the democratization provided by consumer depth cameras. Detecting objects in depth data is more challenging in some ways because of the lack of texture and increased complexity of processing unordered point sets. We present three systems that contribute to solving the object recognition problem from the LiDAR perspective. They are: calibration, registration, and object recognition systems. We propose a novel calibration system that works with both line and raster based LiDAR sensors, and calibrates them with respect to image cameras. Our system can be extended to calibrate LiDAR sensors that do not give intensity information. We demonstrate a novel system that produces registrations between different LiDAR scans by transforming the input point cloud into a Constellation Extended Gaussian Image (CEGI) and then uses this CEGI to estimate the rotational alignment of the scans independently. Finally we present a method for object recognition which uses local (Spin Images) and global (CEGI) information to recognize cars in a large urban dataset. We present real world results from these three systems. Compelling experiments show that object recognition systems can gain much information using only 3D geometry. There are many object recognition and navigation algorithms that work on images; the work we propose in this thesis is more complimentary to those image based methods than competitive. This is an important step along the way to more intelligent robots

Three-Dimensional Biplanar Reconstruction of the Scoliotic Spine for Standard Clinical Setup

Tese de Doutoramento. Engenharia Informática. Faculdade de Engenharia. Universidade do Porto. 201

A branch-and-bound algorithm for checkerboard extraction in camera-laser calibration

We address the problem of camera-to-laserscanner calibration using a checkerboard and multiple imagelaser scan pairs. Distinguishing which laser points measure the checkerboard and which lie on the background is essential to any such system. We formulate the checkerboard extraction as a combinatorial optimization problem with a clear cut objective function. We propose a branch-and-bound technique that deterministically and globally optimizes the objective. Unlike what is available in the literature, the proposed method is not heuristic and does not require assumptions such as constraints on the background or relying on discontinuity of the range measurements to partition the data into line segments. The proposed approach is generic and can be applied to both 3D or 2D laser scanners as well as the cases where multiple checkerboards are present. We demonstrate the effectiveness of the proposed approach by providing numerical simulations as well as experimental results.Alireza Khosravian, Tat-Jun Chin, Ian Rei

LiDAR and Camera Detection Fusion in a Real Time Industrial Multi-Sensor Collision Avoidance System

Collision avoidance is a critical task in many applications, such as ADAS (advanced driver-assistance systems), industrial automation and robotics. In an industrial automation setting, certain areas should be off limits to an automated vehicle for protection of people and high-valued assets. These areas can be quarantined by mapping (e.g., GPS) or via beacons that delineate a no-entry area. We propose a delineation method where the industrial vehicle utilizes a LiDAR {(Light Detection and Ranging)} and a single color camera to detect passive beacons and model-predictive control to stop the vehicle from entering a restricted space. The beacons are standard orange traffic cones with a highly reflective vertical pole attached. The LiDAR can readily detect these beacons, but suffers from false positives due to other reflective surfaces such as worker safety vests. Herein, we put forth a method for reducing false positive detection from the LiDAR by projecting the beacons in the camera imagery via a deep learning method and validating the detection using a neural network-learned projection from the camera to the LiDAR space. Experimental data collected at Mississippi State University's Center for Advanced Vehicular Systems (CAVS) shows the effectiveness of the proposed system in keeping the true detection while mitigating false positives.Comment: 34 page