1,831 research outputs found
Automatic laser and camera extrinsic calibration for data fusion using road plane
Driving Assistance Systems and Autonomous Driving applications require trustable detections. These demanding requirements need sensor fusion to provide information reliable enough. But data fusion presents the problem of data alignment in both rotation and translation. Laser scanner and video cameras are widely used in sensor fusion. Laser provides operation in darkness, long range detection and accurate measurement but lacks the means for reliable classification due to the limited information provided. The camera provides classification thanks to the amount of data provided but lacks accuracy for measurements and is sensitive to illumination conditions. Data alignment processes require supervised and accurate measurements, that should be performed by experts, or require specific patterns or shapes. This paper presents an algorithm for inter-calibration between the two sensors of our system, requiring only a flat surface for pitch and roll calibration and an obstacle visible for both sensors for determining the yaw. The advantage of this system is that it does not need any particular shape to be located in front of the vehicle apart from a flat surface, which is usually the road. This way, calibration can be achieved at virtually any time without human intervention.This work was supported by Automation Engineering
Department from de La Salle University, Bogotá-Colombia;
Administrative Department of Science, Technology and
Innovation (COLCIENCIAS), Bogotá-Colombia and the
Spanish Government through the Cicyt projects (GRANT
TRA2010-20225-C03-01) and (GRANT TRA 2011-29454-
C03-02)
Multi-FEAT: Multi-Feature Edge AlignmenT for Targetless Camera-LiDAR Calibration
The accurate environment perception of automobiles and UAVs (Unmanned Ariel
Vehicles) relies on the precision of onboard sensors, which require reliable
in-field calibration. This paper introduces a novel approach for targetless
camera-LiDAR extrinsic calibration called Multi-FEAT (Multi-Feature Edge
AlignmenT). Multi-FEAT uses the cylindrical projection model to transform the
2D(Camera)-3D(LiDAR) calibration problem into a 2D-2D calibration problem, and
exploits various LiDAR feature information to supplement the sparse LiDAR point
cloud boundaries. In addition, a feature matching function with a precision
factor is designed to improve the smoothness of the solution space. The
performance of the proposed Multi-FEAT algorithm is evaluated using the KITTI
dataset, and our approach shows more reliable results, as compared with several
existing targetless calibration methods. We summarize our results and present
potential directions for future work
Sensor fusion in driving assistance systems
Mención Internacional en el título de doctorLa vida diaria en los países desarrollados y en vías de desarrollo depende en
gran medida del transporte urbano y en carretera. Esta actividad supone un
coste importante para sus usuarios activos y pasivos en términos de polución
y accidentes, muy habitualmente debidos al factor humano. Los nuevos desarrollos
en seguridad y asistencia a la conducción, llamados Advanced Driving
Assistance Systems (ADAS), buscan mejorar la seguridad en el transporte, y
a medio plazo, llegar a la conducción autónoma.
Los ADAS, al igual que la conducción humana, están basados en sensores
que proporcionan información acerca del entorno, y la fiabilidad de los sensores
es crucial para las aplicaciones ADAS al igual que las capacidades
sensoriales lo son para la conducción humana. Una de las formas de aumentar
la fiabilidad de los sensores es el uso de la Fusión Sensorial, desarrollando
nuevas estrategias para el modelado del entorno de conducción gracias al uso
de diversos sensores, y obteniendo una información mejorada a partid de los
datos disponibles.
La presente tesis pretende ofrecer una solución novedosa para la detección
y clasificación de obstáculos en aplicaciones de automoción, usando fusión
vii
sensorial con dos sensores ampliamente disponibles en el mercado: la cámara
de espectro visible y el escáner láser. Cámaras y láseres son sensores
comúnmente usados en la literatura científica, cada vez más accesibles y listos
para ser empleados en aplicaciones reales. La solución propuesta permite la
detección y clasificación de algunos de los obstáculos comúnmente presentes
en la vía, como son ciclistas y peatones.
En esta tesis se han explorado novedosos enfoques para la detección y clasificación,
desde la clasificación empleando clusters de nubes de puntos obtenidas
desde el escáner láser, hasta las técnicas de domain adaptation para la creación
de bases de datos de imágenes sintéticas, pasando por la extracción inteligente
de clusters y la detección y eliminación del suelo en nubes de puntos.Life in developed and developing countries is highly dependent on road and
urban motor transport. This activity involves a high cost for its active and passive
users in terms of pollution and accidents, which are largely attributable to
the human factor. New developments in safety and driving assistance, called
Advanced Driving Assistance Systems (ADAS), are intended to improve
security in transportation, and, in the mid-term, lead to autonomous driving.
ADAS, like the human driving, are based on sensors, which provide information
about the environment, and sensors’ reliability is crucial for ADAS
applications in the same way the sensing abilities are crucial for human driving.
One of the ways to improve reliability for sensors is the use of Sensor
Fusion, developing novel strategies for environment modeling with the help of
several sensors and obtaining an enhanced information from the combination
of the available data.
The present thesis is intended to offer a novel solution for obstacle detection
and classification in automotive applications using sensor fusion with two
highly available sensors in the market: visible spectrum camera and laser
scanner. Cameras and lasers are commonly used sensors in the scientific
literature, increasingly affordable and ready to be deployed in real world
applications. The solution proposed provides obstacle detection and classification
for some obstacles commonly present in the road, such as pedestrians and bicycles.
Novel approaches for detection and classification have been explored in this
thesis, from point cloud clustering classification for laser scanner, to domain
adaptation techniques for synthetic dataset creation, and including intelligent
clustering extraction and ground detection and removal from point clouds.Programa Oficial de Doctorado en Ingeniería Eléctrica, Electrónica y AutomáticaPresidente: Cristina Olaverri Monreal.- Secretario: Arturo de la Escalera Hueso.- Vocal: José Eugenio Naranjo Hernánde
Multimodal perception for autonomous driving
Mención Internacional en el título de doctorAutonomous driving is set to play an important role among intelligent
transportation systems in the coming decades. The advantages
of its large-scale implementation –reduced accidents, shorter commuting
times, or higher fuel efficiency– have made its development a priority
for academia and industry. However, there is still a long way to
go to achieve full self-driving vehicles, capable of dealing with any
scenario without human intervention. To this end, advances in control,
navigation and, especially, environment perception technologies
are yet required. In particular, the detection of other road users that
may interfere with the vehicle’s trajectory is a key element, since it
allows to model the current traffic situation and, thus, to make decisions
accordingly.
The objective of this thesis is to provide solutions to some of
the main challenges of on-board perception systems, such as extrinsic
calibration of sensors, object detection, and deployment on
real platforms. First, a calibration method for obtaining the relative
transformation between pairs of sensors is introduced, eliminating
the complex manual adjustment of these parameters. The algorithm
makes use of an original calibration pattern and supports LiDARs,
and monocular and stereo cameras. Second, different deep learning
models for 3D object detection using LiDAR data in its bird’s eye
view projection are presented. Through a novel encoding, the use
of architectures tailored to image detection is proposed to process
the 3D information of point clouds in real time. Furthermore, the
effectiveness of using this projection together with image features is
analyzed. Finally, a method to mitigate the accuracy drop of LiDARbased
detection networks when deployed in ad-hoc configurations is
introduced. For this purpose, the simulation of virtual signals mimicking
the specifications of the desired real device is used to generate
new annotated datasets that can be used to train the models.
The performance of the proposed methods is evaluated against
other existing alternatives using reference benchmarks in the field of
computer vision (KITTI and nuScenes) and through experiments in
open traffic with an automated vehicle. The results obtained demonstrate
the relevance of the presented work and its suitability for commercial
use.La conducción autónoma está llamada a jugar un papel importante en
los sistemas inteligentes de transporte de las próximas décadas. Las
ventajas de su implementación a larga escala –disminución de accidentes,
reducción del tiempo de trayecto, u optimización del consumo–
han convertido su desarrollo en una prioridad para la academia y
la industria. Sin embargo, todavía hay un largo camino por delante
hasta alcanzar una automatización total, capaz de enfrentarse a cualquier
escenario sin intervención humana. Para ello, aún se requieren
avances en las tecnologías de control, navegación y, especialmente,
percepción del entorno. Concretamente, la detección de otros usuarios
de la carretera que puedan interferir en la trayectoria del vehículo
es una pieza fundamental para conseguirlo, puesto que permite modelar
el estado actual del tráfico y tomar decisiones en consecuencia.
El objetivo de esta tesis es aportar soluciones a algunos de los
principales retos de los sistemas de percepción embarcados, como
la calibración extrínseca de los sensores, la detección de objetos, y su
despliegue en plataformas reales. En primer lugar, se introduce un
método para la obtención de la transformación relativa entre pares
de sensores, eliminando el complejo ajuste manual de estos parámetros.
El algoritmo hace uso de un patrón de calibración propio y da
soporte a cámaras monoculares, estéreo, y LiDAR. En segundo lugar,
se presentan diferentes modelos de aprendizaje profundo para la detección
de objectos en 3D utilizando datos de escáneres LiDAR en su
proyección en vista de pájaro. A través de una nueva codificación, se
propone la utilización de arquitecturas de detección en imagen para
procesar en tiempo real la información tridimensional de las nubes
de puntos. Además, se analiza la efectividad del uso de esta proyección
junto con características procedentes de imágenes. Por último,
se introduce un método para mitigar la pérdida de precisión de las
redes de detección basadas en LiDAR cuando son desplegadas en
configuraciones ad-hoc. Para ello, se plantea la simulación de señales
virtuales con las características del modelo real que se quiere utilizar,
generando así nuevos conjuntos anotados para entrenar los modelos.
El rendimiento de los métodos propuestos es evaluado frente a
otras alternativas existentes haciendo uso de bases de datos de referencia
en el campo de la visión por computador (KITTI y nuScenes),
y mediante experimentos en tráfico abierto empleando un vehículo
automatizado. Los resultados obtenidos demuestran la relevancia de
los trabajos presentados y su viabilidad para un uso comercial.Programa de Doctorado en Ingeniería Eléctrica, Electrónica y Automática por la Universidad Carlos III de MadridPresidente: Jesús García Herrero.- Secretario: Ignacio Parra Alonso.- Vocal: Gustavo Adolfo Peláez Coronad
Robust Extrinsic Self-Calibration of Camera and Solid State LiDAR
This letter proposes an extrinsic calibration approach for a pair of
monocular camera and prism-spinning solid-state LiDAR. The unique
characteristics of the point cloud measured resulting from the flower-like
scanning pattern is first disclosed as the vacant points, a type of outlier
between foreground target and background objects. Unlike existing method using
only depth continuous measurements, we use depth discontinuous measurements to
retain more valid features and efficiently remove vacant points. The larger
number of detected 3D corners thus contain more robust a priori information
than usual which, together with the 2D corners detected by overlapping cameras
and constrained by the proposed circularity and rectangularity rules, produce
accurate extrinsic estimates. The algorithm is evaluated with real field
experiments adopting both qualitative and quantitative performance criteria,
and found to be superior to existing algorithms. The code is available on
GitHub
Building an Omnidirectional 3D Color Laser Ranging System through a Novel Calibration Method
3D color laser ranging technology plays a crucial role in many applications. This paper develops a new omnidirectional 3D color laser ranging system. It consists of a 2D laser rangefinder (LRF), a color camera, and a rotating platform. Both the 2D LRF and the camera rotate with the rotating platform to collect line point clouds and images synchronously. The line point clouds and the images are then fused into a 3D color point cloud by a novel calibration method of a 2D LRF and a camera based on an improved checkerboard pattern with rectangle holes. In the calibration, boundary constraint and mean approximation are deployed to accurately compute the centers of rectangle holes from the raw sensor data based on data correction. Then, the data association between the 2D LRF and the camera is directly established to determine their geometric mapping relationship. These steps make the calibration process simple, accurate, and reliable. The experiments show that the proposed calibration method is accurate, robust to noise, and suitable for different geometric structures, and the developed 3D color laser ranging system has good performance for both indoor and outdoor scenes
Real-Time fusion of visual images and laser data images for safe navigation in outdoor environments
[EN]In recent years, two dimensional laser range finders mounted on vehicles is becoming a
fruitful solution to achieve safety and environment recognition requirements (Keicher &
Seufert, 2000), (Stentz et al., 2002), (DARPA, 2007). They provide real-time accurate range
measurements in large angular fields at a fixed height above the ground plane, and enable
robots and vehicles to perform more confidently a variety of tasks by fusing images from
visual cameras with range data (Baltzakis et al., 2003). Lasers have normally been used in
industrial surveillance applications to detect unexpected objects and persons in indoor
environments. In the last decade, laser range finder are moving from indoor to outdoor rural
and urban applications for 3D imaging (Yokota et al., 2004), vehicle guidance (Barawid et
al., 2007), autonomous navigation (Garcia-Pérez et al., 2008), and objects recognition and
classification (Lee & Ehsani, 2008), (Edan & Kondo, 2009), (Katz et al., 2010). Unlike
industrial applications, which deal with simple, repetitive and well-defined objects, cameralaser
systems on board off-road vehicles require advanced real-time techniques and
algorithms to deal with dynamic unexpected objects. Natural environments are complex
and loosely structured with great differences among consecutive scenes and scenarios.
Vision systems still present severe drawbacks, caused by lighting variability that depends
on unpredictable weather conditions. Camera-laser objects feature fusion and classification
is still a challenge within the paradigm of artificial perception and mobile robotics in
outdoor environments with the presence of dust, dirty, rain, and extreme temperature and
humidity. Real time relevant objects perception, task driven, is a main issue for subsequent
actions decision in safe unmanned navigation. In comparison with industrial automation
systems, the precision required in objects location is usually low, as it is the speed of most
rural vehicles that operate in bounded and low structured outdoor environments.
To this aim, current work is focused on the development of algorithms and strategies for
fusing 2D laser data and visual images, to accomplish real-time detection and classification
of unexpected objects close to the vehicle, to guarantee safe navigation. Next, class
information can be integrated within the global navigation architecture, in control modules,
such as, stop, obstacle avoidance, tracking or mapping.Section 2 includes a description of the commercial vehicle, robot-tractor DEDALO and the
vision systems on board. Section 3 addresses some drawbacks in outdoor perception.
Section 4 analyses the proposed laser data and visual images fusion method, focused in the
reduction of the visual image area to the region of interest wherein objects are detected by
the laser. Two methods of segmentation are described in Section 5, to extract the shorter area
of the visual image (ROI) resulting from the fusion process. Section 6 displays the colour
based classification results of the largest segmented object in the region of interest. Some
conclusions are outlined in Section 7, and acknowledgements and references are displayed
in Section 8 and Section 9.projects: CICYT- DPI-2006-14497 by the Science
and Innovation Ministry, ROBOCITY2030 I y II: Service Robots-PRICIT-CAM-P-DPI-000176-
0505, and SEGVAUTO: Vehicle Safety-PRICIT-CAM-S2009-DPI-1509 by Madrid State
Government.Peer reviewe
3D Visual Perception for Self-Driving Cars using a Multi-Camera System: Calibration, Mapping, Localization, and Obstacle Detection
Cameras are a crucial exteroceptive sensor for self-driving cars as they are
low-cost and small, provide appearance information about the environment, and
work in various weather conditions. They can be used for multiple purposes such
as visual navigation and obstacle detection. We can use a surround multi-camera
system to cover the full 360-degree field-of-view around the car. In this way,
we avoid blind spots which can otherwise lead to accidents. To minimize the
number of cameras needed for surround perception, we utilize fisheye cameras.
Consequently, standard vision pipelines for 3D mapping, visual localization,
obstacle detection, etc. need to be adapted to take full advantage of the
availability of multiple cameras rather than treat each camera individually. In
addition, processing of fisheye images has to be supported. In this paper, we
describe the camera calibration and subsequent processing pipeline for
multi-fisheye-camera systems developed as part of the V-Charge project. This
project seeks to enable automated valet parking for self-driving cars. Our
pipeline is able to precisely calibrate multi-camera systems, build sparse 3D
maps for visual navigation, visually localize the car with respect to these
maps, generate accurate dense maps, as well as detect obstacles based on
real-time depth map extraction
- …