1,558 research outputs found
Survey on LiDAR Perception in Adverse Weather Conditions
Autonomous vehicles rely on a variety of sensors to gather information about
their surrounding. The vehicle's behavior is planned based on the environment
perception, making its reliability crucial for safety reasons. The active LiDAR
sensor is able to create an accurate 3D representation of a scene, making it a
valuable addition for environment perception for autonomous vehicles. Due to
light scattering and occlusion, the LiDAR's performance change under adverse
weather conditions like fog, snow or rain. This limitation recently fostered a
large body of research on approaches to alleviate the decrease in perception
performance. In this survey, we gathered, analyzed, and discussed different
aspects on dealing with adverse weather conditions in LiDAR-based environment
perception. We address topics such as the availability of appropriate data, raw
point cloud processing and denoising, robust perception algorithms and sensor
fusion to mitigate adverse weather induced shortcomings. We furthermore
identify the most pressing gaps in the current literature and pinpoint
promising research directions.Comment: published at IEEE IV 202
Efficient-VRNet: An Exquisite Fusion Network for Riverway Panoptic Perception based on Asymmetric Fair Fusion of Vision and 4D mmWave Radar
Panoptic perception is essential to unmanned surface vehicles (USVs) for
autonomous navigation. The current panoptic perception scheme is mainly based
on vision only, that is, object detection and semantic segmentation are
performed simultaneously based on camera sensors. Nevertheless, the fusion of
camera and radar sensors is regarded as a promising method which could
substitute pure vision methods, but almost all works focus on object detection
only. Therefore, how to maximize and subtly fuse the features of vision and
radar to improve both detection and segmentation is a challenge. In this paper,
we focus on riverway panoptic perception based on USVs, which is a considerably
unexplored field compared with road panoptic perception. We propose
Efficient-VRNet, a model based on Contextual Clustering (CoC) and the
asymmetric fusion of vision and 4D mmWave radar, which treats both vision and
radar modalities fairly. Efficient-VRNet can simultaneously perform detection
and segmentation of riverway objects and drivable area segmentation.
Furthermore, we adopt an uncertainty-based panoptic perception training
strategy to train Efficient-VRNet. In the experiments, our Efficient-VRNet
achieves better performances on our collected dataset than other uni-modal
models, especially in adverse weather and environment with poor lighting
conditions. Our code and models are available at
\url{https://github.com/GuanRunwei/Efficient-VRNet}
Benchmarking the Robustness of LiDAR Semantic Segmentation Models
When using LiDAR semantic segmentation models for safety-critical
applications such as autonomous driving, it is essential to understand and
improve their robustness with respect to a large range of LiDAR corruptions. In
this paper, we aim to comprehensively analyze the robustness of LiDAR semantic
segmentation models under various corruptions. To rigorously evaluate the
robustness and generalizability of current approaches, we propose a new
benchmark called SemanticKITTI-C, which features 16 out-of-domain LiDAR
corruptions in three groups, namely adverse weather, measurement noise and
cross-device discrepancy. Then, we systematically investigate 11 LiDAR semantic
segmentation models, especially spanning different input representations (e.g.,
point clouds, voxels, projected images, and etc.), network architectures and
training schemes. Through this study, we obtain two insights: 1) We find out
that the input representation plays a crucial role in robustness. Specifically,
under specific corruptions, different representations perform variously. 2)
Although state-of-the-art methods on LiDAR semantic segmentation achieve
promising results on clean data, they are less robust when dealing with noisy
data. Finally, based on the above observations, we design a robust LiDAR
segmentation model (RLSeg) which greatly boosts the robustness with simple but
effective modifications. It is promising that our benchmark, comprehensive
analysis, and observations can boost future research in robust LiDAR semantic
segmentation for safety-critical applications.Comment: IJCV-2024. The benchmark will be made available at
https://yanx27.github.io/RobustLidarSeg
LIDAR-Camera Fusion for Road Detection Using Fully Convolutional Neural Networks
In this work, a deep learning approach has been developed to carry out road
detection by fusing LIDAR point clouds and camera images. An unstructured and
sparse point cloud is first projected onto the camera image plane and then
upsampled to obtain a set of dense 2D images encoding spatial information.
Several fully convolutional neural networks (FCNs) are then trained to carry
out road detection, either by using data from a single sensor, or by using
three fusion strategies: early, late, and the newly proposed cross fusion.
Whereas in the former two fusion approaches, the integration of multimodal
information is carried out at a predefined depth level, the cross fusion FCN is
designed to directly learn from data where to integrate information; this is
accomplished by using trainable cross connections between the LIDAR and the
camera processing branches.
To further highlight the benefits of using a multimodal system for road
detection, a data set consisting of visually challenging scenes was extracted
from driving sequences of the KITTI raw data set. It was then demonstrated
that, as expected, a purely camera-based FCN severely underperforms on this
data set. A multimodal system, on the other hand, is still able to provide high
accuracy. Finally, the proposed cross fusion FCN was evaluated on the KITTI
road benchmark where it achieved excellent performance, with a MaxF score of
96.03%, ranking it among the top-performing approaches
RADIATE: A Radar Dataset for Automotive Perception in Bad Weather
Datasets for autonomous cars are essential for the development and
benchmarking of perception systems. However, most existing datasets are
captured with camera and LiDAR sensors in good weather conditions. In this
paper, we present the RAdar Dataset In Adverse weaThEr (RADIATE), aiming to
facilitate research on object detection, tracking and scene understanding using
radar sensing for safe autonomous driving. RADIATE includes 3 hours of
annotated radar images with more than 200K labelled road actors in total, on
average about 4.6 instances per radar image. It covers 8 different categories
of actors in a variety of weather conditions (e.g., sun, night, rain, fog and
snow) and driving scenarios (e.g., parked, urban, motorway and suburban),
representing different levels of challenge. To the best of our knowledge, this
is the first public radar dataset which provides high-resolution radar images
on public roads with a large amount of road actors labelled. The data collected
in adverse weather, e.g., fog and snowfall, is unique. Some baseline results of
radar based object detection and recognition are given to show that the use of
radar data is promising for automotive applications in bad weather, where
vision and LiDAR can fail. RADIATE also has stereo images, 32-channel LiDAR and
GPS data, directed at other applications such as sensor fusion, localisation
and mapping. The public dataset can be accessed at
http://pro.hw.ac.uk/radiate/.Comment: Accepted at IEEE International Conference on Robotics and Automation
2021 (ICRA 2021
- …