3,098 research outputs found
WaterScenes: A Multi-Task 4D Radar-Camera Fusion Dataset and Benchmark for Autonomous Driving on Water Surfaces
Autonomous driving on water surfaces plays an essential role in executing
hazardous and time-consuming missions, such as maritime surveillance, survivors
rescue, environmental monitoring, hydrography mapping and waste cleaning. This
work presents WaterScenes, the first multi-task 4D radar-camera fusion dataset
for autonomous driving on water surfaces. Equipped with a 4D radar and a
monocular camera, our Unmanned Surface Vehicle (USV) proffers all-weather
solutions for discerning object-related information, including color, shape,
texture, range, velocity, azimuth, and elevation. Focusing on typical static
and dynamic objects on water surfaces, we label the camera images and radar
point clouds at pixel-level and point-level, respectively. In addition to basic
perception tasks, such as object detection, instance segmentation and semantic
segmentation, we also provide annotations for free-space segmentation and
waterline segmentation. Leveraging the multi-task and multi-modal data, we
conduct numerous experiments on the single modality of radar and camera, as
well as the fused modalities. Results demonstrate that 4D radar-camera fusion
can considerably enhance the robustness of perception on water surfaces,
especially in adverse lighting and weather conditions. WaterScenes dataset is
public on https://waterscenes.github.io
Human Semantic Segmentation using Millimeter-Wave Radar Sparse Point Clouds
This paper presents a framework for semantic segmentation on sparse
sequential point clouds of millimeter-wave radar. Compared with cameras and
lidars, millimeter-wave radars have the advantage of not revealing privacy,
having a strong anti-interference ability, and having long detection distance.
The sparsity and capturing temporal-topological features of mmWave data is
still a problem. However, the issue of capturing the temporal-topological
coupling features under the human semantic segmentation task prevents previous
advanced segmentation methods (e.g PointNet, PointCNN, Point Transformer) from
being well utilized in practical scenarios. To address the challenge caused by
the sparsity and temporal-topological feature of the data, we (i) introduce
graph structure and topological features to the point cloud, (ii) propose a
semantic segmentation framework including a global feature-extracting module
and a sequential feature-extracting module. In addition, we design an efficient
and more fitting loss function for a better training process and segmentation
results based on graph clustering. Experimentally, we deploy representative
semantic segmentation algorithms (Transformer, GCNN, etc.) on a custom dataset.
Experimental results indicate that our model achieves mean accuracy on the
custom dataset by and outperforms the state-of-the-art
algorithms. Moreover, to validate the model's robustness, we deploy our model
on the well-known S3DIS dataset. On the S3DIS dataset, our model achieves mean
accuracy by , outperforming baseline algorithms
Machine learning applied to radar data: classification and semantic instance segmentation of moving road users
Classification and semantic instance segmentation applications are rarely considered for automotive radar sensors. In current implementations, objects have to be
tracked over time before some semantic information can be extracted. In this thesis,
data from a network of 77 GHz automotive radar sensors is used to construct, train
and evaluate machine learning algorithms for the classification of moving road
users. The classification step is deliberately performed early in the process chain so
that a subsequent tracking algorithm can benefit from this extra information. For
this purpose, a large data set with real-world scenarios from about 5 h of driving
was recorded and annotated.
Given that the point clouds measured by the radar sensors are both sparse and
noisy, the proposed methods have to be sensitive to those features that discern the
individual classes from each other and at the same time, they have to be robust to
outliers and measurement errors. Two groups of applications are considered: classi-
fication of clustered data and semantic (instance) segmentation of whole scenes.
In the first category, specifically designed density-based clustering algorithms are
used to group individual measurements to objects. These objects are then used
either as input to a manual feature extraction step or as input to a neural network,
which operates directly on the bare input points. Different classifiers are trained
and evaluated on these input data.
For the algorithms of the second category, the measurements of a whole scene
are used as input, so that the clustering step becomes obsolete. A newly designed
recurrent neural network for instance segmentation of point clouds is utilized. This
approach outperforms all of the other proposed methods and exceeds the baseline
score by about ten percentage points.
In additional experiments, the performance of human test candidates on the same
task is analyzed. This study shows that temporal correlations in the data are of
great use for the test candidates, who are nevertheless outrun by the recurrent
network
A LiDAR Point Cloud Generator: from a Virtual World to Autonomous Driving
3D LiDAR scanners are playing an increasingly important role in autonomous
driving as they can generate depth information of the environment. However,
creating large 3D LiDAR point cloud datasets with point-level labels requires a
significant amount of manual annotation. This jeopardizes the efficient
development of supervised deep learning algorithms which are often data-hungry.
We present a framework to rapidly create point clouds with accurate point-level
labels from a computer game. The framework supports data collection from both
auto-driving scenes and user-configured scenes. Point clouds from auto-driving
scenes can be used as training data for deep learning algorithms, while point
clouds from user-configured scenes can be used to systematically test the
vulnerability of a neural network, and use the falsifying examples to make the
neural network more robust through retraining. In addition, the scene images
can be captured simultaneously in order for sensor fusion tasks, with a method
proposed to do automatic calibration between the point clouds and captured
scene images. We show a significant improvement in accuracy (+9%) in point
cloud segmentation by augmenting the training dataset with the generated
synthesized data. Our experiments also show by testing and retraining the
network using point clouds from user-configured scenes, the weakness/blind
spots of the neural network can be fixed
Pre-Trained Driving in Localized Surroundings with Semantic Radar Information and Machine Learning
Entlang der Signalverarbeitungskette von Radar Detektionen bis zur Fahrzeugansteuerung, diskutiert diese Arbeit eine semantischen Radar Segmentierung, einen darauf aufbauenden Radar SLAM, sowie eine im Verbund realisierte autonome Parkfunktion. Die Radarsegmentierung der (statischen) Umgebung wird durch ein Radar-spezifisches neuronales Netzwerk RadarNet erreicht. Diese Segmentierung ermöglicht die Entwicklung des semantischen Radar Graph-SLAM SERALOC. Auf der Grundlage der semantischen Radar SLAM Karte wird eine beispielhafte autonome Parkfunktionalität in einem realen Versuchsträger umgesetzt.
Entlang eines aufgezeichneten Referenzfades parkt die Funktion ausschließlich auf Basis der Radar Wahrnehmung mit bisher unerreichter Positioniergenauigkeit.
Im ersten Schritt wird ein Datensatz von 8.2 · 10^6 punktweise semantisch gelabelten Radarpunktwolken über eine Strecke von 2507.35m generiert. Es sind keine vergleichbaren Datensätze dieser Annotationsebene und Radarspezifikation öffentlich verfügbar. Das überwachte
Training der semantischen Segmentierung RadarNet erreicht 28.97% mIoU auf sechs Klassen.
Außerdem wird ein automatisiertes Radar-Labeling-Framework SeRaLF vorgestellt, welches das Radarlabeling multimodal mittels Referenzkameras und LiDAR unterstützt.
Für die kohärente Kartierung wird ein Radarsignal-Vorfilter auf der Grundlage einer Aktivierungskarte entworfen, welcher Rauschen und andere dynamische Mehrwegreflektionen unterdrückt. Ein speziell für Radar angepasstes Graph-SLAM-Frontend mit Radar-Odometrie
Kanten zwischen Teil-Karten und semantisch separater NDT Registrierung setzt die vorgefilterten semantischen Radarscans zu einer konsistenten metrischen Karte zusammen. Die Kartierungsgenauigkeit und die Datenassoziation werden somit erhöht und der erste semantische Radar Graph-SLAM für beliebige statische Umgebungen realisiert.
Integriert in ein reales Testfahrzeug, wird das Zusammenspiel der live RadarNet Segmentierung und des semantischen Radar Graph-SLAM anhand einer rein Radar-basierten autonomen Parkfunktionalität evaluiert. Im Durchschnitt über 42 autonome Parkmanöver
(∅3.73 km/h) bei durchschnittlicher Manöverlänge von ∅172.75m wird ein Median absoluter Posenfehler von 0.235m und End-Posenfehler von 0.2443m erreicht, der vergleichbare
Radar-Lokalisierungsergebnisse um ≈ 50% übertrifft. Die Kartengenauigkeit von veränderlichen, neukartierten Orten über eine Kartierungsdistanz von ∅165m ergibt eine ≈ 56%-ige Kartenkonsistenz bei einer Abweichung von ∅0.163m. Für das autonome Parken wurde ein gegebener Trajektorienplaner und Regleransatz verwendet
Complexer-YOLO: Real-Time 3D Object Detection and Tracking on Semantic Point Clouds
Accurate detection of 3D objects is a fundamental problem in computer vision
and has an enormous impact on autonomous cars, augmented/virtual reality and
many applications in robotics. In this work we present a novel fusion of neural
network based state-of-the-art 3D detector and visual semantic segmentation in
the context of autonomous driving. Additionally, we introduce
Scale-Rotation-Translation score (SRTs), a fast and highly parameterizable
evaluation metric for comparison of object detections, which speeds up our
inference time up to 20\% and halves training time. On top, we apply
state-of-the-art online multi target feature tracking on the object
measurements to further increase accuracy and robustness utilizing temporal
information. Our experiments on KITTI show that we achieve same results as
state-of-the-art in all related categories, while maintaining the performance
and accuracy trade-off and still run in real-time. Furthermore, our model is
the first one that fuses visual semantic with 3D object detection
- …