2,239 research outputs found
Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes
During the last half decade, convolutional neural networks (CNNs) have
triumphed over semantic segmentation, which is one of the core tasks in many
applications such as autonomous driving. However, to train CNNs requires a
considerable amount of data, which is difficult to collect and laborious to
annotate. Recent advances in computer graphics make it possible to train CNNs
on photo-realistic synthetic imagery with computer-generated annotations.
Despite this, the domain mismatch between the real images and the synthetic
data cripples the models' performance. Hence, we propose a curriculum-style
learning approach to minimize the domain gap in urban scenery semantic
segmentation. The curriculum domain adaptation solves easy tasks first to infer
necessary properties about the target domain; in particular, the first task is
to learn global label distributions over images and local distributions over
landmark superpixels. These are easy to estimate because images of urban scenes
have strong idiosyncrasies (e.g., the size and spatial relations of buildings,
streets, cars, etc.). We then train a segmentation network while regularizing
its predictions in the target domain to follow those inferred properties. In
experiments, our method outperforms the baselines on two datasets and two
backbone networks. We also report extensive ablation studies about our
approach.Comment: This is the extended version of the ICCV 2017 paper "Curriculum
Domain Adaptation for Semantic Segmentation of Urban Scenes" with additional
GTA experimen
Model Adaptation with Synthetic and Real Data for Semantic Dense Foggy Scene Understanding
This work addresses the problem of semantic scene understanding under dense
fog. Although considerable progress has been made in semantic scene
understanding, it is mainly related to clear-weather scenes. Extending
recognition methods to adverse weather conditions such as fog is crucial for
outdoor applications. In this paper, we propose a novel method, named
Curriculum Model Adaptation (CMAda), which gradually adapts a semantic
segmentation model from light synthetic fog to dense real fog in multiple
steps, using both synthetic and real foggy data. In addition, we present three
other main stand-alone contributions: 1) a novel method to add synthetic fog to
real, clear-weather scenes using semantic input; 2) a new fog density
estimator; 3) the Foggy Zurich dataset comprising real foggy images,
with pixel-level semantic annotations for images with dense fog. Our
experiments show that 1) our fog simulation slightly outperforms a
state-of-the-art competing simulation with respect to the task of semantic
foggy scene understanding (SFSU); 2) CMAda improves the performance of
state-of-the-art models for SFSU significantly by leveraging unlabeled real
foggy data. The datasets and code are publicly available.Comment: final version, ECCV 201
Listening for Sirens: Locating and Classifying Acoustic Alarms in City Scenes
This paper is about alerting acoustic event detection and sound source
localisation in an urban scenario. Specifically, we are interested in spotting
the presence of horns, and sirens of emergency vehicles. In order to obtain a
reliable system able to operate robustly despite the presence of traffic noise,
which can be copious, unstructured and unpredictable, we propose to treat the
spectrograms of incoming stereo signals as images, and apply semantic
segmentation, based on a Unet architecture, to extract the target sound from
the background noise. In a multi-task learning scheme, together with signal
denoising, we perform acoustic event classification to identify the nature of
the alerting sound. Lastly, we use the denoised signals to localise the
acoustic source on the horizon plane, by regressing the direction of arrival of
the sound through a CNN architecture. Our experimental evaluation shows an
average classification rate of 94%, and a median absolute error on the
localisation of 7.5{\deg} when operating on audio frames of 0.5s, and of
2.5{\deg} when operating on frames of 2.5s. The system offers excellent
performance in particularly challenging scenarios, where the noise level is
remarkably high.Comment: 6 pages, 9 figure
Improving Anomaly Segmentation with Multi-Granularity Cross-Domain Alignment
Anomaly segmentation plays a crucial role in identifying anomalous objects
within images, which facilitates the detection of road anomalies for autonomous
driving. Although existing methods have shown impressive results in anomaly
segmentation using synthetic training data, the domain discrepancies between
synthetic training data and real test data are often neglected. To address this
issue, the Multi-Granularity Cross-Domain Alignment (MGCDA) framework is
proposed for anomaly segmentation in complex driving environments. It uniquely
combines a new Multi-source Domain Adversarial Training (MDAT) module and a
novel Cross-domain Anomaly-aware Contrastive Learning (CACL) method to boost
the generality of the model, seamlessly integrating multi-domain data at both
scene and sample levels. Multi-source domain adversarial loss and a dynamic
label smoothing strategy are integrated into the MDAT module to facilitate the
acquisition of domain-invariant features at the scene level, through
adversarial training across multiple stages. CACL aligns sample-level
representations with contrastive loss on cross-domain data, which utilizes an
anomaly-aware sampling strategy to efficiently sample hard samples and anchors.
The proposed framework has decent properties of parameter-free during the
inference stage and is compatible with other anomaly segmentation networks.
Experimental conducted on Fishyscapes and RoadAnomaly datasets demonstrate that
the proposed framework achieves state-of-the-art performance.Comment: Accepted to ACM Multimedia 202
Learning Aerial Image Segmentation from Online Maps
This study deals with semantic segmentation of high-resolution (aerial)
images where a semantic class label is assigned to each pixel via supervised
classification as a basis for automatic map generation. Recently, deep
convolutional neural networks (CNNs) have shown impressive performance and have
quickly become the de-facto standard for semantic segmentation, with the added
benefit that task-specific feature design is no longer necessary. However, a
major downside of deep learning methods is that they are extremely data-hungry,
thus aggravating the perennial bottleneck of supervised classification, to
obtain enough annotated training data. On the other hand, it has been observed
that they are rather robust against noise in the training labels. This opens up
the intriguing possibility to avoid annotating huge amounts of training data,
and instead train the classifier from existing legacy data or crowd-sourced
maps which can exhibit high levels of noise. The question addressed in this
paper is: can training with large-scale, publicly available labels replace a
substantial part of the manual labeling effort and still achieve sufficient
performance? Such data will inevitably contain a significant portion of errors,
but in return virtually unlimited quantities of it are available in larger
parts of the world. We adapt a state-of-the-art CNN architecture for semantic
segmentation of buildings and roads in aerial images, and compare its
performance when using different training data sets, ranging from manually
labeled, pixel-accurate ground truth of the same city to automatic training
data derived from OpenStreetMap data from distant locations. We report our
results that indicate that satisfying performance can be obtained with
significantly less manual annotation effort, by exploiting noisy large-scale
training data.Comment: Published in IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSIN
Generating All the Roads to Rome: Road Layout Randomization for Improved Road Marking Segmentation
Road markings provide guidance to traffic participants and enforce safe
driving behaviour, understanding their semantic meaning is therefore paramount
in (automated) driving. However, producing the vast quantities of road marking
labels required for training state-of-the-art deep networks is costly,
time-consuming, and simply infeasible for every domain and condition. In
addition, training data retrieved from virtual worlds often lack the richness
and complexity of the real world and consequently cannot be used directly. In
this paper, we provide an alternative approach in which new road marking
training pairs are automatically generated. To this end, we apply principles of
domain randomization to the road layout and synthesize new images from altered
semantic labels. We demonstrate that training on these synthetic pairs improves
mIoU of the segmentation of rare road marking classes during real-world
deployment in complex urban environments by more than 12 percentage points,
while performance for other classes is retained. This framework can easily be
scaled to all domains and conditions to generate large-scale road marking
datasets, while avoiding manual labelling effort.Comment: presented at ITSC 201
Synthetic Datasets for Autonomous Driving: A Survey
Autonomous driving techniques have been flourishing in recent years while
thirsting for huge amounts of high-quality data. However, it is difficult for
real-world datasets to keep up with the pace of changing requirements due to
their expensive and time-consuming experimental and labeling costs. Therefore,
more and more researchers are turning to synthetic datasets to easily generate
rich and changeable data as an effective complement to the real world and to
improve the performance of algorithms. In this paper, we summarize the
evolution of synthetic dataset generation methods and review the work to date
in synthetic datasets related to single and multi-task categories for to
autonomous driving study. We also discuss the role that synthetic dataset plays
the evaluation, gap test, and positive effect in autonomous driving related
algorithm testing, especially on trustworthiness and safety aspects. Finally,
we discuss general trends and possible development directions. To the best of
our knowledge, this is the first survey focusing on the application of
synthetic datasets in autonomous driving. This survey also raises awareness of
the problems of real-world deployment of autonomous driving technology and
provides researchers with a possible solution.Comment: 19 pages, 5 figure
- …