10,910 research outputs found
Map-Guided Curriculum Domain Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation
We address the problem of semantic nighttime image segmentation and improve
the state-of-the-art, by adapting daytime models to nighttime without using
nighttime annotations. Moreover, we design a new evaluation framework to
address the substantial uncertainty of semantics in nighttime images. Our
central contributions are: 1) a curriculum framework to gradually adapt
semantic segmentation models from day to night through progressively darker
times of day, exploiting cross-time-of-day correspondences between daytime
images from a reference map and dark images to guide the label inference in the
dark domains; 2) a novel uncertainty-aware annotation and evaluation framework
and metric for semantic segmentation, including image regions beyond human
recognition capability in the evaluation in a principled fashion; 3) the Dark
Zurich dataset, comprising 2416 unlabeled nighttime and 2920 unlabeled twilight
images with correspondences to their daytime counterparts plus a set of 201
nighttime images with fine pixel-level annotations created with our protocol,
which serves as a first benchmark for our novel evaluation. Experiments show
that our map-guided curriculum adaptation significantly outperforms
state-of-the-art methods on nighttime sets both for standard metrics and our
uncertainty-aware metric. Furthermore, our uncertainty-aware evaluation reveals
that selective invalidation of predictions can improve results on data with
ambiguous content such as our benchmark and profit safety-oriented applications
involving invalid inputs.Comment: IEEE T-PAMI 202
Image-Adaptive YOLO for Object Detection in Adverse Weather Conditions
Though deep learning-based object detection methods have achieved promising
results on the conventional datasets, it is still challenging to locate objects
from the low-quality images captured in adverse weather conditions. The
existing methods either have difficulties in balancing the tasks of image
enhancement and object detection, or often ignore the latent information
beneficial for detection. To alleviate this problem, we propose a novel
Image-Adaptive YOLO (IA-YOLO) framework, where each image can be adaptively
enhanced for better detection performance. Specifically, a differentiable image
processing (DIP) module is presented to take into account the adverse weather
conditions for YOLO detector, whose parameters are predicted by a small
convolutional neural net-work (CNN-PP). We learn CNN-PP and YOLOv3 jointly in
an end-to-end fashion, which ensures that CNN-PP can learn an appropriate DIP
to enhance the image for detection in a weakly supervised manner. Our proposed
IA-YOLO approach can adaptively process images in both normal and adverse
weather conditions. The experimental results are very encouraging,
demonstrating the effectiveness of our proposed IA-YOLO method in both foggy
and low-light scenarios.Comment: AAAI 2022, Preprint version with Appendi
Guided Curriculum Model Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation
Most progress in semantic segmentation reports on daytime images taken under
favorable illumination conditions. We instead address the problem of semantic
segmentation of nighttime images and improve the state-of-the-art, by adapting
daytime models to nighttime without using nighttime annotations. Moreover, we
design a new evaluation framework to address the substantial uncertainty of
semantics in nighttime images. Our central contributions are: 1) a curriculum
framework to gradually adapt semantic segmentation models from day to night via
labeled synthetic images and unlabeled real images, both for progressively
darker times of day, which exploits cross-time-of-day correspondences for the
real images to guide the inference of their labels; 2) a novel
uncertainty-aware annotation and evaluation framework and metric for semantic
segmentation, designed for adverse conditions and including image regions
beyond human recognition capability in the evaluation in a principled fashion;
3) the Dark Zurich dataset, which comprises 2416 unlabeled nighttime and 2920
unlabeled twilight images with correspondences to their daytime counterparts
plus a set of 151 nighttime images with fine pixel-level annotations created
with our protocol, which serves as a first benchmark to perform our novel
evaluation. Experiments show that our guided curriculum adaptation
significantly outperforms state-of-the-art methods on real nighttime sets both
for standard metrics and our uncertainty-aware metric. Furthermore, our
uncertainty-aware evaluation reveals that selective invalidation of predictions
can lead to better results on data with ambiguous content such as our nighttime
benchmark and profit safety-oriented applications which involve invalid inputs.Comment: ICCV 2019 camera-read
Dark Model Adaptation: Semantic Image Segmentation from Daytime to Nighttime
This work addresses the problem of semantic image segmentation of nighttime
scenes. Although considerable progress has been made in semantic image
segmentation, it is mainly related to daytime scenarios. This paper proposes a
novel method to progressive adapt the semantic models trained on daytime
scenes, along with large-scale annotations therein, to nighttime scenes via the
bridge of twilight time -- the time between dawn and sunrise, or between sunset
and dusk. The goal of the method is to alleviate the cost of human annotation
for nighttime images by transferring knowledge from standard daytime
conditions. In addition to the method, a new dataset of road scenes is
compiled; it consists of 35,000 images ranging from daytime to twilight time
and to nighttime. Also, a subset of the nighttime images are densely annotated
for method evaluation. Our experiments show that our method is effective for
model adaptation from daytime scenes to nighttime scenes, without using extra
human annotation.Comment: Accepted to International Conference on Intelligent Transportation
Systems (ITSC 2018
Model Adaptation with Synthetic and Real Data for Semantic Dense Foggy Scene Understanding
This work addresses the problem of semantic scene understanding under dense
fog. Although considerable progress has been made in semantic scene
understanding, it is mainly related to clear-weather scenes. Extending
recognition methods to adverse weather conditions such as fog is crucial for
outdoor applications. In this paper, we propose a novel method, named
Curriculum Model Adaptation (CMAda), which gradually adapts a semantic
segmentation model from light synthetic fog to dense real fog in multiple
steps, using both synthetic and real foggy data. In addition, we present three
other main stand-alone contributions: 1) a novel method to add synthetic fog to
real, clear-weather scenes using semantic input; 2) a new fog density
estimator; 3) the Foggy Zurich dataset comprising real foggy images,
with pixel-level semantic annotations for images with dense fog. Our
experiments show that 1) our fog simulation slightly outperforms a
state-of-the-art competing simulation with respect to the task of semantic
foggy scene understanding (SFSU); 2) CMAda improves the performance of
state-of-the-art models for SFSU significantly by leveraging unlabeled real
foggy data. The datasets and code are publicly available.Comment: final version, ECCV 201
VBLC: Visibility Boosting and Logit-Constraint Learning for Domain Adaptive Semantic Segmentation under Adverse Conditions
Generalizing models trained on normal visual conditions to target domains
under adverse conditions is demanding in the practical systems. One prevalent
solution is to bridge the domain gap between clear- and adverse-condition
images to make satisfactory prediction on the target. However, previous methods
often reckon on additional reference images of the same scenes taken from
normal conditions, which are quite tough to collect in reality. Furthermore,
most of them mainly focus on individual adverse condition such as nighttime or
foggy, weakening the model versatility when encountering other adverse
weathers. To overcome the above limitations, we propose a novel framework,
Visibility Boosting and Logit-Constraint learning (VBLC), tailored for superior
normal-to-adverse adaptation. VBLC explores the potential of getting rid of
reference images and resolving the mixture of adverse conditions
simultaneously. In detail, we first propose the visibility boost module to
dynamically improve target images via certain priors in the image level. Then,
we figure out the overconfident drawback in the conventional cross-entropy loss
for self-training method and devise the logit-constraint learning, which
enforces a constraint on logit outputs during training to mitigate this pain
point. To the best of our knowledge, this is a new perspective for tackling
such a challenging task. Extensive experiments on two normal-to-adverse domain
adaptation benchmarks, i.e., Cityscapes -> ACDC and Cityscapes ->
FoggyCityscapes + RainCityscapes, verify the effectiveness of VBLC, where it
establishes the new state of the art. Code is available at
https://github.com/BIT-DA/VBLC.Comment: Camera ready for AAAI 2023. Code is available at
https://github.com/BIT-DA/VBL
Counting Crowds in Bad Weather
Crowd counting has recently attracted significant attention in the field of
computer vision due to its wide applications to image understanding. Numerous
methods have been proposed and achieved state-of-the-art performance for
real-world tasks. However, existing approaches do not perform well under
adverse weather such as haze, rain, and snow since the visual appearances of
crowds in such scenes are drastically different from those images in clear
weather of typical datasets. In this paper, we propose a method for robust
crowd counting in adverse weather scenarios. Instead of using a two-stage
approach that involves image restoration and crowd counting modules, our model
learns effective features and adaptive queries to account for large appearance
variations. With these weather queries, the proposed model can learn the
weather information according to the degradation of the input image and
optimize with the crowd counting module simultaneously. Experimental results
show that the proposed algorithm is effective in counting crowds under
different weather types on benchmark datasets. The source code and trained
models will be made available to the public.Comment: including supplemental materia
Air pollution and fog detection through vehicular sensors
We describe a method for the automatic recognition of air pollution and fog from a vehicle. Our system consists of sensors to acquire main data from cameras as well as from Light Detection and Recognition (LIDAR) instruments. We discuss how this data can be collected, analyzed and merged to determine the degree of air pollution or fog. Such data is essential for control systems of moving vehicles in making autonomous decisions for avoidance. Backend systems need such data for forecasting and strategic traffic planning and control. Laboratory based experimental results are presented for weather conditions like air pollution and fog, showing that the recognition scenario works with better than adequate results. This paper demonstrates that LIDAR technology, already onboard for the purpose of autonomous driving, can be used to improve weather condition recognition when compared with a camera only system. We conclude that the combination of a front camera and a LIDAR laser scanner is well suited as a sensor instrument set for air pollution and fog recognition that can contribute accurate data to driving assistance and weather alerting-systems
- …