29 research outputs found
GTAV-NightRain: Photometric Realistic Large-scale Dataset for Night-time Rain Streak Removal
Rain is transparent, which reflects and refracts light in the scene to the
camera. In outdoor vision, rain, especially rain streaks degrade visibility and
therefore need to be removed. In existing rain streak removal datasets,
although density, scale, direction and intensity have been considered,
transparency is not fully taken into account. This problem is particularly
serious in night scenes, where the appearance of rain largely depends on the
interaction with scene illuminations and changes drastically on different
positions within the image. This is problematic, because unrealistic dataset
causes serious domain bias. In this paper, we propose GTAV-NightRain dataset,
which is a large-scale synthetic night-time rain streak removal dataset. Unlike
existing datasets, by using 3D computer graphic platform (namely GTA V), we are
allowed to infer the three dimensional interaction between rain and
illuminations, which insures the photometric realness. Current release of the
dataset contains 12,860 HD rainy images and 1,286 corresponding HD ground truth
images in diversified night scenes. A systematic benchmark and analysis are
provided along with the dataset to inspire further research
Rain Removal in Traffic Surveillance: Does it Matter?
Varying weather conditions, including rainfall and snowfall, are generally
regarded as a challenge for computer vision algorithms. One proposed solution
to the challenges induced by rain and snowfall is to artificially remove the
rain from images or video using rain removal algorithms. It is the promise of
these algorithms that the rain-removed image frames will improve the
performance of subsequent segmentation and tracking algorithms. However, rain
removal algorithms are typically evaluated on their ability to remove synthetic
rain on a small subset of images. Currently, their behavior is unknown on
real-world videos when integrated with a typical computer vision pipeline. In
this paper, we review the existing rain removal algorithms and propose a new
dataset that consists of 22 traffic surveillance sequences under a broad
variety of weather conditions that all include either rain or snowfall. We
propose a new evaluation protocol that evaluates the rain removal algorithms on
their ability to improve the performance of subsequent segmentation, instance
segmentation, and feature tracking algorithms under rain and snow. If
successful, the de-rained frames of a rain removal algorithm should improve
segmentation performance and increase the number of accurately tracked
features. The results show that a recent single-frame-based rain removal
algorithm increases the segmentation performance by 19.7% on our proposed
dataset, but it eventually decreases the feature tracking performance and
showed mixed results with recent instance segmentation methods. However, the
best video-based rain removal algorithm improves the feature tracking accuracy
by 7.72%.Comment: Published in IEEE Transactions on Intelligent Transportation System
Model-based occlusion disentanglement for image-to-image translation
Image-to-image translation is affected by entanglement phenomena, which may
occur in case of target data encompassing occlusions such as raindrops, dirt,
etc. Our unsupervised model-based learning disentangles scene and occlusions,
while benefiting from an adversarial pipeline to regress physical parameters of
the occlusion model. The experiments demonstrate our method is able to handle
varying types of occlusions and generate highly realistic translations,
qualitatively and quantitatively outperforming the state-of-the-art on multiple
datasets.Comment: ECCV 202
Rain rendering for evaluating and improving robustness to bad weather
Rain fills the atmosphere with water particles, which breaks the common
assumption that light travels unaltered from the scene to the camera. While it
is well-known that rain affects computer vision algorithms, quantifying its
impact is difficult. In this context, we present a rain rendering pipeline that
enables the systematic evaluation of common computer vision algorithms to
controlled amounts of rain. We present three different ways to add synthetic
rain to existing images datasets: completely physic-based; completely
data-driven; and a combination of both. The physic-based rain augmentation
combines a physical particle simulator and accurate rain photometric modeling.
We validate our rendering methods with a user study, demonstrating our rain is
judged as much as 73% more realistic than the state-of-theart. Using our
generated rain-augmented KITTI, Cityscapes, and nuScenes datasets, we conduct a
thorough evaluation of object detection, semantic segmentation, and depth
estimation algorithms and show that their performance decreases in degraded
weather, on the order of 15% for object detection, 60% for semantic
segmentation, and 6-fold increase in depth estimation error. Finetuning on our
augmented synthetic data results in improvements of 21% on object detection,
37% on semantic segmentation, and 8% on depth estimation.Comment: 19 pages, 19 figures, IJCV 2020 preprint. arXiv admin note: text
overlap with arXiv:1908.1033