38 research outputs found
Hierarchical-level rain image generative model based on GAN
Autonomous vehicles are exposed to various weather during operation, which is
likely to trigger the performance limitations of the perception system, leading
to the safety of the intended functionality (SOTIF) problems. To efficiently
generate data for testing the performance of visual perception algorithms under
various weather conditions, a hierarchical-level rain image generative model,
rain conditional CycleGAN (RCCycleGAN), is constructed. RCCycleGAN is based on
the generative adversarial network (GAN) and can generate images of light,
medium, and heavy rain. Different rain intensities are introduced as labels in
conditional GAN (CGAN). Meanwhile, the model structure is optimized and the
training strategy is adjusted to alleviate the problem of mode collapse. In
addition, natural rain images of different intensities are collected and
processed for model training and validation. Compared with the two baseline
models, CycleGAN and DerainCycleGAN, the peak signal-to-noise ratio (PSNR) of
RCCycleGAN on the test dataset is improved by 2.58 dB and 0.74 dB, and the
structural similarity (SSIM) is improved by 18% and 8%, respectively. The
ablation experiments are also carried out to validate the effectiveness of the
model tuning
GTAV-NightRain: Photometric Realistic Large-scale Dataset for Night-time Rain Streak Removal
Rain is transparent, which reflects and refracts light in the scene to the
camera. In outdoor vision, rain, especially rain streaks degrade visibility and
therefore need to be removed. In existing rain streak removal datasets,
although density, scale, direction and intensity have been considered,
transparency is not fully taken into account. This problem is particularly
serious in night scenes, where the appearance of rain largely depends on the
interaction with scene illuminations and changes drastically on different
positions within the image. This is problematic, because unrealistic dataset
causes serious domain bias. In this paper, we propose GTAV-NightRain dataset,
which is a large-scale synthetic night-time rain streak removal dataset. Unlike
existing datasets, by using 3D computer graphic platform (namely GTA V), we are
allowed to infer the three dimensional interaction between rain and
illuminations, which insures the photometric realness. Current release of the
dataset contains 12,860 HD rainy images and 1,286 corresponding HD ground truth
images in diversified night scenes. A systematic benchmark and analysis are
provided along with the dataset to inspire further research
ASF-Net: Robust Video Deraining via Temporal Alignment and Online Adaptive Learning
In recent times, learning-based methods for video deraining have demonstrated
commendable results. However, there are two critical challenges that these
methods are yet to address: exploiting temporal correlations among adjacent
frames and ensuring adaptability to unknown real-world scenarios. To overcome
these challenges, we explore video deraining from a paradigm design perspective
to learning strategy construction. Specifically, we propose a new computational
paradigm, Alignment-Shift-Fusion Network (ASF-Net), which incorporates a
temporal shift module. This module is novel to this field and provides deeper
exploration of temporal information by facilitating the exchange of
channel-level information within the feature space. To fully discharge the
model's characterization capability, we further construct a LArge-scale RAiny
video dataset (LARA) which also supports the development of this community. On
the basis of the newly-constructed dataset, we explore the parameters learning
process by developing an innovative re-degraded learning strategy. This
strategy bridges the gap between synthetic and real-world scenes, resulting in
stronger scene adaptability. Our proposed approach exhibits superior
performance in three benchmarks and compelling visual quality in real-world
scenarios, underscoring its efficacy. The code is available at
https://github.com/vis-opt-group/ASF-Net