75 research outputs found
Unsupervised Single Image Deraining with Self-supervised Constraints
Most existing single image deraining methods require learning supervised
models from a large set of paired synthetic training data, which limits their
generality, scalability and practicality in real-world multimedia applications.
Besides, due to lack of labeled-supervised constraints, directly applying
existing unsupervised frameworks to the image deraining task will suffer from
low-quality recovery. Therefore, we propose an Unsupervised Deraining
Generative Adversarial Network (UD-GAN) to tackle above problems by introducing
self-supervised constraints from the intrinsic statistics of unpaired rainy and
clean images. Specifically, we firstly design two collaboratively optimized
modules, namely Rain Guidance Module (RGM) and Background Guidance Module
(BGM), to take full advantage of rainy image characteristics: The RGM is
designed to discriminate real rainy images from fake rainy images which are
created based on outputs of the generator with BGM. Simultaneously, the BGM
exploits a hierarchical Gaussian-Blur gradient error to ensure background
consistency between rainy input and de-rained output. Secondly, a novel
luminance-adjusting adversarial loss is integrated into the clean image
discriminator considering the built-in luminance difference between real clean
images and derained images. Comprehensive experiment results on various
benchmarking datasets and different training settings show that UD-GAN
outperforms existing image deraining methods in both quantitative and
qualitative comparisons.Comment: 10 pages, 8 figure
Detail-recovery Image Deraining via Dual Sample-augmented Contrastive Learning
The intricacy of rainy image contents often leads cutting-edge deraining
models to image degradation including remnant rain, wrongly-removed details,
and distorted appearance. Such degradation is further exacerbated when applying
the models trained on synthetic data to real-world rainy images. We observe two
types of domain gaps between synthetic and real-world rainy images: one exists
in rain streak patterns; the other is the pixel-level appearance of rain-free
images. To bridge the two domain gaps, we propose a semi-supervised
detail-recovery image deraining network (Semi-DRDNet) with dual
sample-augmented contrastive learning. Semi-DRDNet consists of three
sub-networks:i) for removing rain streaks without remnants, we present a
squeeze-and-excitation based rain residual network; ii) for encouraging the
lost details to return, we construct a structure detail context aggregation
based detail repair network; to our knowledge, this is the first time; and iii)
for building efficient contrastive constraints for both rain streaks and clean
backgrounds, we exploit a novel dual sample-augmented contrastive
regularization network.Semi-DRDNet operates smoothly on both synthetic and
real-world rainy data in terms of deraining robustness and detail accuracy.
Comparisons on four datasets including our established Real200 show clear
improvements of Semi-DRDNet over fifteen state-of-the-art methods. Code and
dataset are available at https://github.com/syy-whu/DRD-Net.Comment: 17 page
RainDiffusion:When Unsupervised Learning Meets Diffusion Models for Real-world Image Deraining
What will happen when unsupervised learning meets diffusion models for
real-world image deraining? To answer it, we propose RainDiffusion, the first
unsupervised image deraining paradigm based on diffusion models. Beyond the
traditional unsupervised wisdom of image deraining, RainDiffusion introduces
stable training of unpaired real-world data instead of weakly adversarial
training. RainDiffusion consists of two cooperative branches: Non-diffusive
Translation Branch (NTB) and Diffusive Translation Branch (DTB). NTB exploits a
cycle-consistent architecture to bypass the difficulty in unpaired training of
standard diffusion models by generating initial clean/rainy image pairs. DTB
leverages two conditional diffusion modules to progressively refine the desired
output with initial image pairs and diffusive generative prior, to obtain a
better generalization ability of deraining and rain generation. Rain-Diffusion
is a non adversarial training paradigm, serving as a new standard bar for
real-world image deraining. Extensive experiments confirm the superiority of
our RainDiffusion over un/semi-supervised methods and show its competitive
advantages over fully-supervised ones.Comment: 9 page
Unsupervised Deraining: Where Contrastive Learning Meets Self-similarity
Image deraining is a typical low-level image restoration task, which aims at
decomposing the rainy image into two distinguishable layers: the clean image
layer and the rain layer. Most of the existing learning-based deraining methods
are supervisedly trained on synthetic rainy-clean pairs. The domain gap between
the synthetic and real rains makes them less generalized to different real
rainy scenes. Moreover, the existing methods mainly utilize the property of the
two layers independently, while few of them have considered the mutually
exclusive relationship between the two layers. In this work, we propose a novel
non-local contrastive learning (NLCL) method for unsupervised image deraining.
Consequently, we not only utilize the intrinsic self-similarity property within
samples but also the mutually exclusive property between the two layers, so as
to better differ the rain layer from the clean image. Specifically, the
non-local self-similarity image layer patches as the positives are pulled
together and similar rain layer patches as the negatives are pushed away. Thus
the similar positive/negative samples that are close in the original space
benefit us to enrich more discriminative representation. Apart from the
self-similarity sampling strategy, we analyze how to choose an appropriate
feature encoder in NLCL. Extensive experiments on different real rainy datasets
demonstrate that the proposed method obtains state-of-the-art performance in
real deraining.Comment: 10 pages, 10 figures, accept to 2022CVP
Unsupervised Deraining: Where Asymmetric Contrastive Learning Meets Self-similarity
Most of the existing learning-based deraining methods are supervisedly
trained on synthetic rainy-clean pairs. The domain gap between the synthetic
and real rain makes them less generalized to complex real rainy scenes.
Moreover, the existing methods mainly utilize the property of the image or rain
layers independently, while few of them have considered their mutually
exclusive relationship. To solve above dilemma, we explore the intrinsic
intra-similarity within each layer and inter-exclusiveness between two layers
and propose an unsupervised non-local contrastive learning (NLCL) deraining
method. The non-local self-similarity image patches as the positives are
tightly pulled together, rain patches as the negatives are remarkably pushed
away, and vice versa. On one hand, the intrinsic self-similarity knowledge
within positive/negative samples of each layer benefits us to discover more
compact representation; on the other hand, the mutually exclusive property
between the two layers enriches the discriminative decomposition. Thus, the
internal self-similarity within each layer (similarity) and the external
exclusive relationship of the two layers (dissimilarity) serving as a generic
image prior jointly facilitate us to unsupervisedly differentiate the rain from
clean image. We further discover that the intrinsic dimension of the non-local
image patches is generally higher than that of the rain patches. This motivates
us to design an asymmetric contrastive loss to precisely model the compactness
discrepancy of the two layers for better discriminative decomposition. In
addition, considering that the existing real rain datasets are of low quality,
either small scale or downloaded from the internet, we collect a real
large-scale dataset under various rainy kinds of weather that contains
high-resolution rainy images.Comment: 16 pages, 15 figures. arXiv admin note: substantial text overlap with
arXiv:2203.1150
Uni-Removal: A Semi-Supervised Framework for Simultaneously Addressing Multiple Degradations in Real-World Images
Removing multiple degradations, such as haze, rain, and blur, from real-world
images poses a challenging and illposed problem. Recently, unified models that
can handle different degradations have been proposed and yield promising
results. However, these approaches focus on synthetic images and experience a
significant performance drop when applied to realworld images. In this paper,
we introduce Uni-Removal, a twostage semi-supervised framework for addressing
the removal of multiple degradations in real-world images using a unified model
and parameters. In the knowledge transfer stage, Uni-Removal leverages a
supervised multi-teacher and student architecture in the knowledge transfer
stage to facilitate learning from pretrained teacher networks specialized in
different degradation types. A multi-grained contrastive loss is introduced to
enhance learning from feature and image spaces. In the domain adaptation stage,
unsupervised fine-tuning is performed by incorporating an adversarial
discriminator on real-world images. The integration of an extended
multi-grained contrastive loss and generative adversarial loss enables the
adaptation of the student network from synthetic to real-world domains.
Extensive experiments on real-world degraded datasets demonstrate the
effectiveness of our proposed method. We compare our Uni-Removal framework with
state-of-the-art supervised and unsupervised methods, showcasing its promising
results in real-world image dehazing, deraining, and deblurring simultaneously
RCDNet: An Interpretable Rain Convolutional Dictionary Network for Single Image Deraining
As a common weather, rain streaks adversely degrade the image quality. Hence,
removing rains from an image has become an important issue in the field. To
handle such an ill-posed single image deraining task, in this paper, we
specifically build a novel deep architecture, called rain convolutional
dictionary network (RCDNet), which embeds the intrinsic priors of rain streaks
and has clear interpretability. In specific, we first establish a RCD model for
representing rain streaks and utilize the proximal gradient descent technique
to design an iterative algorithm only containing simple operators for solving
the model. By unfolding it, we then build the RCDNet in which every network
module has clear physical meanings and corresponds to each operation involved
in the algorithm. This good interpretability greatly facilitates an easy
visualization and analysis on what happens inside the network and why it works
well in inference process. Moreover, taking into account the domain gap issue
in real scenarios, we further design a novel dynamic RCDNet, where the rain
kernels can be dynamically inferred corresponding to input rainy images and
then help shrink the space for rain layer estimation with few rain maps so as
to ensure a fine generalization performance in the inconsistent scenarios of
rain types between training and testing data. By end-to-end training such an
interpretable network, all involved rain kernels and proximal operators can be
automatically extracted, faithfully characterizing the features of both rain
and clean background layers, and thus naturally lead to better deraining
performance. Comprehensive experiments substantiate the superiority of our
method, especially on its well generality to diverse testing scenarios and good
interpretability for all its modules. Code is available in
\emph{\url{https://github.com/hongwang01/DRCDNet}}
- …