10,760 research outputs found
Semi-supervised Transfer Learning for Image Rain Removal
Single image rain removal is a typical inverse problem in computer vision.
The deep learning technique has been verified to be effective for this task and
achieved state-of-the-art performance. However, previous deep learning methods
need to pre-collect a large set of image pairs with/without synthesized rain
for training, which tends to make the neural network be biased toward learning
the specific patterns of the synthesized rain, while be less able to generalize
to real test samples whose rain types differ from those in the training data.
To this issue, this paper firstly proposes a semi-supervised learning paradigm
toward this task. Different from traditional deep learning methods which only
use supervised image pairs with/without synthesized rain, we further put real
rainy images, without need of their clean ones, into the network training
process. This is realized by elaborately formulating the residual between an
input rainy image and its expected network output (clear image without rain) as
a specific parametrized rain streaks distribution. The network is therefore
trained to adapt real unsupervised diverse rain types through transferring from
the supervised synthesized rain, and thus both the short-of-training-sample and
bias-to-supervised-sample issues can be evidently alleviated. Experiments on
synthetic and real data verify the superiority of our model compared to the
state-of-the-arts.Comment: 10 page
Uni-Removal: A Semi-Supervised Framework for Simultaneously Addressing Multiple Degradations in Real-World Images
Removing multiple degradations, such as haze, rain, and blur, from real-world
images poses a challenging and illposed problem. Recently, unified models that
can handle different degradations have been proposed and yield promising
results. However, these approaches focus on synthetic images and experience a
significant performance drop when applied to realworld images. In this paper,
we introduce Uni-Removal, a twostage semi-supervised framework for addressing
the removal of multiple degradations in real-world images using a unified model
and parameters. In the knowledge transfer stage, Uni-Removal leverages a
supervised multi-teacher and student architecture in the knowledge transfer
stage to facilitate learning from pretrained teacher networks specialized in
different degradation types. A multi-grained contrastive loss is introduced to
enhance learning from feature and image spaces. In the domain adaptation stage,
unsupervised fine-tuning is performed by incorporating an adversarial
discriminator on real-world images. The integration of an extended
multi-grained contrastive loss and generative adversarial loss enables the
adaptation of the student network from synthetic to real-world domains.
Extensive experiments on real-world degraded datasets demonstrate the
effectiveness of our proposed method. We compare our Uni-Removal framework with
state-of-the-art supervised and unsupervised methods, showcasing its promising
results in real-world image dehazing, deraining, and deblurring simultaneously
Detail-recovery Image Deraining via Dual Sample-augmented Contrastive Learning
The intricacy of rainy image contents often leads cutting-edge deraining
models to image degradation including remnant rain, wrongly-removed details,
and distorted appearance. Such degradation is further exacerbated when applying
the models trained on synthetic data to real-world rainy images. We observe two
types of domain gaps between synthetic and real-world rainy images: one exists
in rain streak patterns; the other is the pixel-level appearance of rain-free
images. To bridge the two domain gaps, we propose a semi-supervised
detail-recovery image deraining network (Semi-DRDNet) with dual
sample-augmented contrastive learning. Semi-DRDNet consists of three
sub-networks:i) for removing rain streaks without remnants, we present a
squeeze-and-excitation based rain residual network; ii) for encouraging the
lost details to return, we construct a structure detail context aggregation
based detail repair network; to our knowledge, this is the first time; and iii)
for building efficient contrastive constraints for both rain streaks and clean
backgrounds, we exploit a novel dual sample-augmented contrastive
regularization network.Semi-DRDNet operates smoothly on both synthetic and
real-world rainy data in terms of deraining robustness and detail accuracy.
Comparisons on four datasets including our established Real200 show clear
improvements of Semi-DRDNet over fifteen state-of-the-art methods. Code and
dataset are available at https://github.com/syy-whu/DRD-Net.Comment: 17 page
RainDiffusion:When Unsupervised Learning Meets Diffusion Models for Real-world Image Deraining
What will happen when unsupervised learning meets diffusion models for
real-world image deraining? To answer it, we propose RainDiffusion, the first
unsupervised image deraining paradigm based on diffusion models. Beyond the
traditional unsupervised wisdom of image deraining, RainDiffusion introduces
stable training of unpaired real-world data instead of weakly adversarial
training. RainDiffusion consists of two cooperative branches: Non-diffusive
Translation Branch (NTB) and Diffusive Translation Branch (DTB). NTB exploits a
cycle-consistent architecture to bypass the difficulty in unpaired training of
standard diffusion models by generating initial clean/rainy image pairs. DTB
leverages two conditional diffusion modules to progressively refine the desired
output with initial image pairs and diffusive generative prior, to obtain a
better generalization ability of deraining and rain generation. Rain-Diffusion
is a non adversarial training paradigm, serving as a new standard bar for
real-world image deraining. Extensive experiments confirm the superiority of
our RainDiffusion over un/semi-supervised methods and show its competitive
advantages over fully-supervised ones.Comment: 9 page
Semi-MoreGAN: A New Semi-supervised Generative Adversarial Network for Mixture of Rain Removal
Rain is one of the most common weather which can completely degrade the image
quality and interfere with the performance of many computer vision tasks,
especially under heavy rain conditions. We observe that: (i) rain is a mixture
of rain streaks and rainy haze; (ii) the scene depth determines the intensity
of rain streaks and the transformation into the rainy haze; (iii) most existing
deraining methods are only trained on synthetic rainy images, and hence
generalize poorly to the real-world scenes. Motivated by these observations, we
propose a new SEMI-supervised Mixture Of rain REmoval Generative Adversarial
Network (Semi-MoreGAN), which consists of four key modules: (I) a novel
attentional depth prediction network to provide precise depth estimation; (ii)
a context feature prediction network composed of several well-designed detailed
residual blocks to produce detailed image context features; (iii) a pyramid
depth-guided non-local network to effectively integrate the image context with
the depth information, and produce the final rain-free images; and (iv) a
comprehensive semi-supervised loss function to make the model not limited to
synthetic datasets but generalize smoothly to real-world heavy rainy scenes.
Extensive experiments show clear improvements of our approach over twenty
representative state-of-the-arts on both synthetic and real-world rainy images.Comment: 18 page
Memory augment is All You Need for image restoration
Image restoration is a low-level vision task, most CNN methods are designed
as a black box, lacking transparency and internal aesthetics. Although some
methods combining traditional optimization algorithms with DNNs have been
proposed, they all have some limitations. In this paper, we propose a
three-granularity memory layer and contrast learning named MemoryNet,
specifically, dividing the samples into positive, negative, and actual three
samples for contrastive learning, where the memory layer is able to preserve
the deep features of the image and the contrastive learning converges the
learned features to balance. Experiments on Derain/Deshadow/Deblur task
demonstrate that these methods are effective in improving restoration
performance. In addition, this paper's model obtains significant PSNR, SSIM
gain on three datasets with different degradation types, which is a strong
proof that the recovered images are perceptually realistic. The source code of
MemoryNet can be obtained from https://github.com/zhangbaijin/MemoryNe
- …