124 research outputs found
RCDNet: An Interpretable Rain Convolutional Dictionary Network for Single Image Deraining
As a common weather, rain streaks adversely degrade the image quality. Hence,
removing rains from an image has become an important issue in the field. To
handle such an ill-posed single image deraining task, in this paper, we
specifically build a novel deep architecture, called rain convolutional
dictionary network (RCDNet), which embeds the intrinsic priors of rain streaks
and has clear interpretability. In specific, we first establish a RCD model for
representing rain streaks and utilize the proximal gradient descent technique
to design an iterative algorithm only containing simple operators for solving
the model. By unfolding it, we then build the RCDNet in which every network
module has clear physical meanings and corresponds to each operation involved
in the algorithm. This good interpretability greatly facilitates an easy
visualization and analysis on what happens inside the network and why it works
well in inference process. Moreover, taking into account the domain gap issue
in real scenarios, we further design a novel dynamic RCDNet, where the rain
kernels can be dynamically inferred corresponding to input rainy images and
then help shrink the space for rain layer estimation with few rain maps so as
to ensure a fine generalization performance in the inconsistent scenarios of
rain types between training and testing data. By end-to-end training such an
interpretable network, all involved rain kernels and proximal operators can be
automatically extracted, faithfully characterizing the features of both rain
and clean background layers, and thus naturally lead to better deraining
performance. Comprehensive experiments substantiate the superiority of our
method, especially on its well generality to diverse testing scenarios and good
interpretability for all its modules. Code is available in
\emph{\url{https://github.com/hongwang01/DRCDNet}}
Memory-guided Image De-raining Using Time-Lapse Data
This paper addresses the problem of single image de-raining, that is, the
task of recovering clean and rain-free background scenes from a single image
obscured by a rainy artifact. Although recent advances adopt real-world
time-lapse data to overcome the need for paired rain-clean images, they are
limited to fully exploit the time-lapse data. The main cause is that, in terms
of network architectures, they could not capture long-term rain streak
information in the time-lapse data during training owing to the lack of memory
components. To address this problem, we propose a novel network architecture
based on a memory network that explicitly helps to capture long-term rain
streak information in the time-lapse data. Our network comprises the
encoder-decoder networks and a memory network. The features extracted from the
encoder are read and updated in the memory network that contains several memory
items to store rain streak-aware feature representations. With the read/update
operation, the memory network retrieves relevant memory items in terms of the
queries, enabling the memory items to represent the various rain streaks
included in the time-lapse data. To boost the discriminative power of memory
features, we also present a novel background selective whitening (BSW) loss for
capturing only rain streak information in the memory network by erasing the
background information. Experimental results on standard benchmarks demonstrate
the effectiveness and superiority of our approach
- …