124 research outputs found
Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net
Hyperspectral imaging can help better understand the characteristics of
different materials, compared with traditional image systems. However, only
high-resolution multispectral (HrMS) and low-resolution hyperspectral (LrHS)
images can generally be captured at video rate in practice. In this paper, we
propose a model-based deep learning approach for merging an HrMS and LrHS
images to generate a high-resolution hyperspectral (HrHS) image. In specific,
we construct a novel MS/HS fusion model which takes the observation models of
low-resolution images and the low-rankness knowledge along the spectral mode of
HrHS image into consideration. Then we design an iterative algorithm to solve
the model by exploiting the proximal gradient method. And then, by unfolding
the designed algorithm, we construct a deep network, called MS/HS Fusion Net,
with learning the proximal operators and model parameters by convolutional
neural networks. Experimental results on simulated and real data substantiate
the superiority of our method both visually and quantitatively as compared with
state-of-the-art methods along this line of research.Comment: 10 pages, 7 figure
RCDNet: An Interpretable Rain Convolutional Dictionary Network for Single Image Deraining
As a common weather, rain streaks adversely degrade the image quality. Hence,
removing rains from an image has become an important issue in the field. To
handle such an ill-posed single image deraining task, in this paper, we
specifically build a novel deep architecture, called rain convolutional
dictionary network (RCDNet), which embeds the intrinsic priors of rain streaks
and has clear interpretability. In specific, we first establish a RCD model for
representing rain streaks and utilize the proximal gradient descent technique
to design an iterative algorithm only containing simple operators for solving
the model. By unfolding it, we then build the RCDNet in which every network
module has clear physical meanings and corresponds to each operation involved
in the algorithm. This good interpretability greatly facilitates an easy
visualization and analysis on what happens inside the network and why it works
well in inference process. Moreover, taking into account the domain gap issue
in real scenarios, we further design a novel dynamic RCDNet, where the rain
kernels can be dynamically inferred corresponding to input rainy images and
then help shrink the space for rain layer estimation with few rain maps so as
to ensure a fine generalization performance in the inconsistent scenarios of
rain types between training and testing data. By end-to-end training such an
interpretable network, all involved rain kernels and proximal operators can be
automatically extracted, faithfully characterizing the features of both rain
and clean background layers, and thus naturally lead to better deraining
performance. Comprehensive experiments substantiate the superiority of our
method, especially on its well generality to diverse testing scenarios and good
interpretability for all its modules. Code is available in
\emph{\url{https://github.com/hongwang01/DRCDNet}}
Learning to Purify Noisy Labels via Meta Soft Label Corrector
Recent deep neural networks (DNNs) can easily overfit to biased training data
with noisy labels. Label correction strategy is commonly used to alleviate this
issue by designing a method to identity suspected noisy labels and then correct
them. Current approaches to correcting corrupted labels usually need certain
pre-defined label correction rules or manually preset hyper-parameters. These
fixed settings make it hard to apply in practice since the accurate label
correction usually related with the concrete problem, training data and the
temporal information hidden in dynamic iterations of training process. To
address this issue, we propose a meta-learning model which could estimate soft
labels through meta-gradient descent step under the guidance of noise-free meta
data. By viewing the label correction procedure as a meta-process and using a
meta-learner to automatically correct labels, we could adaptively obtain
rectified soft labels iteratively according to current training problems
without manually preset hyper-parameters. Besides, our method is model-agnostic
and we can combine it with any other existing model with ease. Comprehensive
experiments substantiate the superiority of our method in both synthetic and
real-world problems with noisy labels compared with current SOTA label
correction strategies.Comment: 12 pages,6 figure
Orientation-Shared Convolution Representation for CT Metal Artifact Learning
During X-ray computed tomography (CT) scanning, metallic implants carrying
with patients often lead to adverse artifacts in the captured CT images and
then impair the clinical treatment. Against this metal artifact reduction (MAR)
task, the existing deep-learning-based methods have gained promising
reconstruction performance. Nevertheless, there is still some room for further
improvement of MAR performance and generalization ability, since some important
prior knowledge underlying this specific task has not been fully exploited.
Hereby, in this paper, we carefully analyze the characteristics of metal
artifacts and propose an orientation-shared convolution representation strategy
to adapt the physical prior structures of artifacts, i.e., rotationally
symmetrical streaking patterns. The proposed method rationally adopts
Fourier-series-expansion-based filter parametrization in artifact modeling,
which can better separate artifacts from anatomical tissues and boost the model
generalizability. Comprehensive experiments executed on synthesized and
clinical datasets show the superiority of our method in detail preservation
beyond the current representative MAR methods. Code will be available at
\url{https://github.com/hongwang01/OSCNet
A Learnable Optimization and Regularization Approach to Massive MIMO CSI Feedback
Channel state information (CSI) plays a critical role in achieving the
potential benefits of massive multiple input multiple output (MIMO) systems. In
frequency division duplex (FDD) massive MIMO systems, the base station (BS)
relies on sustained and accurate CSI feedback from the users. However, due to
the large number of antennas and users being served in massive MIMO systems,
feedback overhead can become a bottleneck. In this paper, we propose a
model-driven deep learning method for CSI feedback, called learnable
optimization and regularization algorithm (LORA). Instead of using l1-norm as
the regularization term, a learnable regularization module is introduced in
LORA to automatically adapt to the characteristics of CSI. We unfold the
conventional iterative shrinkage-thresholding algorithm (ISTA) to a neural
network and learn both the optimization process and regularization term by
end-toend training. We show that LORA improves the CSI feedback accuracy and
speed. Besides, a novel learnable quantization method and the corresponding
training scheme are proposed, and it is shown that LORA can operate
successfully at different bit rates, providing flexibility in terms of the CSI
feedback overhead. Various realistic scenarios are considered to demonstrate
the effectiveness and robustness of LORA through numerical simulations
- …