1 research outputs found
AdaptiveWeighted Attention Network with Camera Spectral Sensitivity Prior for Spectral Reconstruction from RGB Images
Recent promising effort for spectral reconstruction (SR) focuses on learning
a complicated mapping through using a deeper and wider convolutional neural
networks (CNNs). Nevertheless, most CNN-based SR algorithms neglect to explore
the camera spectral sensitivity (CSS) prior and interdependencies among
intermediate features, thus limiting the representation ability of the network
and performance of SR. To conquer these issues, we propose a novel adaptive
weighted attention network (AWAN) for SR, whose backbone is stacked with
multiple dual residual attention blocks (DRAB) decorating with long and short
skip connections to form the dual residual learning. Concretely, we investigate
an adaptive weighted channel attention (AWCA) module to reallocate channel-wise
feature responses via integrating correlations between channels. Furthermore, a
patch-level second-order non-local (PSNL) module is developed to capture
long-range spatial contextual information by second-order non-local operations
for more powerful feature representations. Based on the fact that the recovered
RGB images can be projected by the reconstructed hyperspectral image (HSI) and
the given CSS function, we incorporate the discrepancies of the RGB images and
HSIs as a finer constraint for more accurate reconstruction. Experimental
results demonstrate the effectiveness of our proposed AWAN network in terms of
quantitative comparison and perceptual quality over other state-of-the-art SR
methods. In the NTIRE 2020 Spectral Reconstruction Challenge, our entries
obtain the 1st ranking on the Clean track and the 3rd place on the Real World
track. Codes are available at https://github.com/Deep-imagelab/AWAN.Comment: The 1st ranking on the Clean track and the 3rd place only 1.59106e-4
more than the 1st on the Real World track of the NTIRE 2020 Spectral
Reconstruction Challeng