22,360 research outputs found
Image Super-Resolution Using Very Deep Residual Channel Attention Networks
Convolutional neural network (CNN) depth is of crucial importance for image
super-resolution (SR). However, we observe that deeper networks for image SR
are more difficult to train. The low-resolution inputs and features contain
abundant low-frequency information, which is treated equally across channels,
hence hindering the representational ability of CNNs. To solve these problems,
we propose the very deep residual channel attention networks (RCAN).
Specifically, we propose a residual in residual (RIR) structure to form very
deep network, which consists of several residual groups with long skip
connections. Each residual group contains some residual blocks with short skip
connections. Meanwhile, RIR allows abundant low-frequency information to be
bypassed through multiple skip connections, making the main network focus on
learning high-frequency information. Furthermore, we propose a channel
attention mechanism to adaptively rescale channel-wise features by considering
interdependencies among channels. Extensive experiments show that our RCAN
achieves better accuracy and visual improvements against state-of-the-art
methods.Comment: To appear in ECCV 201
Triple Attention Mixed Link Network for Single Image Super Resolution
Single image super resolution is of great importance as a low-level computer
vision task. Recent approaches with deep convolutional neural networks have
achieved im-pressive performance. However, existing architectures have
limitations due to the less sophisticated structure along with less strong
representational power. In this work, to significantly enhance the feature
representation, we proposed Triple Attention mixed link Network (TAN) which
consists of 1) three different aspects (i.e., kernel, spatial and channel) of
attention mechanisms and 2) fu-sion of both powerful residual and dense
connections (i.e., mixed link). Specifically, the network with multi kernel
learns multi hierarchical representations under different receptive fields. The
output features are recalibrated by the effective kernel and channel attentions
and feed into next layer partly residual and partly dense, which filters the
information and enable the network to learn more powerful representations. The
features finally pass through the spatial attention in the reconstruction
network which generates a fusion of local and global information, let the
network restore more details and improves the quality of reconstructed images.
Thanks to the diverse feature recalibrations and the advanced information flow
topology, our proposed model is strong enough to per-form against the
state-of-the-art methods on the bench-mark evaluations
A Deep Journey into Super-resolution: A survey
Deep convolutional networks based super-resolution is a fast-growing field
with numerous practical applications. In this exposition, we extensively
compare 30+ state-of-the-art super-resolution Convolutional Neural Networks
(CNNs) over three classical and three recently introduced challenging datasets
to benchmark single image super-resolution. We introduce a taxonomy for
deep-learning based super-resolution networks that groups existing methods into
nine categories including linear, residual, multi-branch, recursive,
progressive, attention-based and adversarial designs. We also provide
comparisons between the models in terms of network complexity, memory
footprint, model input and output, learning details, the type of network losses
and important architectural differences (e.g., depth, skip-connections,
filters). The extensive evaluation performed, shows the consistent and rapid
growth in the accuracy in the past few years along with a corresponding boost
in model complexity and the availability of large-scale datasets. It is also
observed that the pioneering methods identified as the benchmark have been
significantly outperformed by the current contenders. Despite the progress in
recent years, we identify several shortcomings of existing techniques and
provide future research directions towards the solution of these open problems.Comment: Accepted in ACM Computing Survey
Single Image Super-Resolution via Residual Neuron Attention Networks
Deep Convolutional Neural Networks (DCNNs) have achieved impressive
performance in Single Image Super-Resolution (SISR). To further improve the
performance, existing CNN-based methods generally focus on designing deeper
architecture of the network. However, we argue blindly increasing network's
depth is not the most sensible way. In this paper, we propose a novel
end-to-end Residual Neuron Attention Networks (RNAN) for more efficient and
effective SISR. Structurally, our RNAN is a sequential integration of the
well-designed Global Context-enhanced Residual Groups (GCRGs), which extracts
super-resolved features from coarse to fine. Our GCRG is designed with two
novelties. Firstly, the Residual Neuron Attention (RNA) mechanism is proposed
in each block of GCRG to reveal the relevance of neurons for better feature
representation. Furthermore, the Global Context (GC) block is embedded into
RNAN at the end of each GCRG for effectively modeling the global contextual
information. Experiments results demonstrate that our RNAN achieves the
comparable results with state-of-the-art methods in terms of both quantitative
metrics and visual quality, however, with simplified network architecture.Comment: 6 pages, 4 figures, Accepted by IEEE ICIP 202
Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral Imagery
Recently, single gray/RGB image super-resolution reconstruction task has been
extensively studied and made significant progress by leveraging the advanced
machine learning techniques based on deep convolutional neural networks
(DCNNs). However, there has been limited technical development focusing on
single hyperspectral image super-resolution due to the high-dimensional and
complex spectral patterns in hyperspectral image. In this paper, we make a step
forward by investigating how to adapt state-of-the-art residual learning based
single gray/RGB image super-resolution approaches for computationally efficient
single hyperspectral image super-resolution, referred as SSPSR. Specifically,
we introduce a spatial-spectral prior network (SSPN) to fully exploit the
spatial information and the correlation between the spectra of the
hyperspectral data. Considering that the hyperspectral training samples are
scarce and the spectral dimension of hyperspectral image data is very high, it
is nontrivial to train a stable and effective deep network. Therefore, a group
convolution (with shared network parameters) and progressive upsampling
framework is proposed. This will not only alleviate the difficulty in feature
extraction due to high-dimension of the hyperspectral data, but also make the
training process more stable. To exploit the spatial and spectral prior, we
design a spatial-spectral block (SSB), which consists of a spatial residual
module and a spectral attention residual module. Experimental results on some
hyperspectral images demonstrate that the proposed SSPSR method enhances the
details of the recovered high-resolution hyperspectral images, and outperforms
state-of-the-arts. The source code is available at
\url{https://github.com/junjun-jiang/SSPSRComment: Accepted for publication at IEEE Transactions on Computational
Imagin
A Matrix-in-matrix Neural Network for Image Super Resolution
In recent years, deep learning methods have achieved impressive results with
higher peak signal-to-noise ratio in single image super-resolution (SISR) tasks
by utilizing deeper layers. However, their application is quite limited since
they require high computing power. In addition, most of the existing methods
rarely take full advantage of the intermediate features which are helpful for
restoration. To address these issues, we propose a moderate-size SISR net work
named matrixed channel attention network (MCAN) by constructing a matrix
ensemble of multi-connected channel attention blocks (MCAB). Several models of
different sizes are released to meet various practical requirements.
Conclusions can be drawn from our extensive benchmark experiments that the
proposed models achieve better performance with much fewer multiply-adds and
parameters. Our models will be made publicly available
Channel Attention and Multi-level Features Fusion for Single Image Super-Resolution
Convolutional neural networks (CNNs) have demonstrated superior performance
in super-resolution (SR). However, most CNN-based SR methods neglect the
different importance among feature channels or fail to take full advantage of
the hierarchical features. To address these issues, this paper presents a novel
recursive unit. Firstly, at the beginning of each unit, we adopt a compact
channel attention mechanism to adaptively recalibrate the channel importance of
input features. Then, the multi-level features, rather than only deep-level
features, are extracted and fused. Additionally, we find that it will force our
model to learn more details by using the learnable upsampling method (i.e.,
transposed convolution) only on residual branch (instead of using it both on
residual branch and identity branch) while using the bicubic interpolation on
the other branch. Analytic experiments show that our method achieves
competitive results compared with the state-of-the-art methods and maintains
faster speed as well.Comment: 4 pages, 3 figures, Accepted as an oral presentation at VCI
Adapting Image Super-Resolution State-of-the-arts and Learning Multi-model Ensemble for Video Super-Resolution
Recently, image super-resolution has been widely studied and achieved
significant progress by leveraging the power of deep convolutional neural
networks. However, there has been limited advancement in video super-resolution
(VSR) due to the complex temporal patterns in videos. In this paper, we
investigate how to adapt state-of-the-art methods of image super-resolution for
video super-resolution. The proposed adapting method is straightforward. The
information among successive frames is well exploited, while the overhead on
the original image super-resolution method is negligible. Furthermore, we
propose a learning-based method to ensemble the outputs from multiple
super-resolution models. Our methods show superior performance and rank second
in the NTIRE2019 Video Super-Resolution Challenge Track 1
Pyramid Attention Networks for Image Restoration
Self-similarity refers to the image prior widely used in image restoration
algorithms that small but similar patterns tend to occur at different locations
and scales. However, recent advanced deep convolutional neural network based
methods for image restoration do not take full advantage of self-similarities
by relying on self-attention neural modules that only process information at
the same scale. To solve this problem, we present a novel Pyramid Attention
module for image restoration, which captures long-range feature correspondences
from a multi-scale feature pyramid. Inspired by the fact that corruptions, such
as noise or compression artifacts, drop drastically at coarser image scales,
our attention module is designed to be able to borrow clean signals from their
"clean" correspondences at the coarser levels. The proposed pyramid attention
module is a generic building block that can be flexibly integrated into various
neural architectures. Its effectiveness is validated through extensive
experiments on multiple image restoration tasks: image denoising, demosaicing,
compression artifact reduction, and super resolution. Without any bells and
whistles, our PANet (pyramid attention module with simple network backbones)
can produce state-of-the-art results with superior accuracy and visual quality.
Our code will be available at
https://github.com/SHI-Labs/Pyramid-Attention-Network
NTIRE 2020 Challenge on Image and Video Deblurring
Motion blur is one of the most common degradation artifacts in dynamic scene
photography. This paper reviews the NTIRE 2020 Challenge on Image and Video
Deblurring. In this challenge, we present the evaluation results from 3
competition tracks as well as the proposed solutions. Track 1 aims to develop
single-image deblurring methods focusing on restoration quality. On Track 2,
the image deblurring methods are executed on a mobile platform to find the
balance of the running speed and the restoration accuracy. Track 3 targets
developing video deblurring methods that exploit the temporal relation between
input frames. In each competition, there were 163, 135, and 102 registered
participants and in the final testing phase, 9, 4, and 7 teams competed. The
winning methods demonstrate the state-ofthe-art performance on image and video
deblurring tasks.Comment: To be published in CVPR 2020 Workshop (New Trends in Image
Restoration and Enhancement
- …