29,006 research outputs found
Channel Attention and Multi-level Features Fusion for Single Image Super-Resolution
Convolutional neural networks (CNNs) have demonstrated superior performance
in super-resolution (SR). However, most CNN-based SR methods neglect the
different importance among feature channels or fail to take full advantage of
the hierarchical features. To address these issues, this paper presents a novel
recursive unit. Firstly, at the beginning of each unit, we adopt a compact
channel attention mechanism to adaptively recalibrate the channel importance of
input features. Then, the multi-level features, rather than only deep-level
features, are extracted and fused. Additionally, we find that it will force our
model to learn more details by using the learnable upsampling method (i.e.,
transposed convolution) only on residual branch (instead of using it both on
residual branch and identity branch) while using the bicubic interpolation on
the other branch. Analytic experiments show that our method achieves
competitive results compared with the state-of-the-art methods and maintains
faster speed as well.Comment: 4 pages, 3 figures, Accepted as an oral presentation at VCI
NTIRE 2020 Challenge on Image Demoireing: Methods and Results
This paper reviews the Challenge on Image Demoireing that was part of the New
Trends in Image Restoration and Enhancement (NTIRE) workshop, held in
conjunction with CVPR 2020. Demoireing is a difficult task of removing moire
patterns from an image to reveal an underlying clean image. The challenge was
divided into two tracks. Track 1 targeted the single image demoireing problem,
which seeks to remove moire patterns from a single image. Track 2 focused on
the burst demoireing problem, where a set of degraded moire images of the same
scene were provided as input, with the goal of producing a single demoired
image as output. The methods were ranked in terms of their fidelity, measured
using the peak signal-to-noise ratio (PSNR) between the ground truth clean
images and the restored images produced by the participants' methods. The
tracks had 142 and 99 registered participants, respectively, with a total of 14
and 6 submissions in the final testing stage. The entries span the current
state-of-the-art in image and burst image demoireing problems
A Matrix-in-matrix Neural Network for Image Super Resolution
In recent years, deep learning methods have achieved impressive results with
higher peak signal-to-noise ratio in single image super-resolution (SISR) tasks
by utilizing deeper layers. However, their application is quite limited since
they require high computing power. In addition, most of the existing methods
rarely take full advantage of the intermediate features which are helpful for
restoration. To address these issues, we propose a moderate-size SISR net work
named matrixed channel attention network (MCAN) by constructing a matrix
ensemble of multi-connected channel attention blocks (MCAB). Several models of
different sizes are released to meet various practical requirements.
Conclusions can be drawn from our extensive benchmark experiments that the
proposed models achieve better performance with much fewer multiply-adds and
parameters. Our models will be made publicly available
Triple Attention Mixed Link Network for Single Image Super Resolution
Single image super resolution is of great importance as a low-level computer
vision task. Recent approaches with deep convolutional neural networks have
achieved im-pressive performance. However, existing architectures have
limitations due to the less sophisticated structure along with less strong
representational power. In this work, to significantly enhance the feature
representation, we proposed Triple Attention mixed link Network (TAN) which
consists of 1) three different aspects (i.e., kernel, spatial and channel) of
attention mechanisms and 2) fu-sion of both powerful residual and dense
connections (i.e., mixed link). Specifically, the network with multi kernel
learns multi hierarchical representations under different receptive fields. The
output features are recalibrated by the effective kernel and channel attentions
and feed into next layer partly residual and partly dense, which filters the
information and enable the network to learn more powerful representations. The
features finally pass through the spatial attention in the reconstruction
network which generates a fusion of local and global information, let the
network restore more details and improves the quality of reconstructed images.
Thanks to the diverse feature recalibrations and the advanced information flow
topology, our proposed model is strong enough to per-form against the
state-of-the-art methods on the bench-mark evaluations
NTIRE 2020 Challenge on Image and Video Deblurring
Motion blur is one of the most common degradation artifacts in dynamic scene
photography. This paper reviews the NTIRE 2020 Challenge on Image and Video
Deblurring. In this challenge, we present the evaluation results from 3
competition tracks as well as the proposed solutions. Track 1 aims to develop
single-image deblurring methods focusing on restoration quality. On Track 2,
the image deblurring methods are executed on a mobile platform to find the
balance of the running speed and the restoration accuracy. Track 3 targets
developing video deblurring methods that exploit the temporal relation between
input frames. In each competition, there were 163, 135, and 102 registered
participants and in the final testing phase, 9, 4, and 7 teams competed. The
winning methods demonstrate the state-ofthe-art performance on image and video
deblurring tasks.Comment: To be published in CVPR 2020 Workshop (New Trends in Image
Restoration and Enhancement
EDVR: Video Restoration with Enhanced Deformable Convolutional Networks
Video restoration tasks, including super-resolution, deblurring, etc, are
drawing increasing attention in the computer vision community. A challenging
benchmark named REDS is released in the NTIRE19 Challenge. This new benchmark
challenges existing methods from two aspects: (1) how to align multiple frames
given large motions, and (2) how to effectively fuse different frames with
diverse motion and blur. In this work, we propose a novel Video Restoration
framework with Enhanced Deformable networks, termed EDVR, to address these
challenges. First, to handle large motions, we devise a Pyramid, Cascading and
Deformable (PCD) alignment module, in which frame alignment is done at the
feature level using deformable convolutions in a coarse-to-fine manner. Second,
we propose a Temporal and Spatial Attention (TSA) fusion module, in which
attention is applied both temporally and spatially, so as to emphasize
important features for subsequent restoration. Thanks to these modules, our
EDVR wins the champions and outperforms the second place by a large margin in
all four tracks in the NTIRE19 video restoration and enhancement challenges.
EDVR also demonstrates superior performance to state-of-the-art published
methods on video super-resolution and deblurring. The code is available at
https://github.com/xinntao/EDVR.Comment: To appear in CVPR 2019 Workshop. The winners in all four tracks in
the NTIRE 2019 video restoration and enhancement challenges. Project page:
https://xinntao.github.io/projects/EDVR , Code:
https://github.com/xinntao/EDV
NTIRE 2020 Challenge on Spectral Reconstruction from an RGB Image
This paper reviews the second challenge on spectral reconstruction from RGB
images, i.e., the recovery of whole-scene hyperspectral (HS) information from a
3-channel RGB image. As in the previous challenge, two tracks were provided:
(i) a "Clean" track where HS images are estimated from noise-free RGBs, the RGB
images are themselves calculated numerically using the ground-truth HS images
and supplied spectral sensitivity functions (ii) a "Real World" track,
simulating capture by an uncalibrated and unknown camera, where the HS images
are recovered from noisy JPEG-compressed RGB images. A new, larger-than-ever,
natural hyperspectral image data set is presented, containing a total of 510 HS
images. The Clean and Real World tracks had 103 and 78 registered participants
respectively, with 14 teams competing in the final testing phase. A description
of the proposed methods, alongside their challenge scores and an extensive
evaluation of top performing methods is also provided. They gauge the
state-of-the-art in spectral reconstruction from an RGB image
Adapting Image Super-Resolution State-of-the-arts and Learning Multi-model Ensemble for Video Super-Resolution
Recently, image super-resolution has been widely studied and achieved
significant progress by leveraging the power of deep convolutional neural
networks. However, there has been limited advancement in video super-resolution
(VSR) due to the complex temporal patterns in videos. In this paper, we
investigate how to adapt state-of-the-art methods of image super-resolution for
video super-resolution. The proposed adapting method is straightforward. The
information among successive frames is well exploited, while the overhead on
the original image super-resolution method is negligible. Furthermore, we
propose a learning-based method to ensemble the outputs from multiple
super-resolution models. Our methods show superior performance and rank second
in the NTIRE2019 Video Super-Resolution Challenge Track 1
NTIRE 2020 Challenge on Real Image Denoising: Dataset, Methods and Results
This paper reviews the NTIRE 2020 challenge on real image denoising with
focus on the newly introduced dataset, the proposed methods and their results.
The challenge is a new version of the previous NTIRE 2019 challenge on real
image denoising that was based on the SIDD benchmark. This challenge is based
on a newly collected validation and testing image datasets, and hence, named
SIDD+. This challenge has two tracks for quantitatively evaluating image
denoising performance in (1) the Bayer-pattern rawRGB and (2) the standard RGB
(sRGB) color spaces. Each track ~250 registered participants. A total of 22
teams, proposing 24 methods, competed in the final phase of the challenge. The
proposed methods by the participating teams represent the current
state-of-the-art performance in image denoising targeting real noisy images.
The newly collected SIDD+ datasets are publicly available at:
https://bit.ly/siddplus_data
A Deep Journey into Super-resolution: A survey
Deep convolutional networks based super-resolution is a fast-growing field
with numerous practical applications. In this exposition, we extensively
compare 30+ state-of-the-art super-resolution Convolutional Neural Networks
(CNNs) over three classical and three recently introduced challenging datasets
to benchmark single image super-resolution. We introduce a taxonomy for
deep-learning based super-resolution networks that groups existing methods into
nine categories including linear, residual, multi-branch, recursive,
progressive, attention-based and adversarial designs. We also provide
comparisons between the models in terms of network complexity, memory
footprint, model input and output, learning details, the type of network losses
and important architectural differences (e.g., depth, skip-connections,
filters). The extensive evaluation performed, shows the consistent and rapid
growth in the accuracy in the past few years along with a corresponding boost
in model complexity and the availability of large-scale datasets. It is also
observed that the pioneering methods identified as the benchmark have been
significantly outperformed by the current contenders. Despite the progress in
recent years, we identify several shortcomings of existing techniques and
provide future research directions towards the solution of these open problems.Comment: Accepted in ACM Computing Survey
- …