12,608 research outputs found
Exploiting Raw Images for Real-Scene Super-Resolution
Super-resolution is a fundamental problem in computer vision which aims to
overcome the spatial limitation of camera sensors. While significant progress
has been made in single image super-resolution, most algorithms only perform
well on synthetic data, which limits their applications in real scenarios. In
this paper, we study the problem of real-scene single image super-resolution to
bridge the gap between synthetic data and real captured images. We focus on two
issues of existing super-resolution algorithms: lack of realistic training data
and insufficient utilization of visual information obtained from cameras. To
address the first issue, we propose a method to generate more realistic
training data by mimicking the imaging process of digital cameras. For the
second issue, we develop a two-branch convolutional neural network to exploit
the radiance information originally-recorded in raw images. In addition, we
propose a dense channel-attention block for better image restoration as well as
a learning-based guided filter network for effective color correction. Our
model is able to generalize to different cameras without deliberately training
on images from specific camera types. Extensive experiments demonstrate that
the proposed algorithm can recover fine details and clear structures, and
achieve high-quality results for single image super-resolution in real scenes.Comment: A larger version with higher-resolution figures is available at:
https://sites.google.com/view/xiangyuxu. arXiv admin note: text overlap with
arXiv:1905.1215
Image Super-Resolution Using Deep Convolutional Networks
We propose a deep learning method for single image super-resolution (SR). Our
method directly learns an end-to-end mapping between the low/high-resolution
images. The mapping is represented as a deep convolutional neural network (CNN)
that takes the low-resolution image as the input and outputs the
high-resolution one. We further show that traditional sparse-coding-based SR
methods can also be viewed as a deep convolutional network. But unlike
traditional methods that handle each component separately, our method jointly
optimizes all layers. Our deep CNN has a lightweight structure, yet
demonstrates state-of-the-art restoration quality, and achieves fast speed for
practical on-line usage. We explore different network structures and parameter
settings to achieve trade-offs between performance and speed. Moreover, we
extend our network to cope with three color channels simultaneously, and show
better overall reconstruction quality.Comment: 14 pages, 14 figures, journa
Super-Resolution via Deep Learning
The recent phenomenal interest in convolutional neural networks (CNNs) must
have made it inevitable for the super-resolution (SR) community to explore its
potential. The response has been immense and in the last three years, since the
advent of the pioneering work, there appeared too many works not to warrant a
comprehensive survey. This paper surveys the SR literature in the context of
deep learning. We focus on the three important aspects of multimedia - namely
image, video and multi-dimensions, especially depth maps. In each case, first
relevant benchmarks are introduced in the form of datasets and state of the art
SR methods, excluding deep learning. Next is a detailed analysis of the
individual works, each including a short description of the method and a
critique of the results with special reference to the benchmarking done. This
is followed by minimum overall benchmarking in the form of comparison on some
common dataset, while relying on the results reported in various works
Channel Splitting Network for Single MR Image Super-Resolution
High resolution magnetic resonance (MR) imaging is desirable in many clinical
applications due to its contribution to more accurate subsequent analyses and
early clinical diagnoses. Single image super resolution (SISR) is an effective
and cost efficient alternative technique to improve the spatial resolution of
MR images. In the past few years, SISR methods based on deep learning
techniques, especially convolutional neural networks (CNNs), have achieved
state-of-the-art performance on natural images. However, the information is
gradually weakened and training becomes increasingly difficult as the network
deepens. The problem is more serious for medical images because lacking high
quality and effective training samples makes deep models prone to underfitting
or overfitting. Nevertheless, many current models treat the hierarchical
features on different channels equivalently, which is not helpful for the
models to deal with the hierarchical features discriminatively and targetedly.
To this end, we present a novel channel splitting network (CSN) to ease the
representational burden of deep models. The proposed CSN model divides the
hierarchical features into two branches, i.e., residual branch and dense
branch, with different information transmissions. The residual branch is able
to promote feature reuse, while the dense branch is beneficial to the
exploration of new features. Besides, we also adopt the merge-and-run mapping
to facilitate information integration between different branches. Extensive
experiments on various MR images, including proton density (PD), T1 and T2
images, show that the proposed CSN model achieves superior performance over
other state-of-the-art SISR methods.Comment: 13 pages, 11 figures and 4 table
A Deep Journey into Super-resolution: A survey
Deep convolutional networks based super-resolution is a fast-growing field
with numerous practical applications. In this exposition, we extensively
compare 30+ state-of-the-art super-resolution Convolutional Neural Networks
(CNNs) over three classical and three recently introduced challenging datasets
to benchmark single image super-resolution. We introduce a taxonomy for
deep-learning based super-resolution networks that groups existing methods into
nine categories including linear, residual, multi-branch, recursive,
progressive, attention-based and adversarial designs. We also provide
comparisons between the models in terms of network complexity, memory
footprint, model input and output, learning details, the type of network losses
and important architectural differences (e.g., depth, skip-connections,
filters). The extensive evaluation performed, shows the consistent and rapid
growth in the accuracy in the past few years along with a corresponding boost
in model complexity and the availability of large-scale datasets. It is also
observed that the pioneering methods identified as the benchmark have been
significantly outperformed by the current contenders. Despite the progress in
recent years, we identify several shortcomings of existing techniques and
provide future research directions towards the solution of these open problems.Comment: Accepted in ACM Computing Survey
Structure-Preserving Image Super-resolution via Contextualized Multi-task Learning
Single image super resolution (SR), which refers to reconstruct a
higher-resolution (HR) image from the observed low-resolution (LR) image, has
received substantial attention due to its tremendous application potentials.
Despite the breakthroughs of recently proposed SR methods using convolutional
neural networks (CNNs), their generated results usually lack of preserving
structural (high-frequency) details. In this paper, regarding global boundary
context and residual context as complimentary information for enhancing
structural details in image restoration, we develop a contextualized multi-task
learning framework to address the SR problem. Specifically, our method first
extracts convolutional features from the input LR image and applies one
deconvolutional module to interpolate the LR feature maps in a content-adaptive
way. Then, the resulting feature maps are fed into two branched sub-networks.
During the neural network training, one sub-network outputs salient image
boundaries and the HR image, and the other sub-network outputs the local
residual map, i.e., the residual difference between the generated HR image and
ground-truth image. On several standard benchmarks (i.e., Set5, Set14 and
BSD200), our extensive evaluations demonstrate the effectiveness of our SR
method on achieving both higher restoration quality and computational
efficiency compared with several state-of-the-art SR approaches. The source
code and some SR results can be found at:
http://hcp.sysu.edu.cn/structure-preserving-image-super-resolution/Comment: To appear in Transactions on Multimedia 201
NTIRE 2020 Challenge on Image Demoireing: Methods and Results
This paper reviews the Challenge on Image Demoireing that was part of the New
Trends in Image Restoration and Enhancement (NTIRE) workshop, held in
conjunction with CVPR 2020. Demoireing is a difficult task of removing moire
patterns from an image to reveal an underlying clean image. The challenge was
divided into two tracks. Track 1 targeted the single image demoireing problem,
which seeks to remove moire patterns from a single image. Track 2 focused on
the burst demoireing problem, where a set of degraded moire images of the same
scene were provided as input, with the goal of producing a single demoired
image as output. The methods were ranked in terms of their fidelity, measured
using the peak signal-to-noise ratio (PSNR) between the ground truth clean
images and the restored images produced by the participants' methods. The
tracks had 142 and 99 registered participants, respectively, with a total of 14
and 6 submissions in the final testing stage. The entries span the current
state-of-the-art in image and burst image demoireing problems
Channel Attention and Multi-level Features Fusion for Single Image Super-Resolution
Convolutional neural networks (CNNs) have demonstrated superior performance
in super-resolution (SR). However, most CNN-based SR methods neglect the
different importance among feature channels or fail to take full advantage of
the hierarchical features. To address these issues, this paper presents a novel
recursive unit. Firstly, at the beginning of each unit, we adopt a compact
channel attention mechanism to adaptively recalibrate the channel importance of
input features. Then, the multi-level features, rather than only deep-level
features, are extracted and fused. Additionally, we find that it will force our
model to learn more details by using the learnable upsampling method (i.e.,
transposed convolution) only on residual branch (instead of using it both on
residual branch and identity branch) while using the bicubic interpolation on
the other branch. Analytic experiments show that our method achieves
competitive results compared with the state-of-the-art methods and maintains
faster speed as well.Comment: 4 pages, 3 figures, Accepted as an oral presentation at VCI
Adapting Image Super-Resolution State-of-the-arts and Learning Multi-model Ensemble for Video Super-Resolution
Recently, image super-resolution has been widely studied and achieved
significant progress by leveraging the power of deep convolutional neural
networks. However, there has been limited advancement in video super-resolution
(VSR) due to the complex temporal patterns in videos. In this paper, we
investigate how to adapt state-of-the-art methods of image super-resolution for
video super-resolution. The proposed adapting method is straightforward. The
information among successive frames is well exploited, while the overhead on
the original image super-resolution method is negligible. Furthermore, we
propose a learning-based method to ensemble the outputs from multiple
super-resolution models. Our methods show superior performance and rank second
in the NTIRE2019 Video Super-Resolution Challenge Track 1
A Matrix-in-matrix Neural Network for Image Super Resolution
In recent years, deep learning methods have achieved impressive results with
higher peak signal-to-noise ratio in single image super-resolution (SISR) tasks
by utilizing deeper layers. However, their application is quite limited since
they require high computing power. In addition, most of the existing methods
rarely take full advantage of the intermediate features which are helpful for
restoration. To address these issues, we propose a moderate-size SISR net work
named matrixed channel attention network (MCAN) by constructing a matrix
ensemble of multi-connected channel attention blocks (MCAB). Several models of
different sizes are released to meet various practical requirements.
Conclusions can be drawn from our extensive benchmark experiments that the
proposed models achieve better performance with much fewer multiply-adds and
parameters. Our models will be made publicly available
- …