6,599 research outputs found
Efficient Deep Neural Network for Photo-realistic Image Super-Resolution
Recent progress in the deep learning-based models has improved
photo-realistic (or perceptual) single-image super-resolution significantly.
However, despite their powerful performance, many methods are difficult to
apply to real-world applications because of the heavy computational
requirements. To facilitate the use of a deep model under such demands, we
focus on keeping the network efficient while maintaining its performance. In
detail, we design an architecture that implements a cascading mechanism on a
residual network to boost the performance with limited resources via
multi-level feature fusion. In addition, our proposed model adopts group
convolution and recursive scheme in order to achieve extreme efficiency. We
further improve the perceptual quality of the output by employing the
adversarial learning paradigm and a multi-scale discriminator approach. The
performance of our method is investigated through extensive internal
experiments and benchmark using various datasets. Our results show that our
models outperform the recent methods with similar complexity, for both
traditional pixel-based and perception-based tasks
Channel Attention and Multi-level Features Fusion for Single Image Super-Resolution
Convolutional neural networks (CNNs) have demonstrated superior performance
in super-resolution (SR). However, most CNN-based SR methods neglect the
different importance among feature channels or fail to take full advantage of
the hierarchical features. To address these issues, this paper presents a novel
recursive unit. Firstly, at the beginning of each unit, we adopt a compact
channel attention mechanism to adaptively recalibrate the channel importance of
input features. Then, the multi-level features, rather than only deep-level
features, are extracted and fused. Additionally, we find that it will force our
model to learn more details by using the learnable upsampling method (i.e.,
transposed convolution) only on residual branch (instead of using it both on
residual branch and identity branch) while using the bicubic interpolation on
the other branch. Analytic experiments show that our method achieves
competitive results compared with the state-of-the-art methods and maintains
faster speed as well.Comment: 4 pages, 3 figures, Accepted as an oral presentation at VCI
Residual Dense Network for Image Super-Resolution
A very deep convolutional neural network (CNN) has recently achieved great
success for image super-resolution (SR) and offered hierarchical features as
well. However, most deep CNN based SR models do not make full use of the
hierarchical features from the original low-resolution (LR) images, thereby
achieving relatively-low performance. In this paper, we propose a novel
residual dense network (RDN) to address this problem in image SR. We fully
exploit the hierarchical features from all the convolutional layers.
Specifically, we propose residual dense block (RDB) to extract abundant local
features via dense connected convolutional layers. RDB further allows direct
connections from the state of preceding RDB to all the layers of current RDB,
leading to a contiguous memory (CM) mechanism. Local feature fusion in RDB is
then used to adaptively learn more effective features from preceding and
current local features and stabilizes the training of wider network. After
fully obtaining dense local features, we use global feature fusion to jointly
and adaptively learn global hierarchical features in a holistic way. Extensive
experiments on benchmark datasets with different degradation models show that
our RDN achieves favorable performance against state-of-the-art methods.Comment: To appear in CVPR 2018 as spotligh
A Deep Journey into Super-resolution: A survey
Deep convolutional networks based super-resolution is a fast-growing field
with numerous practical applications. In this exposition, we extensively
compare 30+ state-of-the-art super-resolution Convolutional Neural Networks
(CNNs) over three classical and three recently introduced challenging datasets
to benchmark single image super-resolution. We introduce a taxonomy for
deep-learning based super-resolution networks that groups existing methods into
nine categories including linear, residual, multi-branch, recursive,
progressive, attention-based and adversarial designs. We also provide
comparisons between the models in terms of network complexity, memory
footprint, model input and output, learning details, the type of network losses
and important architectural differences (e.g., depth, skip-connections,
filters). The extensive evaluation performed, shows the consistent and rapid
growth in the accuracy in the past few years along with a corresponding boost
in model complexity and the availability of large-scale datasets. It is also
observed that the pioneering methods identified as the benchmark have been
significantly outperformed by the current contenders. Despite the progress in
recent years, we identify several shortcomings of existing techniques and
provide future research directions towards the solution of these open problems.Comment: Accepted in ACM Computing Survey
Triple Attention Mixed Link Network for Single Image Super Resolution
Single image super resolution is of great importance as a low-level computer
vision task. Recent approaches with deep convolutional neural networks have
achieved im-pressive performance. However, existing architectures have
limitations due to the less sophisticated structure along with less strong
representational power. In this work, to significantly enhance the feature
representation, we proposed Triple Attention mixed link Network (TAN) which
consists of 1) three different aspects (i.e., kernel, spatial and channel) of
attention mechanisms and 2) fu-sion of both powerful residual and dense
connections (i.e., mixed link). Specifically, the network with multi kernel
learns multi hierarchical representations under different receptive fields. The
output features are recalibrated by the effective kernel and channel attentions
and feed into next layer partly residual and partly dense, which filters the
information and enable the network to learn more powerful representations. The
features finally pass through the spatial attention in the reconstruction
network which generates a fusion of local and global information, let the
network restore more details and improves the quality of reconstructed images.
Thanks to the diverse feature recalibrations and the advanced information flow
topology, our proposed model is strong enough to per-form against the
state-of-the-art methods on the bench-mark evaluations
Residual Dense Network for Image Restoration
Convolutional neural network has recently achieved great success for image
restoration (IR) and also offered hierarchical features. However, most deep CNN
based IR models do not make full use of the hierarchical features from the
original low-quality images, thereby achieving relatively-low performance. In
this paper, we propose a novel residual dense network (RDN) to address this
problem in IR. We fully exploit the hierarchical features from all the
convolutional layers. Specifically, we propose residual dense block (RDB) to
extract abundant local features via densely connected convolutional layers. RDB
further allows direct connections from the state of preceding RDB to all the
layers of current RDB, leading to a contiguous memory mechanism. To adaptively
learn more effective features from preceding and current local features and
stabilize the training of wider network, we proposed local feature fusion in
RDB. After fully obtaining dense local features, we use global feature fusion
to jointly and adaptively learn global hierarchical features in a holistic way.
We demonstrate the effectiveness of RDN with several representative IR
applications, single image super-resolution, Gaussian image denoising, image
compression artifact reduction, and image deblurring. Experiments on benchmark
and real-world datasets show that our RDN achieves favorable performance
against state-of-the-art methods for each IR task quantitatively and visually.Comment: To appear in TPAMI. arXiv admin note: substantial text overlap with
arXiv:1802.0879
Deeply Supervised Depth Map Super-Resolution as Novel View Synthesis
Deep convolutional neural network (DCNN) has been successfully applied to
depth map super-resolution and outperforms existing methods by a wide margin.
However, there still exist two major issues with these DCNN based depth map
super-resolution methods that hinder the performance: i) The low-resolution
depth maps either need to be up-sampled before feeding into the network or
substantial deconvolution has to be used; and ii) The supervision
(high-resolution depth maps) is only applied at the end of the network, thus it
is difficult to handle large up-sampling factors, such as . In this paper, we propose a new framework to tackle the above problems.
First, we propose to represent the task of depth map super-resolution as a
series of novel view synthesis sub-tasks. The novel view synthesis sub-task
aims at generating (synthesizing) a depth map from different camera pose, which
could be learned in parallel. Second, to handle large up-sampling factors, we
present a deeply supervised network structure to enforce strong supervision in
each stage of the network. Third, a multi-scale fusion strategy is proposed to
effectively exploit the feature maps at different scales and handle the
blocking effect. In this way, our proposed framework could deal with
challenging depth map super-resolution efficiently under large up-sampling
factors (e.g. ). Our method only uses the low-resolution
depth map as input, and the support of color image is not needed, which greatly
reduces the restriction of our method. Extensive experiments on various
benchmarking datasets demonstrate the superiority of our method over current
state-of-the-art depth map super-resolution methods.Comment: Accepted by IEEE Transactions on Circuits and Systems for Video
Technology (T-CSVT) 201
Lightweight Image Super-Resolution with Adaptive Weighted Learning Network
Deep learning has been successfully applied to the single-image
super-resolution (SISR) task with great performance in recent years. However,
most convolutional neural network based SR models require heavy computation,
which limit their real-world applications. In this work, a lightweight SR
network, named Adaptive Weighted Super-Resolution Network (AWSRN), is proposed
for SISR to address this issue. A novel local fusion block (LFB) is designed in
AWSRN for efficient residual learning, which consists of stacked adaptive
weighted residual units (AWRU) and a local residual fusion unit (LRFU).
Moreover, an adaptive weighted multi-scale (AWMS) module is proposed to make
full use of features in reconstruction layer. AWMS consists of several
different scale convolutions, and the redundancy scale branch can be removed
according to the contribution of adaptive weights in AWMS for lightweight
network. The experimental results on the commonly used datasets show that the
proposed lightweight AWSRN achieves superior performance on x2, x3, x4, and x8
scale factors to state-of-the-art methods with similar parameters and
computational overhead. Code is avaliable at:
https://github.com/ChaofWang/AWSRNComment: 9 pages, 6 figure
A Matrix-in-matrix Neural Network for Image Super Resolution
In recent years, deep learning methods have achieved impressive results with
higher peak signal-to-noise ratio in single image super-resolution (SISR) tasks
by utilizing deeper layers. However, their application is quite limited since
they require high computing power. In addition, most of the existing methods
rarely take full advantage of the intermediate features which are helpful for
restoration. To address these issues, we propose a moderate-size SISR net work
named matrixed channel attention network (MCAN) by constructing a matrix
ensemble of multi-connected channel attention blocks (MCAB). Several models of
different sizes are released to meet various practical requirements.
Conclusions can be drawn from our extensive benchmark experiments that the
proposed models achieve better performance with much fewer multiply-adds and
parameters. Our models will be made publicly available
Channel Splitting Network for Single MR Image Super-Resolution
High resolution magnetic resonance (MR) imaging is desirable in many clinical
applications due to its contribution to more accurate subsequent analyses and
early clinical diagnoses. Single image super resolution (SISR) is an effective
and cost efficient alternative technique to improve the spatial resolution of
MR images. In the past few years, SISR methods based on deep learning
techniques, especially convolutional neural networks (CNNs), have achieved
state-of-the-art performance on natural images. However, the information is
gradually weakened and training becomes increasingly difficult as the network
deepens. The problem is more serious for medical images because lacking high
quality and effective training samples makes deep models prone to underfitting
or overfitting. Nevertheless, many current models treat the hierarchical
features on different channels equivalently, which is not helpful for the
models to deal with the hierarchical features discriminatively and targetedly.
To this end, we present a novel channel splitting network (CSN) to ease the
representational burden of deep models. The proposed CSN model divides the
hierarchical features into two branches, i.e., residual branch and dense
branch, with different information transmissions. The residual branch is able
to promote feature reuse, while the dense branch is beneficial to the
exploration of new features. Besides, we also adopt the merge-and-run mapping
to facilitate information integration between different branches. Extensive
experiments on various MR images, including proton density (PD), T1 and T2
images, show that the proposed CSN model achieves superior performance over
other state-of-the-art SISR methods.Comment: 13 pages, 11 figures and 4 table
- …