2,470 research outputs found
End-to-End Image Super-Resolution via Deep and Shallow Convolutional Networks
One impressive advantage of convolutional neural networks (CNNs) is their
ability to automatically learn feature representation from raw pixels,
eliminating the need for hand-designed procedures. However, recent methods for
single image super-resolution (SR) fail to maintain this advantage. They
utilize CNNs in two decoupled steps, i.e., first upsampling the low resolution
(LR) image to the high resolution (HR) size with hand-designed techniques
(e.g., bicubic interpolation), and then applying CNNs on the upsampled LR image
to reconstruct HR results. In this paper, we seek an alternative and propose a
new image SR method, which jointly learns the feature extraction, upsampling
and HR reconstruction modules, yielding a completely end-to-end trainable deep
CNN. As opposed to existing approaches, the proposed method conducts upsampling
in the latent feature space with filters that are optimized for the task of
image SR. In addition, the HR reconstruction is performed in a multi-scale
manner to simultaneously incorporate both short- and long-range contextual
information, ensuring more accurate restoration of HR images. To facilitate
network training, a new training approach is designed, which jointly trains the
proposed deep network with a relatively shallow network, leading to faster
convergence and more superior performance. The proposed method is extensively
evaluated on widely adopted data sets and improves the performance of
state-of-the-art methods with a considerable margin. Moreover, in-depth
ablation studies are conducted to verify the contribution of different network
designs to image SR, providing additional insights for future research
Channel Splitting Network for Single MR Image Super-Resolution
High resolution magnetic resonance (MR) imaging is desirable in many clinical
applications due to its contribution to more accurate subsequent analyses and
early clinical diagnoses. Single image super resolution (SISR) is an effective
and cost efficient alternative technique to improve the spatial resolution of
MR images. In the past few years, SISR methods based on deep learning
techniques, especially convolutional neural networks (CNNs), have achieved
state-of-the-art performance on natural images. However, the information is
gradually weakened and training becomes increasingly difficult as the network
deepens. The problem is more serious for medical images because lacking high
quality and effective training samples makes deep models prone to underfitting
or overfitting. Nevertheless, many current models treat the hierarchical
features on different channels equivalently, which is not helpful for the
models to deal with the hierarchical features discriminatively and targetedly.
To this end, we present a novel channel splitting network (CSN) to ease the
representational burden of deep models. The proposed CSN model divides the
hierarchical features into two branches, i.e., residual branch and dense
branch, with different information transmissions. The residual branch is able
to promote feature reuse, while the dense branch is beneficial to the
exploration of new features. Besides, we also adopt the merge-and-run mapping
to facilitate information integration between different branches. Extensive
experiments on various MR images, including proton density (PD), T1 and T2
images, show that the proposed CSN model achieves superior performance over
other state-of-the-art SISR methods.Comment: 13 pages, 11 figures and 4 table
MAANet: Multi-view Aware Attention Networks for Image Super-Resolution
In most recent years, deep convolutional neural networks (DCNNs) based image
super-resolution (SR) has gained increasing attention in multimedia and
computer vision communities, focusing on restoring the high-resolution (HR)
image from a low-resolution (LR) image. However, one nonnegligible flaw of
DCNNs based methods is that most of them are not able to restore
high-resolution images containing sufficient high-frequency information from
low-resolution images with low-frequency information redundancy. Worse still,
as the depth of DCNNs increases, the training easily encounters the problem of
vanishing gradients, which makes the training more difficult. These problems
hinder the effectiveness of DCNNs in image SR task. To solve these problems, we
propose the Multi-view Aware Attention Networks (MAANet) for image SR task.
Specifically, we propose the local aware (LA) and global aware (GA) attention
to deal with LR features in unequal manners, which can highlight the
high-frequency components and discriminate each feature from LR images in the
local and the global views, respectively. Furthermore, we propose the local
attentive residual-dense (LARD) block, which combines the LA attention with
multiple residual and dense connections, to fit a deeper yet easy to train
architecture. The experimental results show that our proposed approach can
achieve remarkable performance compared with other state-of-the-art methods
Compression Artifacts Reduction by a Deep Convolutional Network
Lossy compression introduces complex compression artifacts, particularly the
blocking artifacts, ringing effects and blurring. Existing algorithms either
focus on removing blocking artifacts and produce blurred output, or restores
sharpened images that are accompanied with ringing effects. Inspired by the
deep convolutional networks (DCN) on super-resolution, we formulate a compact
and efficient network for seamless attenuation of different compression
artifacts. We also demonstrate that a deeper model can be effectively trained
with the features learned in a shallow network. Following a similar "easy to
hard" idea, we systematically investigate several practical transfer settings
and show the effectiveness of transfer learning in low-level vision problems.
Our method shows superior performance than the state-of-the-arts both on the
benchmark datasets and the real-world use case (i.e. Twitter). In addition, we
show that our method can be applied as pre-processing to facilitate other
low-level vision routines when they take compressed images as input.Comment: 9 pages, 12 figures, conferenc
Residual Dense Network for Image Super-Resolution
A very deep convolutional neural network (CNN) has recently achieved great
success for image super-resolution (SR) and offered hierarchical features as
well. However, most deep CNN based SR models do not make full use of the
hierarchical features from the original low-resolution (LR) images, thereby
achieving relatively-low performance. In this paper, we propose a novel
residual dense network (RDN) to address this problem in image SR. We fully
exploit the hierarchical features from all the convolutional layers.
Specifically, we propose residual dense block (RDB) to extract abundant local
features via dense connected convolutional layers. RDB further allows direct
connections from the state of preceding RDB to all the layers of current RDB,
leading to a contiguous memory (CM) mechanism. Local feature fusion in RDB is
then used to adaptively learn more effective features from preceding and
current local features and stabilizes the training of wider network. After
fully obtaining dense local features, we use global feature fusion to jointly
and adaptively learn global hierarchical features in a holistic way. Extensive
experiments on benchmark datasets with different degradation models show that
our RDN achieves favorable performance against state-of-the-art methods.Comment: To appear in CVPR 2018 as spotligh
Optical Flow Super-Resolution Based on Image Guidence Using Convolutional Neural Network
The convolutional neural network model for optical flow estimation usually
outputs a low-resolution(LR) optical flow field. To obtain the corresponding
full image resolution,interpolation and variational approach are the most
common options, which do not effectively improve the results. With the
motivation of various convolutional neural network(CNN) structures succeeded in
single image super-resolution(SISR) task, an end-to-end convolutional neural
network is proposed to reconstruct the high resolution(HR) optical flow field
from initial LR optical flow with the guidence of the first frame used in
optical flow estimation. Our optical flow super-resolution(OFSR) problem
differs from the general SISR problem in two main aspects. Firstly, the optical
flow includes less texture information than image so that the SISR CNN
structures can't be directly used in our OFSR problem. Secondly, the initial LR
optical flow data contains estimation error, while the LR image data for SISR
is generally a bicubic downsampled, blurred, and noisy version of HR ground
truth. We evaluate the proposed approach on two different optical flow
estimation mehods and show that it can not only obtain the full image
resolution, but generate more accurate optical flow field (Accuracy improve 15%
on FlyingChairs, 13% on MPI Sintel) with sharper edges than the estimation
result of original method.Comment: 20 pages,7 figure
Super Resolution Convolutional Neural Network Models for Enhancing Resolution of Rock Micro-CT Images
Single Image Super Resolution (SISR) techniques based on Super Resolution
Convolutional Neural Networks (SRCNN) are applied to micro-computed tomography
({\mu}CT) images of sandstone and carbonate rocks. Digital rock imaging is
limited by the capability of the scanning device resulting in trade-offs
between resolution and field of view, and super resolution methods tested in
this study aim to compensate for these limits. SRCNN models SR-Resnet, Enhanced
Deep SR (EDSR), and Wide-Activation Deep SR (WDSR) are used on the Digital Rock
Super Resolution 1 (DRSRD1) Dataset of 4x downsampled images, comprising of
2000 high resolution (800x800) raw micro-CT images of Bentheimer sandstone and
Estaillades carbonate. The trained models are applied to the validation and
test data within the dataset and show a 3-5 dB rise in image quality compared
to bicubic interpolation, with all tested models performing within a 0.1 dB
range. Difference maps indicate that edge sharpness is completely recovered in
images within the scope of the trained model, with only high frequency noise
related detail loss. We find that aside from generation of high-resolution
images, a beneficial side effect of super resolution methods applied to
synthetically downgraded images is the removal of image noise while recovering
edgewise sharpness which is beneficial for the segmentation process. The model
is also tested against real low-resolution images of Bentheimer rock with image
augmentation to account for natural noise and blur. The SRCNN method is shown
to act as a preconditioner for image segmentation under these circumstances
which naturally leads to further future development and training of models that
segment an image directly. Image restoration by SRCNN on the rock images is of
significantly higher quality than traditional methods and suggests SRCNN
methods are a viable processing step in a digital rock workflow.Comment: 24 page
Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral Imagery
Recently, single gray/RGB image super-resolution reconstruction task has been
extensively studied and made significant progress by leveraging the advanced
machine learning techniques based on deep convolutional neural networks
(DCNNs). However, there has been limited technical development focusing on
single hyperspectral image super-resolution due to the high-dimensional and
complex spectral patterns in hyperspectral image. In this paper, we make a step
forward by investigating how to adapt state-of-the-art residual learning based
single gray/RGB image super-resolution approaches for computationally efficient
single hyperspectral image super-resolution, referred as SSPSR. Specifically,
we introduce a spatial-spectral prior network (SSPN) to fully exploit the
spatial information and the correlation between the spectra of the
hyperspectral data. Considering that the hyperspectral training samples are
scarce and the spectral dimension of hyperspectral image data is very high, it
is nontrivial to train a stable and effective deep network. Therefore, a group
convolution (with shared network parameters) and progressive upsampling
framework is proposed. This will not only alleviate the difficulty in feature
extraction due to high-dimension of the hyperspectral data, but also make the
training process more stable. To exploit the spatial and spectral prior, we
design a spatial-spectral block (SSB), which consists of a spatial residual
module and a spectral attention residual module. Experimental results on some
hyperspectral images demonstrate that the proposed SSPSR method enhances the
details of the recovered high-resolution hyperspectral images, and outperforms
state-of-the-arts. The source code is available at
\url{https://github.com/junjun-jiang/SSPSRComment: Accepted for publication at IEEE Transactions on Computational
Imagin
Reverse Attention for Salient Object Detection
Benefit from the quick development of deep learning techniques, salient
object detection has achieved remarkable progresses recently. However, there
still exists following two major challenges that hinder its application in
embedded devices, low resolution output and heavy model weight. To this end,
this paper presents an accurate yet compact deep network for efficient salient
object detection. More specifically, given a coarse saliency prediction in the
deepest layer, we first employ residual learning to learn side-output residual
features for saliency refinement, which can be achieved with very limited
convolutional parameters while keep accuracy. Secondly, we further propose
reverse attention to guide such side-output residual learning in a top-down
manner. By erasing the current predicted salient regions from side-output
features, the network can eventually explore the missing object parts and
details which results in high resolution and accuracy. Experiments on six
benchmark datasets demonstrate that the proposed approach compares favorably
against state-of-the-art methods, and with advantages in terms of simplicity,
efficiency (45 FPS) and model size (81 MB).Comment: ECCV 201
Image Restoration Using Convolutional Auto-encoders with Symmetric Skip Connections
Image restoration, including image denoising, super resolution, inpainting,
and so on, is a well-studied problem in computer vision and image processing,
as well as a test bed for low-level image modeling algorithms. In this work, we
propose a very deep fully convolutional auto-encoder network for image
restoration, which is a encoding-decoding framework with symmetric
convolutional-deconvolutional layers. In other words, the network is composed
of multiple layers of convolution and de-convolution operators, learning
end-to-end mappings from corrupted images to the original ones. The
convolutional layers capture the abstraction of image contents while
eliminating corruptions. Deconvolutional layers have the capability to upsample
the feature maps and recover the image details. To deal with the problem that
deeper networks tend to be more difficult to train, we propose to symmetrically
link convolutional and deconvolutional layers with skip-layer connections, with
which the training converges much faster and attains better results.Comment: 17 pages. A journal extension of the version at arXiv:1603.0905
- …