23,721 research outputs found
Structure-Preserving Image Super-resolution via Contextualized Multi-task Learning
Single image super resolution (SR), which refers to reconstruct a
higher-resolution (HR) image from the observed low-resolution (LR) image, has
received substantial attention due to its tremendous application potentials.
Despite the breakthroughs of recently proposed SR methods using convolutional
neural networks (CNNs), their generated results usually lack of preserving
structural (high-frequency) details. In this paper, regarding global boundary
context and residual context as complimentary information for enhancing
structural details in image restoration, we develop a contextualized multi-task
learning framework to address the SR problem. Specifically, our method first
extracts convolutional features from the input LR image and applies one
deconvolutional module to interpolate the LR feature maps in a content-adaptive
way. Then, the resulting feature maps are fed into two branched sub-networks.
During the neural network training, one sub-network outputs salient image
boundaries and the HR image, and the other sub-network outputs the local
residual map, i.e., the residual difference between the generated HR image and
ground-truth image. On several standard benchmarks (i.e., Set5, Set14 and
BSD200), our extensive evaluations demonstrate the effectiveness of our SR
method on achieving both higher restoration quality and computational
efficiency compared with several state-of-the-art SR approaches. The source
code and some SR results can be found at:
http://hcp.sysu.edu.cn/structure-preserving-image-super-resolution/Comment: To appear in Transactions on Multimedia 201
New Techniques for Preserving Global Structure and Denoising with Low Information Loss in Single-Image Super-Resolution
This work identifies and addresses two important technical challenges in
single-image super-resolution: (1) how to upsample an image without magnifying
noise and (2) how to preserve large scale structure when upsampling. We
summarize the techniques we developed for our second place entry in Track 1
(Bicubic Downsampling), seventh place entry in Track 2 (Realistic Adverse
Conditions), and seventh place entry in Track 3 (Realistic difficult) in the
2018 NTIRE Super-Resolution Challenge. Furthermore, we present new neural
network architectures that specifically address the two challenges listed
above: denoising and preservation of large-scale structure.Comment: 8 pages, CVPR workshop 201
Super-Resolution via Deep Learning
The recent phenomenal interest in convolutional neural networks (CNNs) must
have made it inevitable for the super-resolution (SR) community to explore its
potential. The response has been immense and in the last three years, since the
advent of the pioneering work, there appeared too many works not to warrant a
comprehensive survey. This paper surveys the SR literature in the context of
deep learning. We focus on the three important aspects of multimedia - namely
image, video and multi-dimensions, especially depth maps. In each case, first
relevant benchmarks are introduced in the form of datasets and state of the art
SR methods, excluding deep learning. Next is a detailed analysis of the
individual works, each including a short description of the method and a
critique of the results with special reference to the benchmarking done. This
is followed by minimum overall benchmarking in the form of comparison on some
common dataset, while relying on the results reported in various works
Decouple Learning for Parameterized Image Operators
Many different deep networks have been used to approximate, accelerate or
improve traditional image operators, such as image smoothing, super-resolution
and denoising. Among these traditional operators, many contain parameters which
need to be tweaked to obtain the satisfactory results, which we refer to as
"parameterized image operators". However, most existing deep networks trained
for these operators are only designed for one specific parameter configuration,
which does not meet the needs of real scenarios that usually require flexible
parameters settings. To overcome this limitation, we propose a new decouple
learning algorithm to learn from the operator parameters to dynamically adjust
the weights of a deep network for image operators, denoted as the base network.
The learned algorithm is formed as another network, namely the weight learning
network, which can be end-to-end jointly trained with the base network.
Experiments demonstrate that the proposed framework can be successfully applied
to many traditional parameterized image operators. We provide more analysis to
better understand the proposed framework, which may inspire more promising
research in this direction. Our codes and models have been released in
https://github.com/fqnchina/DecoupleLearningComment: Accepted by ECCV201
cvpaper.challenge in 2015 - A review of CVPR2015 and DeepSurvey
The "cvpaper.challenge" is a group composed of members from AIST, Tokyo Denki
Univ. (TDU), and Univ. of Tsukuba that aims to systematically summarize papers
on computer vision, pattern recognition, and related fields. For this
particular review, we focused on reading the ALL 602 conference papers
presented at the CVPR2015, the premier annual computer vision event held in
June 2015, in order to grasp the trends in the field. Further, we are proposing
"DeepSurvey" as a mechanism embodying the entire process from the reading
through all the papers, the generation of ideas, and to the writing of paper.Comment: Survey Pape
Data Driven Robust Image Guided Depth Map Restoration
Depth maps captured by modern depth cameras such as Kinect and Time-of-Flight
(ToF) are usually contaminated by missing data, noises and suffer from being of
low resolution. In this paper, we present a robust method for high-quality
restoration of a degraded depth map with the guidance of the corresponding
color image. We solve the problem in an energy optimization framework that
consists of a novel robust data term and smoothness term. To accommodate not
only the noise but also the inconsistency between depth discontinuities and the
color edges, we model both the data term and smoothness term with a robust
exponential error norm function. We propose to use Iteratively Re-weighted
Least Squares (IRLS) methods for efficiently solving the resulting highly
non-convex optimization problem. More importantly, we further develop a
data-driven adaptive parameter selection scheme to properly determine the
parameter in the model. We show that the proposed approach can preserve fine
details and sharp depth discontinuities even for a large upsampling factor
( for example). Experimental results on both simulated and real
datasets demonstrate that the proposed method outperforms recent
state-of-the-art methods in coping with the heavy noise, preserving sharp depth
discontinuities and suppressing the texture copy artifacts.Comment: 9 pages, 9 figures, conference pape
Rain O'er Me: Synthesizing real rain to derain with data distillation
We present a supervised technique for learning to remove rain from images
without using synthetic rain software. The method is based on a two-stage data
distillation approach: 1) A rainy image is first paired with a coarsely
derained version using on a simple filtering technique ("rain-to-clean"). 2)
Then a clean image is randomly matched with the rainy soft-labeled pair.
Through a shared deep neural network, the rain that is removed from the first
image is then added to the clean image to generate a second pair
("clean-to-rain"). The neural network simultaneously learns to map both images
such that high resolution structure in the clean images can inform the
deraining of the rainy images. Demonstrations show that this approach can
address those visual characteristics of rain not easily synthesized by software
in the usual way
Joint Image Filtering with Deep Convolutional Networks
Joint image filters leverage the guidance image as a prior and transfer the
structural details from the guidance image to the target image for suppressing
noise or enhancing spatial resolution. Existing methods either rely on various
explicit filter constructions or hand-designed objective functions, thereby
making it difficult to understand, improve, and accelerate these filters in a
coherent framework. In this paper, we propose a learning-based approach for
constructing joint filters based on Convolutional Neural Networks. In contrast
to existing methods that consider only the guidance image, the proposed
algorithm can selectively transfer salient structures that are consistent with
both guidance and target images. We show that the model trained on a certain
type of data, e.g., RGB and depth images, generalizes well to other modalities,
e.g., flash/non-Flash and RGB/NIR images. We validate the effectiveness of the
proposed joint filter through extensive experimental evaluations with
state-of-the-art methods.Comment: Accepted by TPAM
Super Resolution Convolutional Neural Network Models for Enhancing Resolution of Rock Micro-CT Images
Single Image Super Resolution (SISR) techniques based on Super Resolution
Convolutional Neural Networks (SRCNN) are applied to micro-computed tomography
({\mu}CT) images of sandstone and carbonate rocks. Digital rock imaging is
limited by the capability of the scanning device resulting in trade-offs
between resolution and field of view, and super resolution methods tested in
this study aim to compensate for these limits. SRCNN models SR-Resnet, Enhanced
Deep SR (EDSR), and Wide-Activation Deep SR (WDSR) are used on the Digital Rock
Super Resolution 1 (DRSRD1) Dataset of 4x downsampled images, comprising of
2000 high resolution (800x800) raw micro-CT images of Bentheimer sandstone and
Estaillades carbonate. The trained models are applied to the validation and
test data within the dataset and show a 3-5 dB rise in image quality compared
to bicubic interpolation, with all tested models performing within a 0.1 dB
range. Difference maps indicate that edge sharpness is completely recovered in
images within the scope of the trained model, with only high frequency noise
related detail loss. We find that aside from generation of high-resolution
images, a beneficial side effect of super resolution methods applied to
synthetically downgraded images is the removal of image noise while recovering
edgewise sharpness which is beneficial for the segmentation process. The model
is also tested against real low-resolution images of Bentheimer rock with image
augmentation to account for natural noise and blur. The SRCNN method is shown
to act as a preconditioner for image segmentation under these circumstances
which naturally leads to further future development and training of models that
segment an image directly. Image restoration by SRCNN on the rock images is of
significantly higher quality than traditional methods and suggests SRCNN
methods are a viable processing step in a digital rock workflow.Comment: 24 page
Densely Connected Pyramid Dehazing Network
We propose a new end-to-end single image dehazing method, called Densely
Connected Pyramid Dehazing Network (DCPDN), which can jointly learn the
transmission map, atmospheric light and dehazing all together. The end-to-end
learning is achieved by directly embedding the atmospheric scattering model
into the network, thereby ensuring that the proposed method strictly follows
the physics-driven scattering model for dehazing. Inspired by the dense network
that can maximize the information flow along features from different levels, we
propose a new edge-preserving densely connected encoder-decoder structure with
multi-level pyramid pooling module for estimating the transmission map. This
network is optimized using a newly introduced edge-preserving loss function. To
further incorporate the mutual structural information between the estimated
transmission map and the dehazed result, we propose a joint-discriminator based
on generative adversarial network framework to decide whether the corresponding
dehazed image and the estimated transmission map are real or fake. An ablation
study is conducted to demonstrate the effectiveness of each module evaluated at
both estimated transmission map and dehazed result. Extensive experiments
demonstrate that the proposed method achieves significant improvements over the
state-of-the-art methods. Code will be made available at:
https://github.com/hezhangsprinte
- …