1,013 research outputs found
Extreme Channel Prior Embedded Network for Dynamic Scene Deblurring
Recent years have witnessed the significant progress on convolutional neural
networks (CNNs) in dynamic scene deblurring. While CNN models are generally
learned by the reconstruction loss defined on training data, incorporating
suitable image priors as well as regularization terms into the network
architecture could boost the deblurring performance. In this work, we propose
an Extreme Channel Prior embedded Network (ECPeNet) to plug the extreme channel
priors (i.e., priors on dark and bright channels) into a network architecture
for effective dynamic scene deblurring. A novel trainable extreme channel prior
embedded layer (ECPeL) is developed to aggregate both extreme channel and
blurry image representations, and sparse regularization is introduced to
regularize the ECPeNet model learning. Furthermore, we present an effective
multi-scale network architecture that works in both coarse-to-fine and
fine-to-coarse manners for better exploiting information flow across scales.
Experimental results on GoPro and Kohler datasets show that our proposed
ECPeNet performs favorably against state-of-the-art deep image deblurring
methods in terms of both quantitative metrics and visual quality.Comment: 10 page
Blind Image Deconvolution using Deep Generative Priors
This paper proposes a novel approach to regularize the \textit{ill-posed} and
\textit{non-linear} blind image deconvolution (blind deblurring) using deep
generative networks as priors. We employ two separate generative models --- one
trained to produce sharp images while the other trained to generate blur
kernels from lower-dimensional parameters. To deblur, we propose an alternating
gradient descent scheme operating in the latent lower-dimensional space of each
of the pretrained generative models. Our experiments show promising deblurring
results on images even under large blurs, and heavy noise. To address the
shortcomings of generative models such as mode collapse, we augment our
generative priors with classical image priors and report improved performance
on complex image datasets. The deblurring performance depends on how well the
range of the generator spans the image class. Interestingly, our experiments
show that even an untrained structured (convolutional) generative networks acts
as an image prior in the image deblurring context allowing us to extend our
results to more diverse natural image datasets
Adaptive Quantile Sparse Image (AQuaSI) Prior for Inverse Imaging Problems
Inverse problems play a central role for many classical computer vision and
image processing tasks. Many inverse problems are ill-posed, and hence require
a prior to regularize the solution space. However, many of the existing priors,
like total variation, are based on ad-hoc assumptions that have difficulties to
represent the actual distribution of natural images. Thus, a key challenge in
research on image processing is to find better suited priors to represent
natural images.
In this work, we propose the Adaptive Quantile Sparse Image (AQuaSI) prior.
It is based on a quantile filter, can be used as a joint filter on guidance
data, and be readily plugged into a wide range of numerical optimization
algorithms. We demonstrate the efficacy of the proposed prior in joint
RGB/depth upsampling, on RGB/NIR image restoration, and in a comparison with
related regularization by denoising approaches
Physics-Based Generative Adversarial Models for Image Restoration and Beyond
We present an algorithm to directly solve numerous image restoration problems
(e.g., image deblurring, image dehazing, image deraining, etc.). These problems
are highly ill-posed, and the common assumptions for existing methods are
usually based on heuristic image priors. In this paper, we find that these
problems can be solved by generative models with adversarial learning. However,
the basic formulation of generative adversarial networks (GANs) does not
generate realistic images, and some structures of the estimated images are
usually not preserved well. Motivated by an interesting observation that the
estimated results should be consistent with the observed inputs under the
physics models, we propose a physics model constrained learning algorithm so
that it can guide the estimation of the specific task in the conventional GAN
framework. The proposed algorithm is trained in an end-to-end fashion and can
be applied to a variety of image restoration and related low-level vision
problems. Extensive experiments demonstrate that our method performs favorably
against the state-of-the-art algorithms.Comment: IEEE TPAM
Learning to Deblur Images with Exemplars
Human faces are one interesting object class with numerous applications.
While significant progress has been made in the generic deblurring problem,
existing methods are less effective for blurry face images. The success of the
state-of-the-art image deblurring algorithms stems mainly from implicit or
explicit restoration of salient edges for kernel estimation. However, existing
methods are less effective as only few edges can be restored from blurry face
images for kernel estimation. In this paper, we address the problem of
deblurring face images by exploiting facial structures. We propose a deblurring
algorithm based on an exemplar dataset without using coarse-to-fine strategies
or heuristic edge selections. In addition, we develop a convolutional neural
network to restore sharp edges from blurry images for deblurring. Extensive
experiments against the state-of-the-art methods demonstrate the effectiveness
of the proposed algorithms for deblurring face images. In addition, we show the
proposed algorithms can be applied to image deblurring for other object
classes.Comment: Accepted by IEEE Transactions on Pattern Analysis and Machine
Intelligence 201
DAVANet: Stereo Deblurring with View Aggregation
Nowadays stereo cameras are more commonly adopted in emerging devices such as
dual-lens smartphones and unmanned aerial vehicles. However, they also suffer
from blurry images in dynamic scenes which leads to visual discomfort and
hampers further image processing. Previous works have succeeded in monocular
deblurring, yet there are few studies on deblurring for stereoscopic images. By
exploiting the two-view nature of stereo images, we propose a novel stereo
image deblurring network with Depth Awareness and View Aggregation, named
DAVANet. In our proposed network, 3D scene cues from the depth and varying
information from two views are incorporated, which help to remove complex
spatially-varying blur in dynamic scenes. Specifically, with our proposed
fusion network, we integrate the bidirectional disparities estimation and
deblurring into a unified framework. Moreover, we present a large-scale
multi-scene dataset for stereo deblurring, containing 20,637 blurry-sharp
stereo image pairs from 135 diverse sequences and their corresponding
bidirectional disparities. The experimental results on our dataset demonstrate
that DAVANet outperforms state-of-the-art methods in terms of accuracy, speed,
and model size.Comment: CVPR 2019 (Oral
Modelling the Scene Dependent Imaging in Cameras with a Deep Neural Network
We present a novel deep learning framework that models the scene dependent
image processing inside cameras. Often called as the radiometric calibration,
the process of recovering RAW images from processed images (JPEG format in the
sRGB color space) is essential for many computer vision tasks that rely on
physically accurate radiance values. All previous works rely on the
deterministic imaging model where the color transformation stays the same
regardless of the scene and thus they can only be applied for images taken
under the manual mode. In this paper, we propose a data-driven approach to
learn the scene dependent and locally varying image processing inside cameras
under the automode. Our method incorporates both the global and the local scene
context into pixel-wise features via multi-scale pyramid of learnable histogram
layers. The results show that we can model the imaging pipeline of different
cameras that operate under the automode accurately in both directions (from RAW
to sRGB, from sRGB to RAW) and we show how we can apply our method to improve
the performance of image deblurring.Comment: To appear in ICCV 201
Deep joint rain and haze removal from single images
Rain removal from a single image is a challenge which has been studied for a
long time. In this paper, a novel convolutional neural network based on wavelet
and dark channel is proposed. On one hand, we think that rain streaks
correspond to high frequency component of the image. Therefore, haar wavelet
transform is a good choice to separate the rain streaks and background to some
extent. More specifically, the LL subband of a rain image is more inclined to
express the background information, while LH, HL, HH subband tend to represent
the rain streaks and the edges. On the other hand, the accumulation of rain
streaks from long distance makes the rain image look like haze veil. We extract
dark channel of rain image as a feature map in network. By increasing this
mapping between the dark channel of input and output images, we achieve haze
removal in an indirect way. All of the parameters are optimized by
back-propagation. Experiments on both synthetic and real- world datasets reveal
that our method outperforms other state-of- the-art methods from a qualitative
and quantitative perspective.Comment: 6 page
Blind Motion Deblurring with Cycle Generative Adversarial Networks
Blind motion deblurring is one of the most basic and challenging problems in
image processing and computer vision. It aims to recover a sharp image from its
blurred version knowing nothing about the blur process. Many existing methods
use Maximum A Posteriori (MAP) or Expectation Maximization (EM) frameworks to
deal with this kind of problems, but they cannot handle well the figh frequency
features of natural images. Most recently, deep neural networks have been
emerging as a powerful tool for image deblurring. In this paper, we prove that
encoder-decoder architecture gives better results for image deblurring tasks.
In addition, we propose a novel end-to-end learning model which refines
generative adversarial network by many novel training strategies so as to
tackle the problem of deblurring. Experimental results show that our model can
capture high frequency features well, and the results on benchmark dataset show
that proposed model achieves the competitive performance
Spatio-Temporal Filter Adaptive Network for Video Deblurring
Video deblurring is a challenging task due to the spatially variant blur
caused by camera shake, object motions, and depth variations, etc. Existing
methods usually estimate optical flow in the blurry video to align consecutive
frames or approximate blur kernels. However, they tend to generate artifacts or
cannot effectively remove blur when the estimated optical flow is not accurate.
To overcome the limitation of separate optical flow estimation, we propose a
Spatio-Temporal Filter Adaptive Network (STFAN) for the alignment and
deblurring in a unified framework. The proposed STFAN takes both blurry and
restored images of the previous frame as well as blurry image of the current
frame as input, and dynamically generates the spatially adaptive filters for
the alignment and deblurring. We then propose the new Filter Adaptive
Convolutional (FAC) layer to align the deblurred features of the previous frame
with the current frame and remove the spatially variant blur from the features
of the current frame. Finally, we develop a reconstruction network which takes
the fusion of two transformed features to restore the clear frames. Both
quantitative and qualitative evaluation results on the benchmark datasets and
real-world videos demonstrate that the proposed algorithm performs favorably
against state-of-the-art methods in terms of accuracy, speed as well as model
size.Comment: ICCV 201
- …