4,140 research outputs found
Deep Burst Denoising
Noise is an inherent issue of low-light image capture, one which is
exacerbated on mobile devices due to their narrow apertures and small sensors.
One strategy for mitigating noise in a low-light situation is to increase the
shutter time of the camera, thus allowing each photosite to integrate more
light and decrease noise variance. However, there are two downsides of long
exposures: (a) bright regions can exceed the sensor range, and (b) camera and
scene motion will result in blurred images. Another way of gathering more light
is to capture multiple short (thus noisy) frames in a "burst" and intelligently
integrate the content, thus avoiding the above downsides. In this paper, we use
the burst-capture strategy and implement the intelligent integration via a
recurrent fully convolutional deep neural net (CNN). We build our novel,
multiframe architecture to be a simple addition to any single frame denoising
model, and design to handle an arbitrary number of noisy input frames. We show
that it achieves state of the art denoising results on our burst dataset,
improving on the best published multi-frame techniques, such as VBM4D and
FlexISP. Finally, we explore other applications of image enhancement by
integrating content from multiple frames and demonstrate that our DNN
architecture generalizes well to image super-resolution
Signal reconstruction via operator guiding
Signal reconstruction from a sample using an orthogonal projector onto a
guiding subspace is theoretically well justified, but may be difficult to
practically implement. We propose more general guiding operators, which
increase signal components in the guiding subspace relative to those in a
complementary subspace, e.g., iterative low-pass edge-preserving filters for
super-resolution of images. Two examples of super-resolution illustrate our
technology: a no-flash RGB photo guided using a high resolution flash RGB
photo, and a depth image guided using a high resolution RGB photo.Comment: 5 pages, 8 figures. To appear in Proceedings of SampTA 2017: Sampling
Theory and Applications, 12th International Conference, July 3-7, 2017,
Tallinn, Estoni
Rethinking the Pipeline of Demosaicing, Denoising and Super-Resolution
Incomplete color sampling, noise degradation, and limited resolution are the
three key problems that are unavoidable in modern camera systems. Demosaicing
(DM), denoising (DN), and super-resolution (SR) are core components in a
digital image processing pipeline to overcome the three problems above,
respectively. Although each of these problems has been studied actively, the
mixture problem of DM, DN, and SR, which is a higher practical value, lacks
enough attention. Such a mixture problem is usually solved by a sequential
solution (applying each method independently in a fixed order: DM DN
SR), or is simply tackled by an end-to-end network without enough
analysis into interactions among tasks, resulting in an undesired performance
drop in the final image quality. In this paper, we rethink the mixture problem
from a holistic perspective and propose a new image processing pipeline: DN
SR DM. Extensive experiments show that simply modifying the usual
sequential solution by leveraging our proposed pipeline could enhance the image
quality by a large margin. We further adopt the proposed pipeline into an
end-to-end network, and present Trinity Enhancement Network (TENet).
Quantitative and qualitative experiments demonstrate the superiority of our
TENet to the state-of-the-art. Besides, we notice the literature lacks a full
color sampled dataset. To this end, we contribute a new high-quality full color
sampled real-world dataset, namely PixelShift200. Our experiments show the
benefit of the proposed PixelShift200 dataset for raw image processing.Comment: Code is available at: https://github.com/guochengqian/TENe
Learning Deep CNN Denoiser Prior for Image Restoration
Model-based optimization methods and discriminative learning methods have
been the two dominant strategies for solving various inverse problems in
low-level vision. Typically, those two kinds of methods have their respective
merits and drawbacks, e.g., model-based optimization methods are flexible for
handling different inverse problems but are usually time-consuming with
sophisticated priors for the purpose of good performance; in the meanwhile,
discriminative learning methods have fast testing speed but their application
range is greatly restricted by the specialized task. Recent works have revealed
that, with the aid of variable splitting techniques, denoiser prior can be
plugged in as a modular part of model-based optimization methods to solve other
inverse problems (e.g., deblurring). Such an integration induces considerable
advantage when the denoiser is obtained via discriminative learning. However,
the study of integration with fast discriminative denoiser prior is still
lacking. To this end, this paper aims to train a set of fast and effective CNN
(convolutional neural network) denoisers and integrate them into model-based
optimization method to solve other inverse problems. Experimental results
demonstrate that the learned set of denoisers not only achieve promising
Gaussian denoising results but also can be used as prior to deliver good
performance for various low-level vision applications.Comment: Accepted to CVPR 2017. Code: https://github.com/cszn/ircn
Modulating Image Restoration with Continual Levels via Adaptive Feature Modification Layers
In image restoration tasks, like denoising and super resolution, continual
modulation of restoration levels is of great importance for real-world
applications, but has failed most of existing deep learning based image
restoration methods. Learning from discrete and fixed restoration levels, deep
models cannot be easily generalized to data of continuous and unseen levels.
This topic is rarely touched in literature, due to the difficulty of modulating
well-trained models with certain hyper-parameters. We make a step forward by
proposing a unified CNN framework that consists of few additional parameters
than a single-level model yet could handle arbitrary restoration levels between
a start and an end level. The additional module, namely AdaFM layer, performs
channel-wise feature modification, and can adapt a model to another restoration
level with high accuracy. By simply tweaking an interpolation coefficient, the
intermediate model - AdaFM-Net could generate smooth and continuous restoration
effects without artifacts. Extensive experiments on three image restoration
tasks demonstrate the effectiveness of both model training and modulation
testing. Besides, we carefully investigate the properties of AdaFM layers,
providing a detailed guidance on the usage of the proposed method.Comment: Accepted by CVPR 2019 (oral); code is available:
https://github.com/hejingwenhejingwen/AdaF
- …