667 research outputs found
Burst Denoising with Kernel Prediction Networks
We present a technique for jointly denoising bursts of images taken from a
handheld camera. In particular, we propose a convolutional neural network
architecture for predicting spatially varying kernels that can both align and
denoise frames, a synthetic data generation approach based on a realistic noise
formation model, and an optimization guided by an annealed loss function to
avoid undesirable local minima. Our model matches or outperforms the
state-of-the-art across a wide range of noise levels on both real and synthetic
data.Comment: To appear in CVPR 2018 (spotlight). Project page:
http://people.eecs.berkeley.edu/~bmild/kpn
Efficient Burst Raw Denoising with Variance Stabilization and Multi-frequency Denoising Network
With the growing popularity of smartphones, capturing high-quality images is
of vital importance to smartphones. The cameras of smartphones have small
apertures and small sensor cells, which lead to the noisy images in low light
environment. Denoising based on a burst of multiple frames generally
outperforms single frame denoising but with the larger compututional cost. In
this paper, we propose an efficient yet effective burst denoising system. We
adopt a three-stage design: noise prior integration, multi-frame alignment and
multi-frame denoising. First, we integrate noise prior by pre-processing raw
signals into a variance-stabilization space, which allows using a small-scale
network to achieve competitive performance. Second, we observe that it is
essential to adopt an explicit alignment for burst denoising, but it is not
necessary to integrate a learning-based method to perform multi-frame
alignment. Instead, we resort to a conventional and efficient alignment method
and combine it with our multi-frame denoising network. At last, we propose a
denoising strategy that processes multiple frames sequentially. Sequential
denoising avoids filtering a large number of frames by decomposing multiple
frames denoising into several efficient sub-network denoising. As for each
sub-network, we propose an efficient multi-frequency denoising network to
remove noise of different frequencies. Our three-stage design is efficient and
shows strong performance on burst denoising. Experiments on synthetic and real
raw datasets demonstrate that our method outperforms state-of-the-art methods,
with less computational cost. Furthermore, the low complexity and high-quality
performance make deployment on smartphones possible.Comment: Accepted for publication in International Journal of Computer Visio
Gated Multi-Resolution Transfer Network for Burst Restoration and Enhancement
Burst image processing is becoming increasingly popular in recent years.
However, it is a challenging task since individual burst images undergo
multiple degradations and often have mutual misalignments resulting in ghosting
and zipper artifacts. Existing burst restoration methods usually do not
consider the mutual correlation and non-local contextual information among
burst frames, which tends to limit these approaches in challenging cases.
Another key challenge lies in the robust up-sampling of burst frames. The
existing up-sampling methods cannot effectively utilize the advantages of
single-stage and progressive up-sampling strategies with conventional and/or
recent up-samplers at the same time. To address these challenges, we propose a
novel Gated Multi-Resolution Transfer Network (GMTNet) to reconstruct a
spatially precise high-quality image from a burst of low-quality raw images.
GMTNet consists of three modules optimized for burst processing tasks:
Multi-scale Burst Feature Alignment (MBFA) for feature denoising and alignment,
Transposed-Attention Feature Merging (TAFM) for multi-frame feature
aggregation, and Resolution Transfer Feature Up-sampler (RTFU) to up-scale
merged features and construct a high-quality output image. Detailed
experimental analysis on five datasets validates our approach and sets a
state-of-the-art for burst super-resolution, burst denoising, and low-light
burst enhancement.Comment: Accepted at CVPR 202
- …