17 research outputs found
Joint Demosaicking and Denoising in the Wild: The Case of Training Under Ground Truth Uncertainty
Image demosaicking and denoising are the two key fundamental steps in digital
camera pipelines, aiming to reconstruct clean color images from noisy luminance
readings. In this paper, we propose and study Wild-JDD, a novel learning
framework for joint demosaicking and denoising in the wild. In contrast to
previous works which generally assume the ground truth of training data is a
perfect reflection of the reality, we consider here the more common imperfect
case of ground truth uncertainty in the wild. We first illustrate its
manifestation as various kinds of artifacts including zipper effect, color
moire and residual noise. Then we formulate a two-stage data degradation
process to capture such ground truth uncertainty, where a conjugate prior
distribution is imposed upon a base distribution. After that, we derive an
evidence lower bound (ELBO) loss to train a neural network that approximates
the parameters of the conjugate prior distribution conditioned on the degraded
input. Finally, to further enhance the performance for out-of-distribution
input, we design a simple but effective fine-tuning strategy by taking the
input as a weakly informative prior. Taking into account ground truth
uncertainty, Wild-JDD enjoys good interpretability during optimization.
Extensive experiments validate that it outperforms state-of-the-art schemes on
joint demosaicking and denoising tasks on both synthetic and realistic raw
datasets.Comment: Accepted by AAAI202
Optimizing Image Compression via Joint Learning with Denoising
High levels of noise usually exist in today's captured images due to the
relatively small sensors equipped in the smartphone cameras, where the noise
brings extra challenges to lossy image compression algorithms. Without the
capacity to tell the difference between image details and noise, general image
compression methods allocate additional bits to explicitly store the undesired
image noise during compression and restore the unpleasant noisy image during
decompression. Based on the observations, we optimize the image compression
algorithm to be noise-aware as joint denoising and compression to resolve the
bits misallocation problem. The key is to transform the original noisy images
to noise-free bits by eliminating the undesired noise during compression, where
the bits are later decompressed as clean images. Specifically, we propose a
novel two-branch, weight-sharing architecture with plug-in feature denoisers to
allow a simple and effective realization of the goal with little computational
cost. Experimental results show that our method gains a significant improvement
over the existing baseline methods on both the synthetic and real-world
datasets. Our source code is available at
https://github.com/felixcheng97/DenoiseCompression.Comment: Accepted to ECCV 202
Lucas-Kanade Reloaded: End-to-End Super-Resolution from Raw Image Bursts
International audienceThis presentation addresses the problem of reconstructing a high-resolution image from multiple lower-resolution snapshots captured from slightly different viewpoints in space and time. Key challenges for solving this super-resolution problem include (i) aligning the input pictures with sub-pixel accuracy, (ii) handling raw (noisy) images for maximal faithfulness to native camera data, and (iii) designing/learning an image prior (regularizer) well suited to the task. We address these three challenges with a hybrid algorithm building on the insight from Wronski et al. that aliasing is an ally in this setting, with parameters that can be learned end to end, while retaining the interpretability of classical approaches to inverse problems. The effectiveness of our approach is demonstrated on synthetic and real image bursts, setting a new state of the art on several benchmarks and delivering excellent qualitative results on real raw bursts captured by smartphones and prosumer cameras
Joint Demosaicing and Denoising with Double Deep Image Priors
Demosaicing and denoising of RAW images are crucial steps in the processing
pipeline of modern digital cameras. As only a third of the color information
required to produce a digital image is captured by the camera sensor, the
process of demosaicing is inherently ill-posed. The presence of noise further
exacerbates this problem. Performing these two steps sequentially may distort
the content of the captured RAW images and accumulate errors from one step to
another. Recent deep neural-network-based approaches have shown the
effectiveness of joint demosaicing and denoising to mitigate such challenges.
However, these methods typically require a large number of training samples and
do not generalize well to different types and intensities of noise. In this
paper, we propose a novel joint demosaicing and denoising method, dubbed
JDD-DoubleDIP, which operates directly on a single RAW image without requiring
any training data. We validate the effectiveness of our method on two popular
datasets -- Kodak and McMaster -- with various noises and noise intensities.
The experimental results show that our method consistently outperforms other
compared methods in terms of PSNR, SSIM, and qualitative visual perception