1,853 research outputs found
Physics-guided Noise Neural Proxy for Low-light Raw Image Denoising
Low-light raw image denoising plays a crucial role in mobile photography, and
learning-based methods have become the mainstream approach. Training the
learning-based methods with synthetic data emerges as an efficient and
practical alternative to paired real data. However, the quality of synthetic
data is inherently limited by the low accuracy of the noise model, which
decreases the performance of low-light raw image denoising. In this paper, we
develop a novel framework for accurate noise modeling that learns a
physics-guided noise neural proxy (PNNP) from dark frames. PNNP integrates
three efficient techniques: physics-guided noise decoupling (PND),
physics-guided proxy model (PPM), and differentiable distribution-oriented loss
(DDL). The PND decouples the dark frame into different components and handles
different levels of noise in a flexible manner, which reduces the complexity of
the noise neural proxy. The PPM incorporates physical priors to effectively
constrain the generated noise, which promotes the accuracy of the noise neural
proxy. The DDL provides explicit and reliable supervision for noise modeling,
which promotes the precision of the noise neural proxy. Extensive experiments
on public low-light raw image denoising datasets and real low-light imaging
scenarios demonstrate the superior performance of our PNNP framework
Towards General Low-Light Raw Noise Synthesis and Modeling
Modeling and synthesizing low-light raw noise is a fundamental problem for
computational photography and image processing applications. Although most
recent works have adopted physics-based models to synthesize noise, the
signal-independent noise in low-light conditions is far more complicated and
varies dramatically across camera sensors, which is beyond the description of
these models. To address this issue, we introduce a new perspective to
synthesize the signal-independent noise by a generative model. Specifically, we
synthesize the signal-dependent and signal-independent noise in a physics- and
learning-based manner, respectively. In this way, our method can be considered
as a general model, that is, it can simultaneously learn different noise
characteristics for different ISO levels and generalize to various sensors.
Subsequently, we present an effective multi-scale discriminator termed Fourier
transformer discriminator (FTD) to distinguish the noise distribution
accurately. Additionally, we collect a new low-light raw denoising (LRD)
dataset for training and benchmarking. Qualitative validation shows that the
noise generated by our proposed noise model can be highly similar to the real
noise in terms of distribution. Furthermore, extensive denoising experiments
demonstrate that our method performs favorably against state-of-the-art methods
on different sensors.Comment: 11 pages, 7 figures. Accepted by ICCV 202
ExposureDiffusion: Learning to Expose for Low-light Image Enhancement
Previous raw image-based low-light image enhancement methods predominantly
relied on feed-forward neural networks to learn deterministic mappings from
low-light to normally-exposed images. However, they failed to capture critical
distribution information, leading to visually undesirable results. This work
addresses the issue by seamlessly integrating a diffusion model with a
physics-based exposure model. Different from a vanilla diffusion model that has
to perform Gaussian denoising, with the injected physics-based exposure model,
our restoration process can directly start from a noisy image instead of pure
noise. As such, our method obtains significantly improved performance and
reduced inference time compared with vanilla diffusion models. To make full use
of the advantages of different intermediate steps, we further propose an
adaptive residual layer that effectively screens out the side-effect in the
iterative refinement when the intermediate results have been already
well-exposed. The proposed framework can work with both real-paired datasets,
SOTA noise models, and different backbone networks. Note that, the proposed
framework is compatible with real-paired datasets, real/synthetic noise models,
and different backbone networks. We evaluate the proposed method on various
public benchmarks, achieving promising results with consistent improvements
using different exposure models and backbones. Besides, the proposed method
achieves better generalization capacity for unseen amplifying ratios and better
performance than a larger feedforward neural model when few parameters are
adopted.Comment: accepted by ICCV202
Efficient Burst Raw Denoising with Variance Stabilization and Multi-frequency Denoising Network
With the growing popularity of smartphones, capturing high-quality images is
of vital importance to smartphones. The cameras of smartphones have small
apertures and small sensor cells, which lead to the noisy images in low light
environment. Denoising based on a burst of multiple frames generally
outperforms single frame denoising but with the larger compututional cost. In
this paper, we propose an efficient yet effective burst denoising system. We
adopt a three-stage design: noise prior integration, multi-frame alignment and
multi-frame denoising. First, we integrate noise prior by pre-processing raw
signals into a variance-stabilization space, which allows using a small-scale
network to achieve competitive performance. Second, we observe that it is
essential to adopt an explicit alignment for burst denoising, but it is not
necessary to integrate a learning-based method to perform multi-frame
alignment. Instead, we resort to a conventional and efficient alignment method
and combine it with our multi-frame denoising network. At last, we propose a
denoising strategy that processes multiple frames sequentially. Sequential
denoising avoids filtering a large number of frames by decomposing multiple
frames denoising into several efficient sub-network denoising. As for each
sub-network, we propose an efficient multi-frequency denoising network to
remove noise of different frequencies. Our three-stage design is efficient and
shows strong performance on burst denoising. Experiments on synthetic and real
raw datasets demonstrate that our method outperforms state-of-the-art methods,
with less computational cost. Furthermore, the low complexity and high-quality
performance make deployment on smartphones possible.Comment: Accepted for publication in International Journal of Computer Visio
- …