366 research outputs found
Toward Convolutional Blind Denoising of Real Photographs
While deep convolutional neural networks (CNNs) have achieved impressive
success in image denoising with additive white Gaussian noise (AWGN), their
performance remains limited on real-world noisy photographs. The main reason is
that their learned models are easy to overfit on the simplified AWGN model
which deviates severely from the complicated real-world noise model. In order
to improve the generalization ability of deep CNN denoisers, we suggest
training a convolutional blind denoising network (CBDNet) with more realistic
noise model and real-world noisy-clean image pairs. On the one hand, both
signal-dependent noise and in-camera signal processing pipeline is considered
to synthesize realistic noisy images. On the other hand, real-world noisy
photographs and their nearly noise-free counterparts are also included to train
our CBDNet. To further provide an interactive strategy to rectify denoising
result conveniently, a noise estimation subnetwork with asymmetric learning to
suppress under-estimation of noise level is embedded into CBDNet. Extensive
experimental results on three datasets of real-world noisy photographs clearly
demonstrate the superior performance of CBDNet over state-of-the-arts in terms
of quantitative metrics and visual quality. The code has been made available at
https://github.com/GuoShi28/CBDNet
Real Image Denoising with Feature Attention
Deep convolutional neural networks perform better on images containing
spatially invariant noise (synthetic noise); however, their performance is
limited on real-noisy photographs and requires multiple stage network modeling.
To advance the practicability of denoising algorithms, this paper proposes a
novel single-stage blind real image denoising network (RIDNet) by employing a
modular architecture. We use a residual on the residual structure to ease the
flow of low-frequency information and apply feature attention to exploit the
channel dependencies. Furthermore, the evaluation in terms of quantitative
metrics and visual quality on three synthetic and four real noisy datasets
against 19 state-of-the-art algorithms demonstrate the superiority of our
RIDNet.Comment: Accepted in ICCV (Oral), 201
Model-blind Video Denoising Via Frame-to-frame Training
Modeling the processing chain that has produced a video is a difficult
reverse engineering task, even when the camera is available. This makes model
based video processing a still more complex task. In this paper we propose a
fully blind video denoising method, with two versions off-line and on-line.
This is achieved by fine-tuning a pre-trained AWGN denoising network to the
video with a novel frame-to-frame training strategy. Our denoiser can be used
without knowledge of the origin of the video or burst and the post processing
steps applied from the camera sensor. The on-line process only requires a
couple of frames before achieving visually-pleasing results for a wide range of
perturbations. It nonetheless reaches state of the art performance for standard
Gaussian noise, and can be used off-line with still better performance.Comment: CVPR 201
GRDN:Grouped Residual Dense Network for Real Image Denoising and GAN-based Real-world Noise Modeling
Recent research on image denoising has progressed with the development of
deep learning architectures, especially convolutional neural networks. However,
real-world image denoising is still very challenging because it is not possible
to obtain ideal pairs of ground-truth images and real-world noisy images. Owing
to the recent release of benchmark datasets, the interest of the image
denoising community is now moving toward the real-world denoising problem. In
this paper, we propose a grouped residual dense network (GRDN), which is an
extended and generalized architecture of the state-of-the-art residual dense
network (RDN). The core part of RDN is defined as grouped residual dense block
(GRDB) and used as a building module of GRDN. We experimentally show that the
image denoising performance can be significantly improved by cascading GRDBs.
In addition to the network architecture design, we also develop a new
generative adversarial network-based real-world noise modeling method. We
demonstrate the superiority of the proposed methods by achieving the highest
score in terms of both the peak signal-to-noise ratio and the structural
similarity in the NTIRE2019 Real Image Denoising Challenge - Track 2:sRGB.Comment: To appear in CVPR 2019 workshop. The winners of the NTIRE2019
Challenge on Image Denoising Challenge: Track 2 sRG
Generating Training Data for Denoising Real RGB Images via Camera Pipeline Simulation
Image reconstruction techniques such as denoising often need to be applied to
the RGB output of cameras and cellphones. Unfortunately, the commonly used
additive white noise (AWGN) models do not accurately reproduce the noise and
the degradation encountered on these inputs. This is particularly important for
learning-based techniques, because the mismatch between training and real world
data will hurt their generalization. This paper aims to accurately simulate the
degradation and noise transformation performed by camera pipelines. This allows
us to generate realistic degradation in RGB images that can be used to train
machine learning models. We use our simulation to study the importance of noise
modeling for learning-based denoising. Our study shows that a realistic noise
model is required for learning to denoise real JPEG images. A neural network
trained on realistic noise outperforms the one trained with AWGN by 3 dB. An
ablation study of our pipeline shows that simulating denoising and demosaicking
is important to this improvement and that realistic demosaicking algorithms,
which have been rarely considered, is needed. We believe this simulation will
also be useful for other image reconstruction tasks, and we will distribute our
code publicly
Path-Restore: Learning Network Path Selection for Image Restoration
Very deep Convolutional Neural Networks (CNNs) have greatly improved the
performance on various image restoration tasks. However, this comes at a price
of increasing computational burden, hence limiting their practical usages. We
observe that some corrupted image regions are inherently easier to restore than
others since the distortion and content vary within an image. To leverage this,
we propose Path-Restore, a multi-path CNN with a pathfinder that can
dynamically select an appropriate route for each image region. We train the
pathfinder using reinforcement learning with a difficulty-regulated reward.
This reward is related to the performance, complexity and "the difficulty of
restoring a region". A policy mask is further investigated to jointly process
all the image regions. We conduct experiments on denoising and mixed
restoration tasks. The results show that our method achieves comparable or
superior performance to existing approaches with less computational cost. In
particular, Path-Restore is effective for real-world denoising, where the noise
distribution varies across different regions on a single image. Compared to the
state-of-the-art RIDNet, our method achieves comparable performance and runs
2.7x faster on the realistic Darmstadt Noise Dataset.Comment: IEEE TPAMI 2021. Project page:
https://www.mmlab-ntu.com/project/pathrestore
Segmentation-Aware Image Denoising without Knowing True Segmentation
Several recent works discussed application-driven image restoration neural
networks, which are capable of not only removing noise in images but also
preserving their semantic-aware details, making them suitable for various
high-level computer vision tasks as the pre-processing step. However, such
approaches require extra annotations for their high-level vision tasks, in
order to train the joint pipeline using hybrid losses. The availability of
those annotations is yet often limited to a few image sets, potentially
restricting the general applicability of these methods to denoising more unseen
and unannotated images. Motivated by that, we propose a segmentation-aware
image denoising model dubbed U-SAID, based on a novel unsupervised approach
with a pixel-wise uncertainty loss. U-SAID does not need any ground-truth
segmentation map, and thus can be applied to any image dataset. It generates
denoised images with comparable or even better quality, and the denoised
results show stronger robustness for subsequent semantic segmentation tasks,
when compared to either its supervised counterpart or classical
"application-agnostic" denoisers. Moreover, we demonstrate the superior
generalizability of U-SAID in three-folds, by plugging its "universal" denoiser
without fine-tuning: (1) denoising unseen types of images; (2) denoising as
pre-processing for segmenting unseen noisy images; and (3) denoising for unseen
high-level tasks. Extensive experiments demonstrate the effectiveness,
robustness and generalizability of the proposed U-SAID over various popular
image sets
ViDeNN: Deep Blind Video Denoising
We propose ViDeNN: a CNN for Video Denoising without prior knowledge on the
noise distribution (blind denoising). The CNN architecture uses a combination
of spatial and temporal filtering, learning to spatially denoise the frames
first and at the same time how to combine their temporal information, handling
objects motion, brightness changes, low-light conditions and temporal
inconsistencies. We demonstrate the importance of the data used for CNNs
training, creating for this purpose a specific dataset for low-light
conditions. We test ViDeNN on common benchmarks and on self-collected data,
achieving good results comparable with the state-of-the-art.Comment: Submission of NTIRE: New Trends in Image Restoration and Enhancement
workshop and challenges at CVPR 201
Joint Demosaicking and Denoising by Fine-Tuning of Bursts of Raw Images
Demosaicking and denoising are the first steps of any camera image processing
pipeline and are key for obtaining high quality RGB images. A promising current
research trend aims at solving these two problems jointly using convolutional
neural networks. Due to the unavailability of ground truth data these networks
cannot be currently trained using real RAW images. Instead, they resort to
simulated data. In this paper we present a method to learn demosaicking
directly from mosaicked images, without requiring ground truth RGB data. We
apply this to learn joint demosaicking and denoising only from RAW images, thus
enabling the use of real data. In addition we show that for this application
fine-tuning a network to a specific burst improves the quality of restoration
for both demosaicking and denoising.Comment: ICCV 201
CFSNet: Toward a Controllable Feature Space for Image Restoration
Deep learning methods have witnessed the great progress in image restoration
with specific metrics (e.g., PSNR, SSIM). However, the perceptual quality of
the restored image is relatively subjective, and it is necessary for users to
control the reconstruction result according to personal preferences or image
characteristics, which cannot be done using existing deterministic networks.
This motivates us to exquisitely design a unified interactive framework for
general image restoration tasks. Under this framework, users can control
continuous transition of different objectives, e.g., the perception-distortion
trade-off of image super-resolution, the trade-off between noise reduction and
detail preservation. We achieve this goal by controlling the latent features of
the designed network. To be specific, our proposed framework, named
Controllable Feature Space Network (CFSNet), is entangled by two branches based
on different objectives. Our framework can adaptively learn the coupling
coefficients of different layers and channels, which provides finer control of
the restored image quality. Experiments on several typical image restoration
tasks fully validate the effective benefits of the proposed method. Code is
available at https://github.com/qibao77/CFSNet.Comment: Accepted by ICCV 201
- …