17 research outputs found
Leveraging Deep Stein's Unbiased Risk Estimator for Unsupervised X-ray Denoising
Among the plethora of techniques devised to curb the prevalence of noise in
medical images, deep learning based approaches have shown the most promise.
However, one critical limitation of these deep learning based denoisers is the
requirement of high-quality noiseless ground truth images that are difficult to
obtain in many medical imaging applications such as X-rays. To circumvent this
issue, we leverage recently proposed approach of [7] that incorporates Stein's
Unbiased Risk Estimator (SURE) to train a deep convolutional neural network
without requiring denoised ground truth X-ray data. Our experimental results
demonstrate the effectiveness of SURE based approach for denoising X-ray
images.Comment: Machine Learning for Health (ML4H) Workshop at NeurIPS 2018
arXiv:1811.0721
Unsupervised Learning with Stein's Unbiased Risk Estimator
Learning from unlabeled and noisy data is one of the grand challenges of
machine learning. As such, it has seen a flurry of research with new ideas
proposed continuously. In this work, we revisit a classical idea: Stein's
Unbiased Risk Estimator (SURE). We show that, in the context of image recovery,
SURE and its generalizations can be used to train convolutional neural networks
(CNNs) for a range of image denoising and recovery problems without any ground
truth data.
Specifically, our goal is to reconstruct an image from a noisy linear
transformation (measurement) of the image. We consider two scenarios: one where
no additional data is available and one where we have measurements of other
images that are drawn from the same noisy distribution as , but have no
access to the clean images. Such is the case, for instance, in the context of
medical imaging, microscopy, and astronomy, where noise-less ground truth data
is rarely available.
We show that in this situation, SURE can be used to estimate the
mean-squared-error loss associated with an estimate of . Using this estimate
of the loss, we train networks to perform denoising and compressed sensing
recovery. In addition, we also use the SURE framework to partially explain and
improve upon an intriguing results presented by Ulyanov et al. in "Deep Image
Prior": that a network initialized with random weights and fit to a single
noisy image can effectively denoise that image.
Public implementations of the networks and methods described in this paper
can be found at https://github.com/ricedsp/D-AMP_Toolbox
WhiteNNer-Blind Image Denoising via Noise Whiteness Priors
The accuracy of medical imaging-based diagnostics is directly impacted by the
quality of the collected images. A passive approach to improve image quality is
one that lags behind improvements in imaging hardware, awaiting better sensor
technology of acquisition devices. An alternative, active strategy is to
utilize prior knowledge of the imaging system to directly post-process and
improve the acquired images. Traditionally, priors about the image properties
are taken into account to restrict the solution space. However, few techniques
exploit the prior about the noise properties. In this paper, we propose a
neural network-based model for disentangling the signal and noise components of
an input noisy image, without the need for any ground truth training data. We
design a unified loss function that encodes priors about signal as well as
noise estimate in the form of regularization terms. Specifically, by using
total variation and piecewise constancy priors along with noise whiteness
priors such as auto-correlation and stationary losses, our network learns to
decouple an input noisy image into the underlying signal and noise components.
We compare our proposed method to Noise2Noise and Noise2Self, as well as
non-local mean and BM3D, on three public confocal laser endomicroscopy
datasets. Experimental results demonstrate the superiority of our network
compared to state-of-the-art in terms of PSNR and SSIM.Comment: 9 page
Ground Truth Free Denoising by Optimal Transport
We present a learned unsupervised denoising method for arbitrary types of
data, which we explore on images and one-dimensional signals. The training is
solely based on samples of noisy data and examples of noise, which --
critically -- do not need to come in pairs. We only need the assumption that
the noise is independent and additive (although we describe how this can be
extended). The method rests on a Wasserstein Generative Adversarial Network
setting, which utilizes two critics and one generator
Boosting CNN beyond Label in Inverse Problems
Convolutional neural networks (CNN) have been extensively used for inverse
problems. However, their prediction error for unseen test data is difficult to
estimate a priori since the neural networks are trained using only selected
data and their architecture are largely considered a blackbox. This poses a
fundamental challenge to neural networks for unsupervised learning or
improvement beyond the label. In this paper, we show that the recent
unsupervised learning methods such as Noise2Noise, Stein's unbiased risk
estimator (SURE)-based denoiser, and Noise2Void are closely related to each
other in their formulation of an unbiased estimator of the prediction error,
but each of them are associated with its own limitations. Based on these
observations, we provide a novel boosting estimator for the prediction error.
In particular, by employing combinatorial convolutional frame representation of
encoder-decoder CNN and synergistically combining it with the batch
normalization, we provide a close form formulation for the unbiased estimator
of the prediction error that can be minimized for neural network training
beyond the label. Experimental results show that the resulting algorithm, what
we call Noise2Boosting, provides consistent improvement in various inverse
problems under both supervised and unsupervised learning setting
Self-Supervised Poisson-Gaussian Denoising
We extend the blindspot model for self-supervised denoising to handle
Poisson-Gaussian noise and introduce an improved training scheme that avoids
hyperparameters and adapts the denoiser to the test data. Self-supervised
models for denoising learn to denoise from only noisy data and do not require
corresponding clean images, which are difficult or impossible to acquire in
some application areas of interest such as low-light microscopy. We introduce a
new training strategy to handle Poisson-Gaussian noise which is the standard
noise model for microscope images. Our new strategy eliminates hyperparameters
from the loss function, which is important in a self-supervised regime where no
ground truth data is available to guide hyperparameter tuning. We show how our
denoiser can be adapted to the test data to improve performance. Our
evaluations on microscope image denoising benchmarks validate our approach.Comment: to appear in IEEE WACV 202
Degrees of Freedom Analysis of Unrolled Neural Networks
Unrolled neural networks emerged recently as an effective model for learning
inverse maps appearing in image restoration tasks. However, their
generalization risk (i.e., test mean-squared-error) and its link to network
design and train sample size remains mysterious. Leveraging the Stein's
Unbiased Risk Estimator (SURE), this paper analyzes the generalization risk
with its bias and variance components for recurrent unrolled networks. We
particularly investigate the degrees-of-freedom (DOF) component of SURE, trace
of the end-to-end network Jacobian, to quantify the prediction variance. We
prove that DOF is well-approximated by the weighted \textit{path sparsity} of
the network under incoherence conditions on the trained weights. Empirically,
we examine the SURE components as a function of train sample size for both
recurrent and non-recurrent (with many more parameters) unrolled networks. Our
key observations indicate that: 1) DOF increases with train sample size and
converges to the generalization risk for both recurrent and non-recurrent
schemes; 2) recurrent network converges significantly faster (with less train
samples) compared with non-recurrent scheme, hence recurrence serves as a
regularization for low sample size regimes
Unsupervised MRI Reconstruction with Generative Adversarial Networks
Deep learning-based image reconstruction methods have achieved promising
results across multiple MRI applications. However, most approaches require
large-scale fully-sampled ground truth data for supervised training. Acquiring
fully-sampled data is often either difficult or impossible, particularly for
dynamic contrast enhancement (DCE), 3D cardiac cine, and 4D flow. We present a
deep learning framework for MRI reconstruction without any fully-sampled data
using generative adversarial networks. We test the proposed method in two
scenarios: retrospectively undersampled fast spin echo knee exams and
prospectively undersampled abdominal DCE. The method recovers more anatomical
structure compared to conventional methods
Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images
In the last few years, image denoising has benefited a lot from the fast
development of neural networks. However, the requirement of large amounts of
noisy-clean image pairs for supervision limits the wide use of these models.
Although there have been a few attempts in training an image denoising model
with only single noisy images, existing self-supervised denoising approaches
suffer from inefficient network training, loss of useful information, or
dependence on noise modeling. In this paper, we present a very simple yet
effective method named Neighbor2Neighbor to train an effective image denoising
model with only noisy images. Firstly, a random neighbor sub-sampler is
proposed for the generation of training image pairs. In detail, input and
target used to train a network are images sub-sampled from the same noisy
image, satisfying the requirement that paired pixels of paired images are
neighbors and have very similar appearance with each other. Secondly, a
denoising network is trained on sub-sampled training pairs generated in the
first stage, with a proposed regularizer as additional loss for better
performance. The proposed Neighbor2Neighbor framework is able to enjoy the
progress of state-of-the-art supervised denoising networks in network
architecture design. Moreover, it avoids heavy dependence on the assumption of
the noise distribution. We explain our approach from a theoretical perspective
and further validate it through extensive experiments, including synthetic
experiments with different noise distributions in sRGB space and real-world
experiments on a denoising benchmark dataset in raw-RGB space.Comment: CVPR202
Unpaired Learning of Deep Image Denoising
We investigate the task of learning blind image denoising networks from an
unpaired set of clean and noisy images. Such problem setting generally is
practical and valuable considering that it is feasible to collect unpaired
noisy and clean images in most real-world applications. And we further assume
that the noise can be signal dependent but is spatially uncorrelated. In order
to facilitate unpaired learning of denoising network, this paper presents a
two-stage scheme by incorporating self-supervised learning and knowledge
distillation. For self-supervised learning, we suggest a dilated blind-spot
network (D-BSN) to learn denoising solely from real noisy images. Due to the
spatial independence of noise, we adopt a network by stacking 1x1 convolution
layers to estimate the noise level map for each image. Both the D-BSN and
image-specific noise model (CNN\_est) can be jointly trained via maximizing the
constrained log-likelihood. Given the output of D-BSN and estimated noise level
map, improved denoising performance can be further obtained based on the Bayes'
rule. As for knowledge distillation, we first apply the learned noise models to
clean images to synthesize a paired set of training images, and use the real
noisy images and the corresponding denoising results in the first stage to form
another paired set. Then, the ultimate denoising model can be distilled by
training an existing denoising network using these two paired sets. Experiments
show that our unpaired learning method performs favorably on both synthetic
noisy images and real-world noisy photographs in terms of quantitative and
qualitative evaluation.Comment: 20 pages, 6 figures, ECC