Training and refining deep learning based denoisers without ground truth data

Department of Electrical Engineeringclos

Training deep learning based denoisers without ground truth data

Recently developed deep-learning-based denoisers often outperform state-of-the-art conventional denoisers, such as the BM3D. They are typically trained to minimizethe mean squared error (MSE) between the output image of a deep neural networkand a ground truth image. In deep learning based denoisers, it is important to use high quality noiseless ground truth data for high performance, but it is often challenging or even infeasible to obtain noiseless images in application areas such as hyperspectral remote sensing and medical imaging. In this article, we propose a method based on Stein???s unbiased risk estimator (SURE) for training deep neural network denoisers only based on the use of noisy images. We demonstrate that our SURE-based method, without the use of ground truth data, is able to train deep neural network denoisers to yield performances close to those networks trained with ground truth, and to outperform the state-of-the-art denoiser BM3D. Further improvements were achieved when noisy test images were used for training of denoiser networks using our proposed SURE-based method

Unsupervised Training of Denoisers for Low-Dose CT Reconstruction Without Full-Dose Ground Truth

Recently, deep neural network (DNN) based methods for low-dose CT have been investigated to achieve excellent performance in both image quality and computational speed. However, almost all methods using DNNs for low-dose CT require clean ground truth data with full radiation dose to train the DNNs. In this work, we attempt to train DNNs for low-dose CT reconstructions with reduced tube current by investigating unsupervised training of DNNs for denoising sensor measurements or sinograms without full-dose ground truth images. In other words, our proposed methods allow training of DNNs with only noisy low-dose CT measurements. First, the Poisson Unbiased Risk Estimator (PURE) is investigated to train a DNN for denoising CT measurements, and a method is proposed for reconstructing the CT image using filtered back-projection (FBP) and the DNN trained with PURE. Then, the CT forward model-based Weighted Stein's Unbiased Risk Estimator (WSURE) is proposed to train a DNN for denoising CT sinograms and to subsequently reconstruct the CT image using FBP. Our proposed methods achieve excellent performance in both fast computation and reconstructed image quality, which is more comparable to the results of the DNNs trained with full-dose ground truth data than other state-of-the-art denoising methods such as the BM3D, Deep Image Prior, and Deep Decoder

Extending Stein???s Unbiased Risk Estimator To Train Deep Denoisers with Correlated Pairs of Noisy Images

Recently, Stein???s unbiased risk estimator (SURE) has been applied to unsupervised training of deep neural network Gaussian denoisers that outperformed classical non-deep learning based denoisers and yielded comparable performance to those trained with ground truth. While SURE requires only one noise realization per image for training, it does not take advantage of having multiple noise realizations per image when they are available (e.g., two uncorrelated noise realizations per image for Noise2Noise). Here, we propose an extended SURE (eSURE) to train deep denoisers with correlated pairs of noise realizations per image and applied it to the case with two uncorrelated realizations per image to achieve better performance than SURE based method and comparable results to Noise2Noise. Then, we further investigated the case with imperfect ground truth (i.e., mild noise in ground truth) that may be obtained considering painstaking, time-consuming, and even expensive processes of collecting ground truth images with multiple noisy images. For the case of generating noisy training data by adding synthetic noise to imperfect ground truth to yield correlated pairs of images, our proposed eSURE based training method outperformed conventional SURE based method as well as Noise2Noise. Code is available at https://github.com/Magauiya/Extended_SUR

Unsupervised learning of denoisers with compressive sensing measurements

Recently, deep learning based compressive recovery methods have been proposed and have yielded state-of-the-art performances. Ironically, training deep neural networks for them requires ???clean??? ground truth, but obtaining the best quality images from undersampled data requires well-trained deep networks. To resolve this dilemma, we propose methods that are able to train deep denoisers from undersampled measurements without ground truth. Our methods yielded comparable performances to the methods with ground truth for various image recovery problems

Training Deep Learning Based Image Denoisers From Undersampled Measurements Without Ground Truth and Without Image Prior

Compressive sensing is a method to recover the original image from undersampled measurements. In order to overcome the ill-posedness of this inverse problem, image priors are used such as sparsity, minimal total-variation, or self-similarity of images. Recently, deep learning based compressive image recovery methods have been proposed and have yielded state-of-the-art performances. They used data-driven approaches instead of hand-crafted image priors to regularize ill-posed inverse problems with undersampled data. Ironically, training deep neural networks (DNNs) for them requires ???clean??? ground truth images, but obtaining the best quality images from undersampled data requires well-trained DNNs. To resolve this dilemma, we propose novel methods based on two well-grounded theories: denoiser-approximate message passing (D-AMP) and Stein???s unbiased risk estimator (SURE). Our proposed methods were able to train deep learning based image denoisers from undersampled measurements without ground truth images and without additional image priors, and to recover images with state-of-the-art qualities from undersampled data. We evaluated our methods for various compressive sensing recovery problems with Gaussian random, coded diffraction pattern, and compressive sensing MRI measurement matrices. Our proposed methods yielded state-of-the-art performances for all cases without ground truth images. Our methods also yielded comparable performances to the methods with ground truth data

On Divergence Approximations for Unsupervised Training of Deep Denoisers Based on Stein???s Unbiased Risk Estimator

Recently, there have been several works on unsupervised learning for training deep learning based denoisers without clean images. Approaches based on Stein???s unbiased risk estimator (SURE) have shown promising results for training Gaussian deep denoisers. However, their performance is sensitive to hyper-parameter selection in approximating the divergence term in the SURE expression. In this work, we briefly study the computational efficiency of Monte-Carlo (MC) divergence approximation over recently available exact divergence computation using backpropagation. Then, we investigate the relationship between smoothness of nonlinear activation functions in deep denoisers and robust divergence term approximations. Lastly, we propose a new divergence term that does not contain hyper-parameters. Both unsupervised training methods yield comparable performance to supervised training methods with ground truth for denoising on various datasets. While the former method still requires roughly tuned hyper parameter selection, the latter method removes the necessity of choosing one

NTIRE 2019 Challenge on Real Image Denoising: Methods and Results

This paper reviews the NTIRE 2019 challenge on real image denoising with focus on the proposed methods and their results. The challenge has two tracks for quantitatively evaluating image denoising performance in (1) the Bayer- pattern raw-RGB and (2) the standard RGB (sRGB) color spaces. The tracks had 216 and 220 registered participants, respectively. A total of 15 teams, proposing 17 methods, competed in the final phase of the challenge. The proposed methods by the 15 teams represent the current state-of-the- art performance in image denoising targeting real noisy im- ages