The Secrets of Non-Blind Poisson Deconvolution

Non-blind image deconvolution has been studied for several decades but most of the existing work focuses on blur instead of noise. In photon-limited conditions, however, the excessive amount of shot noise makes traditional deconvolution algorithms fail. In searching for reasons why these methods fail, we present a systematic analysis of the Poisson non-blind deconvolution algorithms reported in the literature, covering both classical and deep learning methods. We compile a list of five "secrets" highlighting the do's and don'ts when designing algorithms. Based on this analysis, we build a proof-of-concept method by combining the five secrets. We find that the new method performs on par with some of the latest methods while outperforming some older ones.Comment: Under submission at Transactions on Computational Imagin

Image Restoration using Plug-and-Play CNN MAP Denoisers

Plug-and-play denoisers can be used to perform generic image restoration tasks independent of the degradation type. These methods build on the fact that the Maximum a Posteriori (MAP) optimization can be solved using smaller sub-problems, including a MAP denoising optimization. We present the first end-to-end approach to MAP estimation for image denoising using deep neural networks. We show that our method is guaranteed to minimize the MAP denoising objective, which is then used in an optimization algorithm for generic image restoration. We provide theoretical analysis of our approach and show the quantitative performance of our method in several experiments. Our experimental results show that the proposed method can achieve 70x faster performance compared to the state-of-the-art, while maintaining the theoretical perspective of MAP.Comment: Code and models available at https://github.com/DawyD/cnn-map-denoiser . Accepted for publication in VISAPP 202

Computational Methods for Matrix/Tensor Factorization and Deep Learning Image Denoising

Feature learning is a technique to automatically extract features from raw data. It is widely used in areas such as computer vision, image processing, data mining and natural language processing. In this thesis, we are interested in the computational aspects of feature learning. We focus on rank matrix and tensor factorization and deep neural network models for image denoising. With respect to matrix and tensor factorization, we first present a technique to speed up alternating least squares (ALS) and gradient descent (GD) − two commonly used strategies for tensor factorization. We introduce an efficient, scalable and distributed algorithm that addresses the data explosion problem. Instead of a computationally challenging sub-step of ALS and GD, we implement the algorithm on parallel machines by using only two sparse matrix-vector products. Not only is the algorithm scalable but it is also on average 4 to 10 times faster than competing algorithms on various data sets. Next, we discuss our results of non-negative matrix factorization for hyperspectral image data in the presence of noise. We introduce a spectral total variation regularization and derive four variants of the alternating direction method of multiplier algorithm. While all four methods belong to the same family of algorithms, some perform better than others. Thus, we compare the algorithms using stimulated Raman spectroscopic image will be demonstrated. For deep neural network models, we focus on its application to image denoising. We first demonstrate how an optimal procedure leveraging deep neural networks and convex optimization can combine a given set of denoisers to produce an overall better result. The proposed framework estimates the mean squared error (MSE) of individual denoised outputs using a deep neural network; optimally combines the denoised outputs via convex optimization; and recovers lost details of the combined images using another deep neural network. The framework consistently improves denoising performance for both deterministic denoisers and neural network denoisers. Next, we apply the deep neural network to solve the image reconstruction issues of the Quanta Image Sensor (QIS), which is a single-photon image sensor that oversamples the light field to generate binary measures

Approximate Message Passing with a Colored Aliasing Model for Variable Density Fourier Sampled Images

The Approximate Message Passing (AMP) algorithm efficiently reconstructs signals which have been sampled with large i.i.d. sub-Gaussian sensing matrices. Central to AMP is its "state evolution", which guarantees that the difference between the current estimate and ground truth (the "aliasing") at every iteration obeys a Gaussian distribution that can be fully characterized by a scalar. However, when Fourier coefficients of a signal with non-uniform spectral density are sampled, such as in Magnetic Resonance Imaging (MRI), the aliasing is intrinsically colored, AMP's scalar state evolution is no longer accurate and the algorithm encounters convergence problems. In response, we propose the Variable Density Approximate Message Passing (VDAMP) algorithm, which uses the wavelet domain to model the colored aliasing. We present empirical evidence that VDAMP obeys a "colored state evolution", where the aliasing obeys a Gaussian distribution that can be fully characterized with one scalar per wavelet subband. A benefit of state evolution is that Stein's Unbiased Risk Estimate (SURE) can be effectively implemented, yielding an algorithm with subband-dependent thresholding that has no free parameters. We empirically evaluate the effectiveness of VDAMP on three variations of Fast Iterative Shrinkage-Thresholding (FISTA) and find that it converges in around 10 times fewer iterations on average than the next-fastest method, and to a comparable mean-squared-error.Comment: 13 pages, 7 figures, 3 tables. arXiv admin note: text overlap with arXiv:1911.0123

Advanced Restoration Techniques for Images and Disparity Maps

With increasing popularity of digital cameras, the field of Computa- tional Photography emerges as one of the most demanding areas of research. In this thesis we study and develop novel priors and op- timization techniques to solve inverse problems, including disparity estimation and image restoration. The disparity map estimation method proposed in this thesis incor- porates multiple frames of a stereo video sequence to ensure temporal coherency. To enforce smoothness, we use spatio-temporal connec- tions between the pixels of the disparity map to constrain our solution. Apart from smoothness, we enforce a consistency constraint for the disparity assignments by using connections between the left and right views. These constraints are then formulated in a graphical model, which we solve using mean-field approximation. We use a filter-based mean-field optimization that perform efficiently by updating the dis- parity variables in parallel. The parallel updates scheme, however, is not guaranteed to converge to a stationary point. To compare and demonstrate the effectiveness of our approach, we developed a new optimization technique that uses sequential updates, which runs ef- ficiently and guarantees convergence. Our empirical results indicate that with proper initialization, we can employ the parallel update scheme and efficiently optimize our disparity maps without loss of quality. Our method ranks amongst the state of the art in common benchmarks, and significantly reduces the temporal flickering artifacts in the disparity maps. In the second part of this thesis, we address several image restora- tion problems such as image deblurring, demosaicing and super- resolution. We propose to use denoising autoencoders to learn an approximation of the true natural image distribution. We parametrize our denoisers using deep neural networks and show that they learn the gradient of the smoothed density of natural images. Based on this analysis, we propose a restoration technique that moves the so- lution towards the local extrema of this distribution by minimizing the difference between the input and output of our denoiser. Weii demonstrate the effectiveness of our approach using a single trained neural network in several restoration tasks such as deblurring and super-resolution. In a more general framework, we define a new Bayes formulation for the restoration problem, which leads to a more efficient and robust estimator. The proposed framework achieves state of the art performance in various restoration tasks such as deblurring and demosaicing, and also for more challenging tasks such as noise- and kernel-blind image deblurring. Keywords. disparity map estimation, stereo matching, mean-field optimization, graphical models, image processing, linear inverse prob- lems, image restoration, image deblurring, image denoising, single image super-resolution, image demosaicing, deep neural networks, denoising autoencoder

Denoising Low-Dose CT Images using Multi-frame techniques

This study examines potential methods of achieving a reduction in X-ray radiation dose of Computer Tomography (CT) using multi-frame low-dose CT images. Even though a single-frame low-dose CT image is not very diagnostically useful due to excessive noise, we have found that by using multi-frame low-dose CT images we can denoise these low-dose CT images quite significantly at lower radiation dose. We have proposed two approaches leveraging these multi-frame low-dose CT denoising techniques. In our first method, we proposed a blind source separation (BSS) based CT image method using a multiframe low-dose image sequence. By using BSS technique, we estimated the independent image component and noise components from the image sequences. The extracted image component then is further donoised using a nonlocal groupwise denoiser named BM3D that used the mean standard deviation of the noise components. We have also proposed an extension of this method using a window splitting technique. In our second method, we leveraged the power of deep learning to introduce a collaborative technique to train multiple Noise2Noise generators simultaneously and learn the image representation from LDCT images. We presented three models using this Collaborative Network (CN) principle employing two generators (CN2G), three generators (CN3G), and hybrid three generators (HCN3G) consisting of BSS denoiser with one of the CN generators. The CN3G model showed better performance than the CN2G model in terms of denoised image quality at the expense of an additional LDCT image. The HCN3G model took the advantages of both these models by managing to train three collaborative generators using only two LDCT images by leveraging our first proposed method using blind source separation (BSS) and block matching 3-D (BM3D) filter. By using these multi-frame techniques, we can reduce the radiation dosage quite significantly without losing significant image details, especially for low-contrast areas. Amongst our all methods, the HCN3G model performs the best in terms of PSNR, SSIM, and material noise characteristics, while CN2G and CN3G perform better in terms of contrast difference. In HCN3G model, we have combined two of our methods in a single technique. In addition, we have introduced Collaborative Network (CN) and collaborative loss terms in the L2 losses calculation in our second method which is a significant contribution of this research study

Blind Image Denoising using Supervised and Unsupervised Learning

Image denoising is an important problem in image processing and computer vision. In real-world applications, denoising is often a pre-processing step (so-called low-level vision task) before image segmentation, object detection, and recognition at higher levels. Traditional image denoising algorithms often make idealistic assumptions with the noise (e.g., additive white Gaussian or Poisson). However, the noise in the real-world images such as high-ISO photos and microscopic fluorescence images are more complex. Accordingly, the performance of those traditional approaches degrades rapidly on real-world data. Such blind image denoising has remained an open problem in the literature. In this project, we report two competing approaches toward blind image denoising: supervised and unsupervised learning. We report the principles, performance, differences, merits, and technical potential of a few blind denoising algorithms. Supervised learning is a regression model like CNN with a large number of pairs of corrupted images and clean images. This feed-forward convolution neural network separates noise from the image. The reason for using CNN is its deep architecture for exploiting image characteristics, possible parallel computation with modern powerful GPU’s and advances in regularization and learning methods to train. The integration of residual learning and batch normalization is effective in speeding up the training and improving the denoising performance. Here we apply basic statistical reasoning to signaling reconstruction to map corrupted observations to clean targets Recently, few deep learning algorithms have been investigated that do not require ground truth training images. Noise2Noise is an unsupervised training method created for various applications including denoising with Gaussian, Poisson noise. In the N2N model, we observe that we can often learn to turn bad images to good images just by looking at bad images. An experimental study is conducted on practical properties of noisy-target training at performance levels close to using the clean target data. Further, Noise2Void(N2V) is a self-supervised method that takes one step further. This is method does not require clean image data nor noisy image data for training. It is directly trained on the current image that is to be denoised where other methods cannot do it. This is useful for datasets where we cannot find either a noisy dataset or a pair of clean images for training i.e., biomedical image data