On the Relation between Color Image Denoising and Classification

Large amount of image denoising literature focuses on single channel images and often experimentally validates the proposed methods on tens of images at most. In this paper, we investigate the interaction between denoising and classification on large scale dataset. Inspired by classification models, we propose a novel deep learning architecture for color (multichannel) image denoising and report on thousands of images from ImageNet dataset as well as commonly used imagery. We study the importance of (sufficient) training data, how semantic class information can be traded for improved denoising results. As a result, our method greatly improves PSNR performance by 0.34 - 0.51 dB on average over state-of-the art methods on large scale dataset. We conclude that it is beneficial to incorporate in classification models. On the other hand, we also study how noise affect classification performance. In the end, we come to a number of interesting conclusions, some being counter-intuitive

Adaptive Quantile Sparse Image (AQuaSI) Prior for Inverse Imaging Problems

Inverse problems play a central role for many classical computer vision and image processing tasks. Many inverse problems are ill-posed, and hence require a prior to regularize the solution space. However, many of the existing priors, like total variation, are based on ad-hoc assumptions that have difficulties to represent the actual distribution of natural images. Thus, a key challenge in research on image processing is to find better suited priors to represent natural images. In this work, we propose the Adaptive Quantile Sparse Image (AQuaSI) prior. It is based on a quantile filter, can be used as a joint filter on guidance data, and be readily plugged into a wide range of numerical optimization algorithms. We demonstrate the efficacy of the proposed prior in joint RGB/depth upsampling, on RGB/NIR image restoration, and in a comparison with related regularization by denoising approaches

Fast and Accurate Poisson Denoising with Optimized Nonlinear Diffusion

The degradation of the acquired signal by Poisson noise is a common problem for various imaging applications, such as medical imaging, night vision and microscopy. Up to now, many state-of-the-art Poisson denoising techniques mainly concentrate on achieving utmost performance, with little consideration for the computation efficiency. Therefore, in this study we aim to propose an efficient Poisson denoising model with both high computational efficiency and recovery quality. To this end, we exploit the newly-developed trainable nonlinear reaction diffusion model which has proven an extremely fast image restoration approach with performance surpassing recent state-of-the-arts. We retrain the model parameters, including the linear filters and influence functions by taking into account the Poisson noise statistics, and end up with an optimized nonlinear diffusion model specialized for Poisson denoising. The trained model provides strongly competitive results against state-of-the-art approaches, meanwhile bearing the properties of simple structure and high efficiency. Furthermore, our proposed model comes along with an additional advantage, that the diffusion process is well-suited for parallel computation on GPUs. For images of size $512 \times 512$, our GPU implementation takes less than 0.1 seconds to produce state-of-the-art Poisson denoising performance.Comment: 11 pages, 12 figures, technical repor

Phase asymmetry guided adaptive fractional-order total variation and diffusion for feature-preserving ultrasound despeckling

It is essential for ultrasound despeckling to remove speckle noise while simultaneously preserving edge features for accurate diagnosis and analysis in many applications. To preserve real edges such as ramp edges and low contrast edges, we first detect edges using a phase-based measure called phase asymmetry (PAS), which can distinguish small differences in transition border regions and varies from $0$ to $1$, taking $0$ in ideal smooth regions and taking $1$ at ideal step edges. We further propose three strategies to properly preserve edges. First, in observing that fractional-order anisotropic diffusion (FAD) filter has good performance in smooth regions while the fractional-order TV (FTV) filter performs better at edges, we leverage the PAS metric to keep a balance between FAD filter and FTV filter for achieving the best performance of preserving ramp edges. Second, considering that the FAD filter fails to protect low contrast edges by solely integrating gradient information into the diffusion coefficient, we integrate the PAS metric into the diffusion coefficient to properly preserve low contrast edges. Finally, different from fixed fractional order diffusion filters neglecting the differences between smooth regions and transition border regions, an adaptive fractional order is implemented based on the PAS metric to enhance edges. The experimental results show that our method outperforms other state-of-the-art ultrasound despeckling filters in both speckle reduction and feature preservation

DeepCorrect: Correcting DNN models against Image Distortions

In recent years, the widespread use of deep neural networks (DNNs) has facilitated great improvements in performance for computer vision tasks like image classification and object recognition. In most realistic computer vision applications, an input image undergoes some form of image distortion such as blur and additive noise during image acquisition or transmission. Deep networks trained on pristine images perform poorly when tested on such distortions. In this paper, we evaluate the effect of image distortions like Gaussian blur and additive noise on the activations of pre-trained convolutional filters. We propose a metric to identify the most noise susceptible convolutional filters and rank them in order of the highest gain in classification accuracy upon correction. In our proposed approach called DeepCorrect, we apply small stacks of convolutional layers with residual connections, at the output of these ranked filters and train them to correct the worst distortion affected filter activations, whilst leaving the rest of the pre-trained filter outputs in the network unchanged. Performance results show that applying DeepCorrect models for common vision tasks like image classification (ImageNet), object recognition (Caltech-101, Caltech-256) and scene classification (SUN-397), significantly improves the robustness of DNNs against distorted images and outperforms other alternative approaches..Comment: Accepted to IEEE Transactions on Image Processing, April 2019. For associated code, see https://github.com/tsborkar/DeepCorrec

A New Similarity Measure for Non-Local Means Filtering of MRI Images

The acquisition of MRI images offers a trade-off in terms of acquisition time, spatial/temporal resolution and signal-to-noise ratio (SNR). Thus, for instance, increasing the time efficiency of MRI often comes at the expense of reduced SNR. This, in turn, necessitates the use of post-processing tools for noise rejection, which makes image de-noising an indispensable component of computer assistance diagnosis. In the field of MRI, a multitude of image de-noising methods have been proposed hitherto. In this paper, the application of a particular class of de-noising algorithms - known as non-local mean (NLM) filters - is investigated. Such filters have been recently applied for MRI data enhancement and they have been shown to provide more accurate results as compared to many alternative de-noising algorithms. Unfortunately, virtually all existing methods for NLM filtering have been derived under the assumption of additive white Gaussian (AWG) noise contamination. Since this assumption is known to fail at low values of SNR, an alternative formulation of NLM filtering is required, which would take into consideration the correct Rician statistics of MRI noise. Accordingly, the contribution of the present paper is two-fold. First, it points out some principal disadvantages of the earlier methods of NLM filtering of MRI images and suggests means to rectify them. Second, the paper introduces a new similarity measure for NLM filtering of MRI Images, which is derived under bona fide statistical assumptions and results in more accurate reconstruction of MR scans as compared to alternative NLM approaches. Finally, the utility and viability of the proposed method is demonstrated through a series of numerical experiments using both in silico and in vivo MRI data

Deep Gaussian Conditional Random Field Network: A Model-based Deep Network for Discriminative Denoising

We propose a novel deep network architecture for image\\ denoising based on a Gaussian Conditional Random Field (GCRF) model. In contrast to the existing discriminative denoising methods that train a separate model for each noise level, the proposed deep network explicitly models the input noise variance and hence is capable of handling a range of noise levels. Our deep network, which we refer to as deep GCRF network, consists of two sub-networks: (i) a parameter generation network that generates the pairwise potential parameters based on the noisy input image, and (ii) an inference network whose layers perform the computations involved in an iterative GCRF inference procedure.\ We train the entire deep GCRF network (both parameter generation and inference networks) discriminatively in an end-to-end fashion by maximizing the peak signal-to-noise ratio measure. Experiments on Berkeley segmentation and PASCALVOC datasets show that the proposed deep GCRF network outperforms state-of-the-art image denoising approaches for several noise levels.Comment: 10 pages, 5 figure

Deep Iterative Reconstruction for Phase Retrieval

Classical phase retrieval problem is the recovery of a constrained image from the magnitude of its Fourier transform. Although there are several well-known phase retrieval algorithms including the hybrid input-output (HIO) method, the reconstruction performance is generally sensitive to initialization and measurement noise. Recently, deep neural networks (DNNs) have been shown to provide state-of-the-art performance in solving several inverse problems such as denoising, deconvolution, and superresolution. In this work, we develop a phase retrieval algorithm that utilizes two DNNs together with the model-based HIO method. First, a DNN is trained to remove the HIO artifacts and is used iteratively with the HIO method to improve the reconstructions. After this iterative phase, a second DNN is trained to remove the remaining artifacts. Numerical results demonstrate the effectiveness of ourapproach, which has little additional computational cost compared to the HIO method. Our approach not only achieves state-of-the-art reconstruction performance but also is more robust to different initialization and noise levels.Comment: 14 pages, 8 figures, published in Applied Optics (Vol. 58, Issue 20, pp. 5422-5431 (2019)

Chaining Identity Mapping Modules for Image Denoising

We propose to learn a fully-convolutional network model that consists of a Chain of Identity Mapping Modules (CIMM) for image denoising. The CIMM structure possesses two distinctive features that are important for the noise removal task. Firstly, each residual unit employs identity mappings as the skip connections and receives pre-activated input in order to preserve the gradient magnitude propagated in both the forward and backward directions. Secondly, by utilizing dilated kernels for the convolution layers in the residual branch, in other words within an identity mapping module, each neuron in the last convolution layer can observe the full receptive field of the first layer. After being trained on the BSD400 dataset, the proposed network produces remarkably higher numerical accuracy and better visual image quality than the state-of-the-art when being evaluated on conventional benchmark images and the BSD68 dataset

Compression, Restoration, Re-sampling, Compressive Sensing: Fast Transforms in Digital Imaging

Transform image processing methods are methods that work in domains of image transforms, such as Discrete Fourier, Discrete Cosine, Wavelet and alike. They are the basic tool in image compression, in image restoration, in image re-sampling and geometrical transformations and can be traced back to early 1970-ths. The paper presents a review of these methods with emphasis on their comparison and relationships, from the very first steps of transform image compression methods to adaptive and local adaptive transform domain filters for image restoration, to methods of precise image re-sampling and image reconstruction from sparse samples and up to "compressive sensing" approach that has gained popularity in last few years. The review has a tutorial character and purpose.Comment: 41 pages, 16 figure