Rethinking the Pipeline of Demosaicing, Denoising and Super-Resolution

Incomplete color sampling, noise degradation, and limited resolution are the three key problems that are unavoidable in modern camera systems. Demosaicing (DM), denoising (DN), and super-resolution (SR) are core components in a digital image processing pipeline to overcome the three problems above, respectively. Although each of these problems has been studied actively, the mixture problem of DM, DN, and SR, which is a higher practical value, lacks enough attention. Such a mixture problem is usually solved by a sequential solution (applying each method independently in a fixed order: DM $\to$ DN $\to$ SR), or is simply tackled by an end-to-end network without enough analysis into interactions among tasks, resulting in an undesired performance drop in the final image quality. In this paper, we rethink the mixture problem from a holistic perspective and propose a new image processing pipeline: DN $\to$ SR $\to$ DM. Extensive experiments show that simply modifying the usual sequential solution by leveraging our proposed pipeline could enhance the image quality by a large margin. We further adopt the proposed pipeline into an end-to-end network, and present Trinity Enhancement Network (TENet). Quantitative and qualitative experiments demonstrate the superiority of our TENet to the state-of-the-art. Besides, we notice the literature lacks a full color sampled dataset. To this end, we contribute a new high-quality full color sampled real-world dataset, namely PixelShift200. Our experiments show the benefit of the proposed PixelShift200 dataset for raw image processing.Comment: Code is available at: https://github.com/guochengqian/TENe

Wavelet/shearlet hybridized neural networks for biomedical image restoration

Recently, new programming paradigms have emerged that combine parallelism and numerical computations with algorithmic differentiation. This approach allows for the hybridization of neural network techniques for inverse imaging problems with more traditional methods such as wavelet-based sparsity modelling techniques. The benefits are twofold: on the one hand traditional methods with well-known properties can be integrated in neural networks, either as separate layers or tightly integrated in the network, on the other hand, parameters in traditional methods can be trained end-to-end from datasets in a neural network "fashion" (e.g., using Adagrad or Adam optimizers). In this paper, we explore these hybrid neural networks in the context of shearlet-based regularization for the purpose of biomedical image restoration. Due to the reduced number of parameters, this approach seems a promising strategy especially when dealing with small training data sets