Image restoration: Wavelet frame shrinkage, nonlinear evolution PDEs, and beyond

In the past few decades, mathematics based approaches have been widely adopted in various image restoration problems; the partial differential equation (PDE) based approach (e.g., the total variation model [L. Rudin, S. Osher, and E. Fatemi, Phys. D, 60 (1992), pp. 259-268] and its generalizations, nonlinear diffusions [P. Perona and J. Malik, IEEE Trans. Pattern Anal. Mach. Intel., 12 (1990), pp. 629-639; F. Catte et al., SIAM J. Numer. Anal., 29 (1992), pp. 182-193], etc.) and wavelet frame based approach are some successful examples. These approaches were developed through different paths and generally provided understanding from different angles of the same problem. As shown in numerical simulations, implementations of the wavelet frame based approach and the PDE based approach quite often end up solving a similar numerical problem with similar numerical behaviors, even though different approaches have advantages in different applications. Since wavelet frame based and PDE based approaches have all been modeling the same types of problems with success, it is natural to ask whether the wavelet frame based approach is fundamentally connected with the PDE based approach when we trace them all the way back to their roots. A fundamental connection of a wavelet frame based approach with a total variation model and its generalizations was established in [J. Cai, B. Dong, S. Osher, and Z. Shen, J. Amer. Math. Soc., 25 (2012), pp. 1033-1089]. This connection gives the wavelet frame based approach a geometric explanation and, at the same time, it equips a PDE based approach with a time frequency analysis. Cai et al. showed that a special type of wavelet frame model using generic wavelet frame systems can be regarded as an approximation of a generic variational model (with the total variation model as a special case) in the discrete setting. A systematic convergence analysis, as the resolution of the image goes to infinity, which is the key step in linking the two approaches, is also given in Cai et al. Motivated by Cai et al. and [Q. Jiang, Appl. Numer. Math., 62 (2012), pp. 51-66], this paper establishes a fundamental connection between the wavelet frame based approach and nonlinear evolution PDEs, provides interpretations and analytical studies of such connections, and proposes new algorithms for image restoration based on the new understandings. Together with the results in [J. Cai et al., J. Amer. Math. Soc., 25 (2012), pp. 1033-1089], we now have a better picture of how the wavelet frame based approach can be used to interpret the general PDE based approach (e.g., the variational models or nonlinear evolution PDEs) and can be used as a new and useful tool in numerical analysis to discretize and solve various variational and PDE models. To be more precise, we shall establish the following: (1) The connections between wavelet frame shrinkage and nonlinear evolution PDEs provide new and inspiring interpretations of both approaches that enable us to derive new PDE models and (better) wavelet frame shrinkage algorithms for image restoration. (2) A generic nonlinear evolution PDE (of parabolic or hyperbolic type) can be approximated by wavelet frame shrinkage with properly chosen wavelet frame systems and carefully designed shrinkage functions. (3) The main idea of this work is beyond the scope of image restoration. Our analysis and discussions indicate that wavelet frame shrinkage is a new way of solving PDEs in general, which will provide a new insight that will enrich the existing theory and applications of numerical PDEs, as well as those of wavelet frames

Correspondence between Multiwavelet Shrinkage/Multiple Wavelet Frame Shrinkage and Nonlinear Diffusion

There are numerous methodologies for signal and image denoising. Wavelet, wavelet frame shrinkage, and nonlinear diffusion are effective ways for signal and image denoising. Also, multiwavelet transforms and multiple wavelet frame transforms have been used for signal and image denoising. Multiwavelets have important property that they can possess the orthogonality, short support, good performance at the boundaries, and symmetry simultaneously. The advantage of multiwavelet transform for signal and image denoising was illustrated by Bui et al. in 1998. They showed that the evaluation of thresholding on a multiwavelet basis has produced good results. Further, Strela et al. have showed that the decimated multiwavelet denoising provides superior results than decimated conventional (scalar) wavelet denoising. Mrazek, Weickert, and Steidl in 2003 examined the association between one-dimensional nonlinear diffusion and undecimated Haar wavelet shrinkage. They proved that nonlinear diffusion could be presented by using wavelet shrinkage. High-order nonlinear diffusion in terms of one-dimensional frame shrinkage and two-dimensional frame shrinkage were presented in 2012 by Jiang, and in 2013 by Dong, Jiang, and Shen, respectively. They obtained that the correspondence between both approaches leads to a different form of diffusion equation that mixes benefits from both approaches. The objective of this dissertation is to study the correspondence between one-dimensional multiwavelet shrinkage and high-order nonlinear diffusion, and to study high-order nonlinear diffusion in terms of one-dimensional multiple frame shrinkage also well. Further, this dissertation formulates nonlinear diffusion in terms of 2D multiwavelet shrinkage and 2D multiple wavelet frame shrinkage. From the experiment results, it can be inferred that nonlinear diffusion in terms of multiwavelet shrinkage/multiple frame shrinkage gives better results than a scalar case. On the whole, this dissertation expands nonlinear diffusion in terms of wavelet shrinkage and nonlinear diffusion in terms of frame shrinkage from the scalar wavelets and frames to the multiwavelets and multiple frames

Fast and Provable Algorithms for Spectrally Sparse Signal Reconstruction via Low-Rank Hankel Matrix Completion

A spectrally sparse signal of order $r$ is a mixture of $r$ damped or undamped complex sinusoids. This paper investigates the problem of reconstructing spectrally sparse signals from a random subset of $n$ regular time domain samples, which can be reformulated as a low rank Hankel matrix completion problem. We introduce an iterative hard thresholding (IHT) algorithm and a fast iterative hard thresholding (FIHT) algorithm for efficient reconstruction of spectrally sparse signals via low rank Hankel matrix completion. Theoretical recovery guarantees have been established for FIHT, showing that $O(r^2\log^2(n))$ number of samples are sufficient for exact recovery with high probability. Empirical performance comparisons establish significant computational advantages for IHT and FIHT. In particular, numerical simulations on $3$D arrays demonstrate the capability of FIHT on handling large and high-dimensional real data

CT Image Reconstruction by Spatial-Radon Domain Data-Driven Tight Frame Regularization

This paper proposes a spatial-Radon domain CT image reconstruction model based on data-driven tight frames (SRD-DDTF). The proposed SRD-DDTF model combines the idea of joint image and Radon domain inpainting model of \cite{Dong2013X} and that of the data-driven tight frames for image denoising \cite{cai2014data}. It is different from existing models in that both CT image and its corresponding high quality projection image are reconstructed simultaneously using sparsity priors by tight frames that are adaptively learned from the data to provide optimal sparse approximations. An alternative minimization algorithm is designed to solve the proposed model which is nonsmooth and nonconvex. Convergence analysis of the algorithm is provided. Numerical experiments showed that the SRD-DDTF model is superior to the model by \cite{Dong2013X} especially in recovering some subtle structures in the images

Image Restoration: A General Wavelet Frame Based Model and Its Asymptotic Analysis

Image restoration is one of the most important areas in imaging science. Mathematical tools have been widely used in image restoration, where wavelet frame based approach is one of the successful examples. In this paper, we introduce a generic wavelet frame based image restoration model, called the "general model", which includes most of the existing wavelet frame based models as special cases. Moreover, the general model also includes examples that are new to the literature. Motivated by our earlier studies [1-3], We provide an asymptotic analysis of the general model as image resolution goes to infinity, which establishes a connection between the general model in discrete setting and a new variatonal model in continuum setting. The variational model also includes some of the existing variational models as special cases, such as the total generalized variational model proposed by [4]. In the end, we introduce an algorithm solving the general model and present one numerical simulation as an example