Superoptimal Preconditioned Conjugate Gradient Iteration for Image Deblurring

We study the superoptimal Frobenius operators in the two-level circulant algebra. We consider two specific viewpoints: ( 1) the regularizing properties in imaging and ( 2) the computational effort in connection with the preconditioned conjugate gradient method. Some numerical experiments illustrating the effectiveness of the proposed technique are given and discussed

Spectral analysis for preconditioning of multi-dimensional Riesz fractional diffusion equations

In this paper, we analyze the spectra of the preconditioned matrices arising from discretized multi-dimensional Riesz spatial fractional diffusion equations. The finite difference method is employed to approximate the multi-dimensional Riesz fractional derivatives, which will generate symmetric positive definite ill-conditioned multi-level Toeplitz matrices. The preconditioned conjugate gradient method with a preconditioner based on the sine transform is employed to solve the resulting linear system. Theoretically, we prove that the spectra of the preconditioned matrices are uniformly bounded in the open interval (1/2,3/2) and thus the preconditioned conjugate gradient method converges linearly. The proposed method can be extended to multi-level Toeplitz matrices generated by functions with zeros of fractional order. Our theoretical results fill in a vacancy in the literature. Numerical examples are presented to demonstrate our new theoretical results in the literature and show the convergence performance of the proposed preconditioner that is better than other existing preconditioners

Scalable iterative methods for sampling from massive Gaussian random vectors

Sampling from Gaussian Markov random fields (GMRFs), that is multivariate Gaussian ran- dom vectors that are parameterised by the inverse of their covariance matrix, is a fundamental problem in computational statistics. In this paper, we show how we can exploit arbitrarily accu- rate approximations to a GMRF to speed up Krylov subspace sampling methods. We also show that these methods can be used when computing the normalising constant of a large multivariate Gaussian distribution, which is needed for both any likelihood-based inference method. The method we derive is also applicable to other structured Gaussian random vectors and, in particu- lar, we show that when the precision matrix is a perturbation of a (block) circulant matrix, it is still possible to derive O(n log n) sampling schemes.Comment: 17 Pages, 4 Figure

Preconditioned fast solvers for large linear systems with specific sparse and/or Toeplitz-like structures and applications

In this thesis, the design of the preconditioners we propose starts from applications instead of treating the problem in a completely general way. The reason is that not all types of linear systems can be addressed with the same tools. In this sense, the techniques for designing efficient iterative solvers depends mostly on properties inherited from the continuous problem, that has originated the discretized sequence of matrices. Classical examples are locality, isotropy in the PDE context, whose discrete counterparts are sparsity and matrices constant along the diagonals, respectively. Therefore, it is often important to take into account the properties of the originating continuous model for obtaining better performances and for providing an accurate convergence analysis. We consider linear systems that arise in the solution of both linear and nonlinear partial differential equation of both integer and fractional type. For the latter case, an introduction to both the theory and the numerical treatment is given. All the algorithms and the strategies presented in this thesis are developed having in mind their parallel implementation. In particular, we consider the processor-co-processor framework, in which the main part of the computation is performed on a Graphics Processing Unit (GPU) accelerator. In Part I we introduce our proposal for sparse approximate inverse preconditioners for either the solution of time-dependent Partial Differential Equations (PDEs), Chapter 3, and Fractional Differential Equations (FDEs), containing both classical and fractional terms, Chapter 5. More precisely, we propose a new technique for updating preconditioners for dealing with sequences of linear systems for PDEs and FDEs, that can be used also to compute matrix functions of large matrices via quadrature formula in Chapter 4 and for optimal control of FDEs in Chapter 6. At last, in Part II, we consider structured preconditioners for quasi-Toeplitz systems. The focus is towards the numerical treatment of discretized convection-diffusion equations in Chapter 7 and on the solution of FDEs with linear multistep formula in boundary value form in Chapter 8

Preconditioned fast solvers for large linear systems with specific sparse and/or Toeplitz-like structures and applications

In this thesis, the design of the preconditioners we propose starts from applications instead of treating the problem in a completely general way. The reason is that not all types of linear systems can be addressed with the same tools. In this sense, the techniques for designing efficient iterative solvers depends mostly on properties inherited from the continuous problem, that has originated the discretized sequence of matrices. Classical examples are locality, isotropy in the PDE context, whose discrete counterparts are sparsity and matrices constant along the diagonals, respectively. Therefore, it is often important to take into account the properties of the originating continuous model for obtaining better performances and for providing an accurate convergence analysis. We consider linear systems that arise in the solution of both linear and nonlinear partial differential equation of both integer and fractional type. For the latter case, an introduction to both the theory and the numerical treatment is given. All the algorithms and the strategies presented in this thesis are developed having in mind their parallel implementation. In particular, we consider the processor-co-processor framework, in which the main part of the computation is performed on a Graphics Processing Unit (GPU) accelerator. In Part I we introduce our proposal for sparse approximate inverse preconditioners for either the solution of time-dependent Partial Differential Equations (PDEs), Chapter 3, and Fractional Differential Equations (FDEs), containing both classical and fractional terms, Chapter 5. More precisely, we propose a new technique for updating preconditioners for dealing with sequences of linear systems for PDEs and FDEs, that can be used also to compute matrix functions of large matrices via quadrature formula in Chapter 4 and for optimal control of FDEs in Chapter 6. At last, in Part II, we consider structured preconditioners for quasi-Toeplitz systems. The focus is towards the numerical treatment of discretized convection-diffusion equations in Chapter 7 and on the solution of FDEs with linear multistep formula in boundary value form in Chapter 8

FAST SOLUTION METHODS FOR CONVEX QUADRATIC OPTIMIZATION OF FRACTIONAL DIFFERENTIAL EQUATIONS

In this paper, we present numerical methods suitable for solving convex quadratic Fractional Differential Equation (FDE) constrained optimization problems, with box constraints on the state and/or control variables. We develop an Alternating Direction Method of Multipliers (ADMM) framework, which uses preconditioned Krylov subspace solvers for the resulting sub-problems. The latter allows us to tackle a range of Partial Differential Equation (PDE) optimization problems with box constraints, posed on space-time domains, that were previously out of the reach of state-of-the-art preconditioners. In particular, by making use of the powerful Generalized Locally Toeplitz (GLT) sequences theory, we show that any existing GLT structure present in the problem matrices is preserved by ADMM, and we propose some preconditioning methodologies that could be used within the solver, to demonstrate the generality of the approach. Focusing on convex quadratic programs with time-dependent 2-dimensional FDE constraints, we derive multilevel circulant preconditioners, which may be embedded within Krylov subspace methods, for solving the ADMM sub-problems. Discretized versions of FDEs involve large dense linear systems. In order to overcome this difficulty, we design a recursive linear algebra, which is based on the Fast Fourier Transform (FFT). We manage to keep the storage requirements linear, with respect to the grid size $N$, while ensuring an order $N \log N$ computational complexity per iteration of the Krylov solver. We implement the proposed method, and demonstrate its scalability, generality, and efficiency, through a series of experiments over different setups of the FDE optimization problem

A fast implicit difference scheme for solving the generalized time-space fractional diffusion equations with variable coefficients

In this paper, we first propose an unconditionally stable implicit difference scheme for solving generalized time-space fractional diffusion equations (GTSFDEs) with variable coefficients. The numerical scheme utilizes the $L1$-type formula for the generalized Caputo fractional derivative in time discretization and the second-order weighted and shifted Gr\"{u}nwald difference (WSGD) formula in spatial discretization, respectively. Theoretical results and numerical tests are conducted to verify the $(2 - \gamma)$-order and 2-order of temporal and spatial convergence with $\gamma\in(0,1)$ the order of Caputo fractional derivative, respectively. The fast sum-of-exponential approximation of the generalized Caputo fractional derivative and Toeplitz-like coefficient matrices are also developed to accelerate the proposed implicit difference scheme. Numerical experiments show the effectiveness of the proposed numerical scheme and its good potential for large-scale simulation of GTSFDEs.Comment: 23 pages, 10 tables, 1 figure. Make several corrections again and have been submitted to a journal at Sept. 20, 2019. Version 2: Make some necessary corrections and symbols, 13 Jan. 2020. Revised manuscript has been resubmitted to journa

Matrices associated to two conservative discretizations of Riesz fractional operators and related multigrid solvers

In this article, we focus on a two-dimensional conservative steady-state Riesz fractional diffusion problem. As is typical for problems in conservative form, we adopt a finite volume (FV)-based discretization approach. Precisely, we use both classical FVs and the so-called finite volume elements (FVEs). While FVEs have already been applied in the context of fractional diffusion equations, classical FVs have only been applied in first-order discretizations. By exploiting the Toeplitz-like structure of the resulting coefficient matrices, we perform a qualitative study of their spectrum and conditioning through their symbol, leading to the design of a second-order FV discretization. This same information is leveraged to discuss parameter-free symbol-based multigrid methods for both discretizations. Tests on the approximation error and the performances of the considered solvers are given as well

Regularizing preconditioners based on fit techniques in the image reconstruction problem

Regularizing preconditioners for the approximate solution by gradient-type methods of image restoration problems with two-level band Toeplitz structure, are examined. For problems having separable and positive definite matrices, the fit preconditioner, introduced in [6], has been shown to be effective in conjunction with CG. The cost of this preconditioner is of O(n^2) operations per iteration, where n^2 is the pixels number of the image, whereas the cost of the circulant preconditioners commonly used for this type of problems is of O(n^2 log n) operations per iteration. In this paper the extension of the fit preconditioner to more general cases is proposed: namely the nonseparable positive definite case and the symmetric indefinite case are treated. The major difficulty encountered in this extension concerns the factorization phase, where, unlike the separable case, a further approximation is required. Various approximate factorizations are proposed. The preconditioners thus obtained have still a cost of O(n^2) operations per iteration. A large numerical experimentation compares these preconditioners with the circulant Chan preconditioner, showing often better performances at a lower cost