Spectral behavior of preconditioned non-Hermitian multilevel block Toeplitz matrices with matrix-valued symbol

This note is devoted to preconditioning strategies for non-Hermitian multilevel block Toeplitz linear systems associated with a multivariate Lebesgue integrable matrix-valued symbol. In particular, we consider special preconditioned matrices, where the preconditioner has a band multilevel block Toeplitz structure, and we complement known results on the localization of the spectrum with global distribution results for the eigenvalues of the preconditioned matrices. In this respect, our main result is as follows. Let $I_k:=(-\pi,\pi)^k$, let $\mathcal M_s$ be the linear space of complex $s\times s$ matrices, and let $f,g:I_k\to\mathcal M_s$ be functions whose components $f_{ij},\,g_{ij}:I_k\to\mathbb C,\ i,j=1,\ldots,s,$ belong to $L^\infty$. Consider the matrices $T_n^{-1}(g)T_n(f)$, where $n:=(n_1,\ldots,n_k)$ varies in $\mathbb N^k$ and $T_n(f),T_n(g)$ are the multilevel block Toeplitz matrices of size $n_1\cdots n_ks$ generated by $f,g$. Then $\{T_n^{-1}(g)T_n(f)\}_{n\in\mathbb N^k}\sim_\lambda g^{-1}f$, i.e. the family of matrices $\{T_n^{-1}(g)T_n(f)\}_{n\in\mathbb N^k}$ has a global (asymptotic) spectral distribution described by the function $g^{-1}f$, provided $g$ possesses certain properties (which ensure in particular the invertibility of $T_n^{-1}(g)$ for all $n$) and the following topological conditions are met: the essential range of $g^{-1}f$, defined as the union of the essential ranges of the eigenvalue functions $\lambda_j(g^{-1}f),\ j=1,\ldots,s$, does not disconnect the complex plane and has empty interior. This result generalizes the one obtained by Donatelli, Neytcheva, Serra-Capizzano in a previous work, concerning the non-preconditioned case $g=1$. The last part of this note is devoted to numerical experiments, which confirm the theoretical analysis and suggest the choice of optimal GMRES preconditioning techniques to be used for the considered linear systems.Comment: 18 pages, 26 figure

Superlinear convergence for PCG using band plus algebra preconditioners for Toeplitz systems

AbstractThe paper studies fast and efficient solution algorithms for n×n symmetric ill conditioned Toeplitz systems Tn(f)x=b where the generating function f is known a priori, real valued, nonnegative, and has isolated roots of even order. The preconditioner that we propose is a product of a band Toeplitz matrix and matrices that belong to a certain trigonometric algebra. The basic idea behind the proposed scheme is to combine the advantages of all components of the product that are well known when every component is used as a stand-alone preconditioner. As a result we obtain a flexible preconditioner which can be applied to the system Tn(f)x=b infusing superlinear convergence to the PCG method. The important feature of the proposed technique is that it can be extended to cover the 2D case, i.e. ill-conditioned block Toeplitz matrices with Toeplitz blocks. We perform many numerical experiments, whose results confirm the theoretical analysis and effectiveness of the proposed strategy

A polynomial fit preconditioner for band Toeplitz matrices in image reconstruction

The Preconditioned Conjugate Gradient is often applied in image reconstruction as a regularizing method. When the blurring matrix has Toeplitz structure, the modified circulant preconditioner and the inverse Toeplitz preconditioner have been shown to be effective. We introduce here a preconditioner for symmetric positive definite Toeplitz matrices based on a trigonometric polynomial fit which has the same effectiveness of the previous ones but has a lower cost when applied to band matrices. The case of band block Toeplitz matrices with band Toeplitz blocks (BTTB) corresponding to separable point spread functions is also considered

Spectral features of matrix-sequences, GLT, symbol, and application in preconditioning Krylov methods, image deblurring, and multigrid algorithms.

The final purpose of any scientific discipline can be regarded as the solution of real-world problems. With this aim, a mathematical modeling of the considered phenomenon is often compulsory. Closed-form solutions of the arising functional equations are usually not available and numerical discretization techniques are required. In this setting, the discretization of an infinite-dimensional linear equation via some linear approximation method, leads to a sequence of linear systems of increasing dimension whose coefficient matrices could inherit a structure from the continuous problem. For instance, the numerical approximation by local methods of constant or nonconstant coefficients systems of Partial Differential Equations (PDEs) over multidimensional domains, gives rise to multilevel block Toeplitz or to Generalized Locally Toeplitz (GLT) sequences, respectively. In the context of structured matrices, the convergence properties of iterative methods, like multigrid or preconditioned Krylov techniques, are strictly related to the notion of symbol, a function whose role relies in describing the asymptotical distribution of the spectrum. This thesis can be seen as a byproduct of the combined use of powerful tools like symbol, spectral distribution, and GLT, when dealing with the numerical solution of structured linear systems. We approach such an issue both from a theoretical and practical viewpoint. On the one hand, we enlarge some known spectral distribution tools by proving the eigenvalue distribution of matrix-sequences obtained as combination of some algebraic operations on multilevel block Toeplitz matrices. On the other hand, we take advantage of the obtained results for designing efficient preconditioning techniques. Moreover, we focus on the numerical solution of structured linear systems coming from the following applications: image deblurring, fractional diffusion equations, and coupled PDEs. A spectral analysis of the arising structured sequences allows us either to study the convergence and predict the behavior of preconditioned Krylov and multigrid methods applied to the coefficient matrices, or to design effective preconditioners and multigrid solvers for the associated linear systems

Spectral features of matrix-sequences, GLT, symbol, and application in preconditioning Krylov methods, image deblurring, and multigrid algorithms.

The final purpose of any scientific discipline can be regarded as the solution of real-world problems. With this aim, a mathematical modeling of the considered phenomenon is often compulsory. Closed-form solutions of the arising functional equations are usually not available and numerical discretization techniques are required. In this setting, the discretization of an infinite-dimensional linear equation via some linear approximation method, leads to a sequence of linear systems of increasing dimension whose coefficient matrices could inherit a structure from the continuous problem. For instance, the numerical approximation by local methods of constant or nonconstant coefficients systems of Partial Differential Equations (PDEs) over multidimensional domains, gives rise to multilevel block Toeplitz or to Generalized Locally Toeplitz (GLT) sequences, respectively. In the context of structured matrices, the convergence properties of iterative methods, like multigrid or preconditioned Krylov techniques, are strictly related to the notion of symbol, a function whose role relies in describing the asymptotical distribution of the spectrum. This thesis can be seen as a byproduct of the combined use of powerful tools like symbol, spectral distribution, and GLT, when dealing with the numerical solution of structured linear systems. We approach such an issue both from a theoretical and practical viewpoint. On the one hand, we enlarge some known spectral distribution tools by proving the eigenvalue distribution of matrix-sequences obtained as combination of some algebraic operations on multilevel block Toeplitz matrices. On the other hand, we take advantage of the obtained results for designing efficient preconditioning techniques. Moreover, we focus on the numerical solution of structured linear systems coming from the following applications: image deblurring, fractional diffusion equations, and coupled PDEs. A spectral analysis of the arising structured sequences allows us either to study the convergence and predict the behavior of preconditioned Krylov and multigrid methods applied to the coefficient matrices, or to design effective preconditioners and multigrid solvers for the associated linear systems

Some fast algorithms in signal and image processing.

Kwok-po Ng.Thesis (Ph.D.)--Chinese University of Hong Kong, 1995.Includes bibliographical references (leaves 138-139).AbstractsSummaryIntroduction --- p.1Summary of the papers A-F --- p.2Paper A --- p.15Paper B --- p.36Paper C --- p.63Paper D --- p.87Paper E --- p.109Paper F --- p.12

Preconditioned fast solvers for large linear systems with specific sparse and/or Toeplitz-like structures and applications

In this thesis, the design of the preconditioners we propose starts from applications instead of treating the problem in a completely general way. The reason is that not all types of linear systems can be addressed with the same tools. In this sense, the techniques for designing efficient iterative solvers depends mostly on properties inherited from the continuous problem, that has originated the discretized sequence of matrices. Classical examples are locality, isotropy in the PDE context, whose discrete counterparts are sparsity and matrices constant along the diagonals, respectively. Therefore, it is often important to take into account the properties of the originating continuous model for obtaining better performances and for providing an accurate convergence analysis. We consider linear systems that arise in the solution of both linear and nonlinear partial differential equation of both integer and fractional type. For the latter case, an introduction to both the theory and the numerical treatment is given. All the algorithms and the strategies presented in this thesis are developed having in mind their parallel implementation. In particular, we consider the processor-co-processor framework, in which the main part of the computation is performed on a Graphics Processing Unit (GPU) accelerator. In Part I we introduce our proposal for sparse approximate inverse preconditioners for either the solution of time-dependent Partial Differential Equations (PDEs), Chapter 3, and Fractional Differential Equations (FDEs), containing both classical and fractional terms, Chapter 5. More precisely, we propose a new technique for updating preconditioners for dealing with sequences of linear systems for PDEs and FDEs, that can be used also to compute matrix functions of large matrices via quadrature formula in Chapter 4 and for optimal control of FDEs in Chapter 6. At last, in Part II, we consider structured preconditioners for quasi-Toeplitz systems. The focus is towards the numerical treatment of discretized convection-diffusion equations in Chapter 7 and on the solution of FDEs with linear multistep formula in boundary value form in Chapter 8

Preconditioned fast solvers for large linear systems with specific sparse and/or Toeplitz-like structures and applications

In this thesis, the design of the preconditioners we propose starts from applications instead of treating the problem in a completely general way. The reason is that not all types of linear systems can be addressed with the same tools. In this sense, the techniques for designing efficient iterative solvers depends mostly on properties inherited from the continuous problem, that has originated the discretized sequence of matrices. Classical examples are locality, isotropy in the PDE context, whose discrete counterparts are sparsity and matrices constant along the diagonals, respectively. Therefore, it is often important to take into account the properties of the originating continuous model for obtaining better performances and for providing an accurate convergence analysis. We consider linear systems that arise in the solution of both linear and nonlinear partial differential equation of both integer and fractional type. For the latter case, an introduction to both the theory and the numerical treatment is given. All the algorithms and the strategies presented in this thesis are developed having in mind their parallel implementation. In particular, we consider the processor-co-processor framework, in which the main part of the computation is performed on a Graphics Processing Unit (GPU) accelerator. In Part I we introduce our proposal for sparse approximate inverse preconditioners for either the solution of time-dependent Partial Differential Equations (PDEs), Chapter 3, and Fractional Differential Equations (FDEs), containing both classical and fractional terms, Chapter 5. More precisely, we propose a new technique for updating preconditioners for dealing with sequences of linear systems for PDEs and FDEs, that can be used also to compute matrix functions of large matrices via quadrature formula in Chapter 4 and for optimal control of FDEs in Chapter 6. At last, in Part II, we consider structured preconditioners for quasi-Toeplitz systems. The focus is towards the numerical treatment of discretized convection-diffusion equations in Chapter 7 and on the solution of FDEs with linear multistep formula in boundary value form in Chapter 8

Equations and systems of nonlinear equations: from high order numerical methods to fast Eigensolvers for structured matrices and applications

A parametrized multi-step Newton method is constructed for widening the region of convergence of classical multi-step Newton method. The second improvement is proposed in the context of multistep Newton methods, by introducing preconditioners to enhance their accuracy, without disturbing their original order of convergence and the related computational cost (in most of the cases). To find roots with unknown multiplicities preconditioners are also effective when they are applied to the Newton method for roots with unknown multiplicities. Frozen Jacobian higher order multistep iterative method for the solution of systems of nonlinear equations are developed and the related results better than those obtained when employing the classical frozen Jacobian multi-step Newton method. To get benefit from the past information that is produced by the iterative method, we constructed iterative methods with memory for solving systems of nonlinear equations. Iterative methods with memory have a greater rate of convergence, if compared with the iterative method without memory. In terms of computational cost, iterative methods with memory are marginally superior comparatively. Numerical methods are also introduced for approximating all the eigenvalues of banded symmetric Toeplitz and preconditioned Toeplitz matrices. Our proposed numerical methods work very efficiently, when the generating symbols of the considered Toeplitz matrices are bijective

Equations and systems of nonlinear equations: from high order numerical methods to fast Eigensolvers for structured matrices and applications

A parametrized multi-step Newton method is constructed for widening the region of convergence of classical multi-step Newton method. The second improvement is proposed in the context of multistep Newton methods, by introducing preconditioners to enhance their accuracy, without disturbing their original order of convergence and the related computational cost (in most of the cases). To find roots with unknown multiplicities preconditioners are also effective when they are applied to the Newton method for roots with unknown multiplicities. Frozen Jacobian higher order multistep iterative method for the solution of systems of nonlinear equations are developed and the related results better than those obtained when employing the classical frozen Jacobian multi-step Newton method. To get benefit from the past information that is produced by the iterative method, we constructed iterative methods with memory for solving systems of nonlinear equations. Iterative methods with memory have a greater rate of convergence, if compared with the iterative method without memory. In terms of computational cost, iterative methods with memory are marginally superior comparatively. Numerical methods are also introduced for approximating all the eigenvalues of banded symmetric Toeplitz and preconditioned Toeplitz matrices. Our proposed numerical methods work very efficiently, when the generating symbols of the considered Toeplitz matrices are bijective