Recursive-Based PCG Methods for Toeplitz Systems with Nonnegative Generating Functions

In this paper, we consider the solutions of symmetric positive definite, but ill-conditioned, Toeplitz systems An x = b. Here we propose to solve the system by the recursive-based preconditioned conjugate gradient method. The idea is to use the inverse of Am (the principal submatrix of An with the Gohberg--Semencul formula as a preconditioner for An. The inverse of Am can be generated recursively by using the formula until m is small enough. The construction of the preconditioners requires only the entries of An and does not require the explicit knowledge of the generating function f of An. We show that if f is a nonnegative, bounded, and piecewise continuous even function with a finite number of zeros of even order, the spectra of the preconditioned matrices are uniformly bounded except for a fixed number of outliers. Hence the conjugate gradient method, when applied to solving the preconditioned system, converges very quickly. Numerical results are included to illustrate the effectiveness of our approach.published_or_final_versio

Block diagonal and schur complement preconditioners for block-toeplitz systems with small size blocks

In this paper we consider the solution of Hermitian positive definite block-Toeplitz systems with small size blocks. We propose and study block diagonal and Schur complement preconditioners for such block-Toeplitz matrices. We show that for some block-Toeplitz matrices, the spectra of the preconditioned matrices are uniformly bounded except for a fixed number of outliers where this fixed number depends only on the size of the block. Hence, conjugate gradient type methods, when applied to solving these preconditioned block-Toeplitz systems with small size blocks, converge very fast. Recursive computation of such block diagonal and Schur complement preconditioners is considered by using the nice matrix representation of the inverse of a block-Toeplitz matrix. Applications to block-Toeplitz systems arising from least squares filtering problems and queueing networks are presented. Numerical examples are given to demonstrate the effectiveness of the proposed method. © 2007 Society for Industrial and Applied Mathematics.published_or_final_versio

Symbol Based Convergence Analysis in Block Multigrid Methods with applications for Stokes problems

The main focus of this paper is the study of efficient multigrid methods for large linear system with a particular saddle-point structure. In particular, we propose a symbol based convergence analysis for problems that have a hidden block Toeplitz structure. Then, they can be investigated focusing on the properties of the associated generating function $\mathbf{f}$, which consequently is a matrix-valued function with dimension depending on the block of the problem. As numerical tests we focus on the matrix sequence stemming from the finite element approximation of the Stokes equation. We show the efficiency of the methods studying the hidden $9\times 9$ block structure of the obtained matrix sequence proposing an efficient algebraic multigrid method with convergence rate independent of the matrix size. Moreover, we present several numerical tests comparing the results with different known strategies

V-cycle optimal convergence for certain (multilevel) structured linear systems

In this paper we are interested in the solution by multigrid strategies of multilevel linear systems whose coefficient matrices belong to the circulant, Hartley, or \u3c4 algebras or to the Toeplitz class and are generated by (the Fourier expansion of) a nonnegative multivariate polynomial f. It is well known that these matrices are banded and have eigenvalues equally distributed as f, so they are ill-conditioned whenever f takes the zero value; they can even be singular and need a low-rank correction. We prove the V-cycle multigrid iteration to have a convergence rate independent of the dimension even in presence of ill-conditioning. If the (multilevel) coefficient matrix has partial dimension nr at level r, r = 1, . . . ,d, then the size of the algebraic system is N(n) = \u3a0r=1 d nr, O(N(n)) operations are required by our technique, and therefore the corresponding method is optimal. Some numerical experiments concerning linear systems arising in applications, such as elliptic PDEs with mixed boundary conditions and image restoration problems, are considered and discussed.cussed

Approximation and spectral analysis for large structured linear systems.

In this work we are interested in standard and less standard structured linear systems coming from applications in various _elds of computational mathematics and often modeled by integral and/or di_erential equations. Starting from classical Toeplitz and Circulant structures, we consider some extensions as g-Toeplitz and g-Circulants matrices appearing in several contexts in numerical analysis and applications. Then we consider special matrices arising from collocation methods for di_erential equations: also in this case, under suitable assumptions we observe a Toeplitz structure. More in detail we _rst propose a detailed study of singular values and eigenvalues of g-circulant matrices and then we provide an analysis of distribution of g-Toeplitz sequences. Furthermore, when possible, we consider Krylov space methods with special attention to the minimization of the computational work. When the involved dimensions are large, the Preconditioned Conjugate Gradient (PCG) method is recommended because of the much stronger robustness with respect to the propagation of errors. In that case, crucial issues are the convergence speed of this iterative solver, the use of special techniques (preconditioning, multilevel techniques) for accelerating the convergence, and a careful study of the spectral properties of such matrices. Finally, the use of radial basis functions allow of determining and studying the asymptotic behavior of the spectral radii of collocation matrices approximating elliptic boundary value problems

Approximation and spectral analysis for large structured linear systems.

In this work we are interested in standard and less standard structured linear systems coming from applications in various _elds of computational mathematics and often modeled by integral and/or di_erential equations. Starting from classical Toeplitz and Circulant structures, we consider some extensions as g-Toeplitz and g-Circulants matrices appearing in several contexts in numerical analysis and applications. Then we consider special matrices arising from collocation methods for di_erential equations: also in this case, under suitable assumptions we observe a Toeplitz structure. More in detail we _rst propose a detailed study of singular values and eigenvalues of g-circulant matrices and then we provide an analysis of distribution of g-Toeplitz sequences. Furthermore, when possible, we consider Krylov space methods with special attention to the minimization of the computational work. When the involved dimensions are large, the Preconditioned Conjugate Gradient (PCG) method is recommended because of the much stronger robustness with respect to the propagation of errors. In that case, crucial issues are the convergence speed of this iterative solver, the use of special techniques (preconditioning, multilevel techniques) for accelerating the convergence, and a careful study of the spectral properties of such matrices. Finally, the use of radial basis functions allow of determining and studying the asymptotic behavior of the spectral radii of collocation matrices approximating elliptic boundary value problems

On the regularizing power of multigrid-type algorithms

We consider the deblurring problem of noisy and blurred images in the case of known space invariant point spread functions with four choices of boundary conditions. We combine an algebraic multigrid previously defined ad hoc for structured matrices related to space invariant operators (Toeplitz, circulants, trigonometric matrix algebras, etc.) and the classical geometric multigrid studied in the partial differential equations context. The resulting technique is parameterized in order to have more degrees of freedom: a simple choice of the parameters allows us to devise a quite powerful regularizing method. It defines an iterative regularizing method where the smoother itself has to be an iterative regularizing method (e.g., conjugate gradient, Landweber, conjugate gradient for normal equations, etc.). More precisely, with respect to the smoother, the regularization properties are improved and the total complexity is lower. Furthermore, in several cases, when it is directly applied to the system $A{\bf f}={\bf g}$, the quality of the restored image is comparable with that of all the best known techniques for the normal equations $A^TA{\bf f}=A^T{\bf g}$, but the related convergence is substantially faster. Finally, the associated curves of the relative errors versus the iteration numbers are ``flatter'' with respect to the smoother (the estimation of the stop iteration is less crucial). Therefore, we can choose multigrid procedures which are much more efficient than classical techniques without losing accuracy in the restored image (as often occurs when using preconditioning). Several numerical experiments show the effectiveness of our proposals

Structured matrices coming from PDE approximation theory: spectral analysis, spectral symbol and design of fast iterative solvers.

Partial Differential Equations (PDE) are extensively used in Applied Sciences to model real-world problems. The solution u of a PDE is normally not available in closed form, and so it is necessary to approximate it by means of some numerical method. Despite the differences among the various methods, the principle on which all of them are based is essentially the same: they first discretize the PDE by introducing a mesh, related to some discretization parameter n, and then they compute the corresponding numerical solution u_n, which will converge to u when n tends to infinity, i.e., when the mesh is progressively refined. Now, if both the PDE and the numerical method are linear, the computation of u_n reduces to solving a certain linear system A_n * u_n = f_n whose size d_n tends to infinity with n. In addition, the sequence of discretization matrices A_n often enjoys an asymptotic spectral distribution described by a certain matrix-valued function f, which takes values in the space of Hermitian matrices of a certain size s. This means that, for large n, the eigenvalues of A_n are approximately given by a uniform sampling of the eigenvalue functions lambda_i(f), i=1,...,s, over the domain of f. In this situation, f is called the (spectral) symbol of the sequence of matrices A_n. The identification and the study of the symbol are two interesting issues in themselves, because they provide an accurate information about the asymptotic global behavior of the eigenvalues of A_n. In particular, the number s coincides with the number of "branches" that compose the asymptotic spectrum of A_n. However, the knowledge of the symbol f and of its properties is not only interesting in itself, but can also be used for practical purposes. In particular, it can be used to design effective preconditioned Krylov and multigrid solvers for the linear systems associated with A_n. The reason is clear: the convergence properties of preconditioned Krylov and multigrid methods strongly depend on the spectral features of the matrix to which they are applied. Hence, the spectral information provided by the symbol can be conveniently used for designing fast solvers of this kind. The purpose of this thesis is to present some specific examples, of interest in practical applications, in which the above philosophical discussion comes to life. As our model PDE, we consider classical second-order elliptic differential equations. Concerning the numerical methods that we employ for their solution, we make three choices: the classical Qp Lagrangian Finite Element Method (FEM), the Galerkin B-spline Isogeometric Analysis (IgA) and the B-spline IgA Collocation Method. We first identify and study the symbol f that characterizes the asymptotic spectrum of the discretization matrices A_n arising from these approximation techniques. Then, by exploiting the properties of the symbol, we design fast iterative solvers for the matrices A_n associated with the two numerical methods based on IgA (the Galerkin B-spline IgA and the B-spline IgA Collocation Method)

Structured matrices coming from PDE approximation theory: spectral analysis, spectral symbol and design of fast iterative solvers.

Partial Differential Equations (PDE) are extensively used in Applied Sciences to model real-world problems. The solution u of a PDE is normally not available in closed form, and so it is necessary to approximate it by means of some numerical method. Despite the differences among the various methods, the principle on which all of them are based is essentially the same: they first discretize the PDE by introducing a mesh, related to some discretization parameter n, and then they compute the corresponding numerical solution u_n, which will converge to u when n tends to infinity, i.e., when the mesh is progressively refined. Now, if both the PDE and the numerical method are linear, the computation of u_n reduces to solving a certain linear system A_n * u_n = f_n whose size d_n tends to infinity with n. In addition, the sequence of discretization matrices A_n often enjoys an asymptotic spectral distribution described by a certain matrix-valued function f, which takes values in the space of Hermitian matrices of a certain size s. This means that, for large n, the eigenvalues of A_n are approximately given by a uniform sampling of the eigenvalue functions lambda_i(f), i=1,...,s, over the domain of f. In this situation, f is called the (spectral) symbol of the sequence of matrices A_n. The identification and the study of the symbol are two interesting issues in themselves, because they provide an accurate information about the asymptotic global behavior of the eigenvalues of A_n. In particular, the number s coincides with the number of "branches" that compose the asymptotic spectrum of A_n. However, the knowledge of the symbol f and of its properties is not only interesting in itself, but can also be used for practical purposes. In particular, it can be used to design effective preconditioned Krylov and multigrid solvers for the linear systems associated with A_n. The reason is clear: the convergence properties of preconditioned Krylov and multigrid methods strongly depend on the spectral features of the matrix to which they are applied. Hence, the spectral information provided by the symbol can be conveniently used for designing fast solvers of this kind. The purpose of this thesis is to present some specific examples, of interest in practical applications, in which the above philosophical discussion comes to life. As our model PDE, we consider classical second-order elliptic differential equations. Concerning the numerical methods that we employ for their solution, we make three choices: the classical Qp Lagrangian Finite Element Method (FEM), the Galerkin B-spline Isogeometric Analysis (IgA) and the B-spline IgA Collocation Method. We first identify and study the symbol f that characterizes the asymptotic spectrum of the discretization matrices A_n arising from these approximation techniques. Then, by exploiting the properties of the symbol, we design fast iterative solvers for the matrices A_n associated with the two numerical methods based on IgA (the Galerkin B-spline IgA and the B-spline IgA Collocation Method)