Block diagonal and schur complement preconditioners for block-toeplitz systems with small size blocks

In this paper we consider the solution of Hermitian positive definite block-Toeplitz systems with small size blocks. We propose and study block diagonal and Schur complement preconditioners for such block-Toeplitz matrices. We show that for some block-Toeplitz matrices, the spectra of the preconditioned matrices are uniformly bounded except for a fixed number of outliers where this fixed number depends only on the size of the block. Hence, conjugate gradient type methods, when applied to solving these preconditioned block-Toeplitz systems with small size blocks, converge very fast. Recursive computation of such block diagonal and Schur complement preconditioners is considered by using the nice matrix representation of the inverse of a block-Toeplitz matrix. Applications to block-Toeplitz systems arising from least squares filtering problems and queueing networks are presented. Numerical examples are given to demonstrate the effectiveness of the proposed method. © 2007 Society for Industrial and Applied Mathematics.published_or_final_versio

Preconditioners for Krylov subspace methods: An overview

When simulating a mechanism from science or engineering, or an industrial process, one is frequently required to construct a mathematical model, and then resolve this model numerically. If accurate numerical solutions are necessary or desirable, this can involve solving large-scale systems of equations. One major class of solution methods is that of preconditioned iterative methods, involving preconditioners which are computationally cheap to apply while also capturing information contained in the linear system. In this article, we give a short survey of the field of preconditioning. We introduce a range of preconditioners for partial differential equations, followed by optimization problems, before discussing preconditioners constructed with less standard objectives in mind

Spectral features of matrix-sequences, GLT, symbol, and application in preconditioning Krylov methods, image deblurring, and multigrid algorithms.

The final purpose of any scientific discipline can be regarded as the solution of real-world problems. With this aim, a mathematical modeling of the considered phenomenon is often compulsory. Closed-form solutions of the arising functional equations are usually not available and numerical discretization techniques are required. In this setting, the discretization of an infinite-dimensional linear equation via some linear approximation method, leads to a sequence of linear systems of increasing dimension whose coefficient matrices could inherit a structure from the continuous problem. For instance, the numerical approximation by local methods of constant or nonconstant coefficients systems of Partial Differential Equations (PDEs) over multidimensional domains, gives rise to multilevel block Toeplitz or to Generalized Locally Toeplitz (GLT) sequences, respectively. In the context of structured matrices, the convergence properties of iterative methods, like multigrid or preconditioned Krylov techniques, are strictly related to the notion of symbol, a function whose role relies in describing the asymptotical distribution of the spectrum. This thesis can be seen as a byproduct of the combined use of powerful tools like symbol, spectral distribution, and GLT, when dealing with the numerical solution of structured linear systems. We approach such an issue both from a theoretical and practical viewpoint. On the one hand, we enlarge some known spectral distribution tools by proving the eigenvalue distribution of matrix-sequences obtained as combination of some algebraic operations on multilevel block Toeplitz matrices. On the other hand, we take advantage of the obtained results for designing efficient preconditioning techniques. Moreover, we focus on the numerical solution of structured linear systems coming from the following applications: image deblurring, fractional diffusion equations, and coupled PDEs. A spectral analysis of the arising structured sequences allows us either to study the convergence and predict the behavior of preconditioned Krylov and multigrid methods applied to the coefficient matrices, or to design effective preconditioners and multigrid solvers for the associated linear systems

Spectral features of matrix-sequences, GLT, symbol, and application in preconditioning Krylov methods, image deblurring, and multigrid algorithms.

The final purpose of any scientific discipline can be regarded as the solution of real-world problems. With this aim, a mathematical modeling of the considered phenomenon is often compulsory. Closed-form solutions of the arising functional equations are usually not available and numerical discretization techniques are required. In this setting, the discretization of an infinite-dimensional linear equation via some linear approximation method, leads to a sequence of linear systems of increasing dimension whose coefficient matrices could inherit a structure from the continuous problem. For instance, the numerical approximation by local methods of constant or nonconstant coefficients systems of Partial Differential Equations (PDEs) over multidimensional domains, gives rise to multilevel block Toeplitz or to Generalized Locally Toeplitz (GLT) sequences, respectively. In the context of structured matrices, the convergence properties of iterative methods, like multigrid or preconditioned Krylov techniques, are strictly related to the notion of symbol, a function whose role relies in describing the asymptotical distribution of the spectrum. This thesis can be seen as a byproduct of the combined use of powerful tools like symbol, spectral distribution, and GLT, when dealing with the numerical solution of structured linear systems. We approach such an issue both from a theoretical and practical viewpoint. On the one hand, we enlarge some known spectral distribution tools by proving the eigenvalue distribution of matrix-sequences obtained as combination of some algebraic operations on multilevel block Toeplitz matrices. On the other hand, we take advantage of the obtained results for designing efficient preconditioning techniques. Moreover, we focus on the numerical solution of structured linear systems coming from the following applications: image deblurring, fractional diffusion equations, and coupled PDEs. A spectral analysis of the arising structured sequences allows us either to study the convergence and predict the behavior of preconditioned Krylov and multigrid methods applied to the coefficient matrices, or to design effective preconditioners and multigrid solvers for the associated linear systems

Approximation and spectral analysis for large structured linear systems.

In this work we are interested in standard and less standard structured linear systems coming from applications in various _elds of computational mathematics and often modeled by integral and/or di_erential equations. Starting from classical Toeplitz and Circulant structures, we consider some extensions as g-Toeplitz and g-Circulants matrices appearing in several contexts in numerical analysis and applications. Then we consider special matrices arising from collocation methods for di_erential equations: also in this case, under suitable assumptions we observe a Toeplitz structure. More in detail we _rst propose a detailed study of singular values and eigenvalues of g-circulant matrices and then we provide an analysis of distribution of g-Toeplitz sequences. Furthermore, when possible, we consider Krylov space methods with special attention to the minimization of the computational work. When the involved dimensions are large, the Preconditioned Conjugate Gradient (PCG) method is recommended because of the much stronger robustness with respect to the propagation of errors. In that case, crucial issues are the convergence speed of this iterative solver, the use of special techniques (preconditioning, multilevel techniques) for accelerating the convergence, and a careful study of the spectral properties of such matrices. Finally, the use of radial basis functions allow of determining and studying the asymptotic behavior of the spectral radii of collocation matrices approximating elliptic boundary value problems

Approximation and spectral analysis for large structured linear systems.

In this work we are interested in standard and less standard structured linear systems coming from applications in various _elds of computational mathematics and often modeled by integral and/or di_erential equations. Starting from classical Toeplitz and Circulant structures, we consider some extensions as g-Toeplitz and g-Circulants matrices appearing in several contexts in numerical analysis and applications. Then we consider special matrices arising from collocation methods for di_erential equations: also in this case, under suitable assumptions we observe a Toeplitz structure. More in detail we _rst propose a detailed study of singular values and eigenvalues of g-circulant matrices and then we provide an analysis of distribution of g-Toeplitz sequences. Furthermore, when possible, we consider Krylov space methods with special attention to the minimization of the computational work. When the involved dimensions are large, the Preconditioned Conjugate Gradient (PCG) method is recommended because of the much stronger robustness with respect to the propagation of errors. In that case, crucial issues are the convergence speed of this iterative solver, the use of special techniques (preconditioning, multilevel techniques) for accelerating the convergence, and a careful study of the spectral properties of such matrices. Finally, the use of radial basis functions allow of determining and studying the asymptotic behavior of the spectral radii of collocation matrices approximating elliptic boundary value problems

FAST SOLUTION METHODS FOR CONVEX QUADRATIC OPTIMIZATION OF FRACTIONAL DIFFERENTIAL EQUATIONS

In this paper, we present numerical methods suitable for solving convex quadratic Fractional Differential Equation (FDE) constrained optimization problems, with box constraints on the state and/or control variables. We develop an Alternating Direction Method of Multipliers (ADMM) framework, which uses preconditioned Krylov subspace solvers for the resulting sub-problems. The latter allows us to tackle a range of Partial Differential Equation (PDE) optimization problems with box constraints, posed on space-time domains, that were previously out of the reach of state-of-the-art preconditioners. In particular, by making use of the powerful Generalized Locally Toeplitz (GLT) sequences theory, we show that any existing GLT structure present in the problem matrices is preserved by ADMM, and we propose some preconditioning methodologies that could be used within the solver, to demonstrate the generality of the approach. Focusing on convex quadratic programs with time-dependent 2-dimensional FDE constraints, we derive multilevel circulant preconditioners, which may be embedded within Krylov subspace methods, for solving the ADMM sub-problems. Discretized versions of FDEs involve large dense linear systems. In order to overcome this difficulty, we design a recursive linear algebra, which is based on the Fast Fourier Transform (FFT). We manage to keep the storage requirements linear, with respect to the grid size $N$, while ensuring an order $N \log N$ computational complexity per iteration of the Krylov solver. We implement the proposed method, and demonstrate its scalability, generality, and efficiency, through a series of experiments over different setups of the FDE optimization problem