Rayleigh-Ritz majorization error bounds of the mixed type

The absolute change in the Rayleigh quotient (RQ) for a Hermitian matrix with respect to vectors is bounded in terms of the norms of the residual vectors and the angle between vectors in [\doi{10.1137/120884468}]. We substitute multidimensional subspaces for the vectors and derive new bounds of absolute changes of eigenvalues of the matrix RQ in terms of singular values of residual matrices and principal angles between subspaces, using majorization. We show how our results relate to bounds for eigenvalues after discarding off-diagonal blocks or additive perturbations.Comment: 20 pages, 1 figure. Accepted to SIAM Journal on Matrix Analysis and Application

Bounds on changes in Ritz values for a perturbed invariant subspace of a Hermitian matrix

The Rayleigh-Ritz method is widely used for eigenvalue approximation. Given a matrix $X$ with columns that form an orthonormal basis for a subspace \X, and a Hermitian matrix $A$, the eigenvalues of $X^HAX$ are called Ritz values of $A$ with respect to \X. If the subspace \X is $A$-invariant then the Ritz values are some of the eigenvalues of $A$. If the $A$-invariant subspace \X is perturbed to give rise to another subspace \Y, then the vector of absolute values of changes in Ritz values of $A$ represents the absolute eigenvalue approximation error using \Y. We bound the error in terms of principal angles between \X and \Y. We capitalize on ideas from a recent paper [DOI: 10.1137/060649070] by A. Knyazev and M. Argentati, where the vector of absolute values of differences between Ritz values for subspaces \X and \Y was weakly (sub-)majorized by a constant times the sine of the vector of principal angles between \X and \Y, the constant being the spread of the spectrum of $A$. In that result no assumption was made on either subspace being $A$-invariant. It was conjectured there that if one of the trial subspaces is $A$-invariant then an analogous weak majorization bound should only involve terms of the order of sine squared. Here we confirm this conjecture. Specifically we prove that the absolute eigenvalue error is weakly majorized by a constant times the sine squared of the vector of principal angles between the subspaces \X and \Y, where the constant is proportional to the spread of the spectrum of $A$. For many practical cases we show that the proportionality factor is simply one, and that this bound is sharp. For the general case we can only prove the result with a slightly larger constant, which we believe is artificial.Comment: 12 pages. Accepted to SIAM Journal on Matrix Analysis and Applications (SIMAX

An Accelerated Conjugate Gradient Algorithm to Compute Low-Lying Eigenvalues --- a Study for the Dirac Operator in SU(2) Lattice QCD

The low-lying eigenvalues of a (sparse) hermitian matrix can be computed with controlled numerical errors by a conjugate gradient (CG) method. This CG algorithm is accelerated by alternating it with exact diagonalisations in the subspace spanned by the numerically computed eigenvectors. We study this combined algorithm in case of the Dirac operator with (dynamical) Wilson fermions in four-dimensional \SUtwo gauge fields. The algorithm is numerically very stable and can be parallelized in an efficient way. On lattices of sizes $4^4-16^4$ an acceleration of the pure CG method by a factor of~$4-8$ is found.Comment: 25 pages, uuencoded tar-compressed .ps-fil

On Temple--Kato like inequalities and applications

We give both lower and upper estimates for eigenvalues of unbounded positive definite operators in an arbitrary Hilbert space. We show scaling robust relative eigenvalue estimates for these operators in analogy to such estimates of current interest in Numerical Linear Algebra. Only simple matrix theoretic tools like Schur complements have been used. As prototypes for the strength of our method we discuss a singularly perturbed Schroedinger operator and study convergence estimates for finite element approximations. The estimates can be viewed as a natural quadratic form version of the celebrated Temple--Kato inequality.Comment: submitted to SIAM Journal on Numerical Analysis (a major revision of the paper

Harmonic and Refined Harmonic Shift-Invert Residual Arnoldi and Jacobi--Davidson Methods for Interior Eigenvalue Problems

This paper concerns the harmonic shift-invert residual Arnoldi (HSIRA) and Jacobi--Davidson (HJD) methods as well as their refined variants RHSIRA and RHJD for the interior eigenvalue problem. Each method needs to solve an inner linear system to expand the subspace successively. When the linear systems are solved only approximately, we are led to the inexact methods. We prove that the inexact HSIRA, RHSIRA, HJD and RHJD methods mimic their exact counterparts well when the inner linear systems are solved with only low or modest accuracy. We show that (i) the exact HSIRA and HJD expand subspaces better than the exact SIRA and JD and (ii) the exact RHSIRA and RHJD expand subspaces better than the exact HSIRA and HJD. Based on the theory, we design stopping criteria for inner solves. To be practical, we present restarted HSIRA, HJD, RHSIRA and RHJD algorithms. Numerical results demonstrate that these algorithms are much more efficient than the restarted standard SIRA and JD algorithms and furthermore the refined harmonic algorithms outperform the harmonic ones very substantially.Comment: 15 pages, 4 figure

Angles Between Infinite Dimensional Subspaces with Applications to the Rayleigh-Ritz and Alternating Projectors Methods

We define angles from-to and between infinite dimensional subspaces of a Hilbert space, inspired by the work of E. J. Hannan, 1961/1962 for general canonical correlations of stochastic processes. The spectral theory of selfadjoint operators is used to investigate the properties of the angles, e.g., to establish connections between the angles corresponding to orthogonal complements. The classical gaps and angles of Dixmier and Friedrichs are characterized in terms of the angles. We introduce principal invariant subspaces and prove that they are connected by an isometry that appears in the polar decomposition of the product of corresponding orthogonal projectors. Point angles are defined by analogy with the point operator spectrum. We bound the Hausdorff distance between the sets of the squared cosines of the angles corresponding to the original subspaces and their perturbations. We show that the squared cosines of the angles from one subspace to another can be interpreted as Ritz values in the Rayleigh-Ritz method, where the former subspace serves as a trial subspace and the orthogonal projector of the latter subspace serves as an operator in the Rayleigh-Ritz method. The Hausdorff distance between the Ritz values, corresponding to different trial subspaces, is shown to be bounded by a constant times the gap between the trial subspaces. We prove a similar eigenvalue perturbation bound that involves the gap squared. Finally, we consider the classical alternating projectors method and propose its ultimate acceleration, using the conjugate gradient approach. The corresponding convergence rate estimate is obtained in terms of the angles. We illustrate a possible acceleration for the domain decomposition method with a small overlap for the 1D diffusion equation.Comment: 22 pages. Accepted to Journal of Functional Analysi