A geometric convergence theory for the preconditioned steepest descent iteration

Preconditioned gradient iterations for very large eigenvalue problems are efficient solvers with growing popularity. However, only for the simplest preconditioned eigensolver, namely the preconditioned gradient iteration (or preconditioned inverse iteration) with fixed step size, sharp non-asymptotic convergence estimates are known and these estimates require an ideally scaled preconditioner. In this paper a new sharp convergence estimate is derived for the preconditioned steepest descent iteration which combines the preconditioned gradient iteration with the Rayleigh-Ritz procedure for optimal line search convergence acceleration. The new estimate always improves that of the fixed step size iteration. The practical importance of this new estimate is that arbitrarily scaled preconditioners can be used. The Rayleigh-Ritz procedure implicitly computes the optimal scaling.Comment: 17 pages, 6 figure

Computing interior eigenvalues and corresponding eigenvectors of definite matrix pairs

U prvom dijelu ove disertacije predstavljamo nove algoritme koji za dani hermitski matrični par $(A, B)$ ispituju je li on pozitivno definitan, u smislu da postoji realan broj $\lambda_0$ takav da je matrica $A-\lambda_0B$ pozitivno definitna. Skup svih takvih $\lambda_0$ čini otvoreni interval koji zovemo definitan interval, a bilo koji takav $\lambda_0$ zovemo definitan pomak. Najjednostavniji algoritmi ispitivanja koje predlažemo temelje se na ispitivanju glavnih podmatrica reda 1 ili 2. Također razvijamo efikasniji algoritam ispitivanja potprostora uz pretpostavku indefinitnosti matrice B. Taj se algoritam temelji na iterativnom ispitivanju malih gusto popunjenih komprimiranih parova koji nastaju korištenjem test-potprostora malih dimenzija, a predlažemo i ubrzanje samog algoritma. Algoritam ispitivanja potprostora posebno je pogodan za velike rijetko popunjene vrpčaste matrične parove, a može se primijeniti u ispitivanju hiperbolnosti kvadratnog svojstvenog problema. U drugom dijelu ove disertacije za dani pozitivno definitni matrični par $(A, B)$ reda $n$ s indefinitnom matricom $B$ konstruiramo nove algoritme minimizacije traga funkcije $f(X)=X^HAX$ uz uvjet $X^HBX=diag(I_{k_+}, -I_{k_-})$ gdje je $X \in \mathbb{C}^{n \times (k_++k_-)}, 1 \leq k_+ \leq n_+, 1 \leq k_- \leq n_-$ i $(n_+, n_-, n_0)$ inercija matrice $B$. Predlažemo opći indefinitni algoritam, te razvijamo efikasne algoritme prekondicioniranih gradijentnih iteracija koje smo nazvali indefinitna $m$-shema. Stoga metode indefinitne $m$-sheme za dani pozitivno definitni par i jedan ili dva definitna pomaka (koji se mogu dobiti algoritmom ispitivanja potprostora) istovremeno računaju manji broj unutarnjih svojstvenih vrijednosti oko definitnog intervala i pridružene svojstvene vektore. Također, dajemo ideje kako računati manji broj svojstvenih vrijednosti oko bilo kojeg broja unutar rubova spektra, a izvan definitnog intervala, i pridruženih svojstvenih vektora, danog pozitivno definitnog matričnog para koristeći pozitivno definitnu matricu prekondicioniranja. Algoritmi su posebno pogodni za velike rijetko popunjene matrične parove. Nizom numeričkih eksperimenata pokazujemo efikasnost samih algoritama ispitivanja i algoritama računanja unutarnjih svojstvenih vrijednosti i pridruženih svojstvenih vektora. Efikasnost naših metoda uspoređujemo s nekim postojećim metodama.The generalized eigenvalue problem (GEP) for given matrices $A, B \in \mathbb{C}^{n \times n}$ is to find scalars $\lambda$ and nonzero vectors $x \in \mathbb{C}^n$ such that $Ax = \lambda Bx$ (1). The pair $(\lambda, x)$ is called an eigenpair, $\lambda$ is an eigenvalue and $x$ corresponding eigenvector. GEP (1) where A and B are both Hermitian, or real symmetric, occurs in many applications of mathematics. Very important case is when B (and A) is positive definite (appearing, e.g., in the finite element discretization of self-adjoint and elliptic PDE-eigenvalue problem [25]). Another very important case is when B (and A) is indefinite, but the matrix pair (A, B) is definite, meaning, there exist real numbers $\alpha, \beta$ such that the matrix $\alpha A + \beta B$ is positive definite (appearing, e.g., in mechanics [83] and computational quantum chemistry [4]). Many theoretical properties (variational principles, perturbation theory, etc.) and eigenvalue solvers for Hermitian matrix are extended to definite matrix pairs [64, 79, 83]. A Hermitian matrix pair (A, B) is called positive (negative) definite if there exists a real $\lambda_0$ such that $A- \lambda_0 B$ is positive ( negative) definite. The set of all such $\lambda_0$ is an open interval called the definiteness interval [83], and any such $\lambda_0$ will be called definitizing shift. In the first part of this thesis we propose new algorithms for detecting definite Hermitian matrix pairs (A, B). The most simple algorithms we propose are based on testing the main submatrices of order 1 or 2. These algorithms do not have to give a final answer about (in)definiteness of the given pair, so we develop a more efficient subspace algorithm assuming B is indefinite. Our subspace algorithm for detecting definiteness is based on iterative testing of small full compressed matrix pairs formed using test-subspaces of small dimensions. It is generalization of the method of coordinate relaxation proposed in [36, Section 3.6]. We also propose acceleration of the subspace algorithm in a way that certain linear systems must be solved in every or in some iteration steps. If the matrix pair is definite, the subspace algorithm detects if it is positive or negative definite and returns one definitizing shift. The subspace algorithm is particulary suited for large, sparse and banded matrix pairs, and can be used in testing hyperbolicity of a Hermitian quadratic matrix polynomial. Numerical experiments are given which illustrate efficiency of several variants of our subspace algorithm and comparison is made with an arc algorithm [19, 17, 29]. In the second part of this thesis we are interested in solving partial positive definite GEP (1) where B (and A) is indefinite (both A and B can be singular). Specifically, we are interested in iterative algorithms which will compute a small number of eigenvalues closest to the definiteness interval and corresponding eigenvectors. These algorithms are based on trace minimization property [41, 49]: find the minimum of the trace of the function: $f(X)=X^HAX$ such that $X^HBX=diag(I_{k_+}, -I_{k_-})$ where $X \in \mathbb{C}^{n \times (k_++k_-)}, 1 \leq k_+ \leq n_+, 1 \leq k_- \leq n_-$ and $(n_+, n_-, n_0)$ is the inertia of B. The class of algorithms we propose will be preconditioned gradient type iteration, suitable for large and sparse matrices, previously studied for the case with A and/or B are known to be positive definite (for a survey of preconditioned iterations see [3, 39]). In the recent paper [42] an indefinite variants of LOBPCG algorithm [40] were suggested. The authors of [42] were not aware of any other preconditioned eigenvalue solver tailored to definite matrix pairs with indefinite matrices. In this thesis we propose some new preconditioned eigenvalue solvers suitable for this case, which include truncated and extended versions of indefinite LOBPCG from [42]. Our algorithms use one or two definitizing shifts. For the truncated versions of indefinite LOBPCG, which we call indefinite BPSD/A, we derive a sharp convergence estimates. Since for the LOBPCG type algorithms there are still no sharp convergence estimates, estimates derived for BPSD/A type methods serve as an upper (non-sharp) convergence estimates. We also devise some possibilities of using our algorithms to compute a modest number of eigenvalues around any spectral gap of a definite matrix pair (A, B). Numerical experiments are given which illustrate efficiency and some limitations of our algorithms

Computing interior eigenvalues and corresponding eigenvectors of definite matrix pairs

U prvom dijelu ove disertacije predstavljamo nove algoritme koji za dani hermitski matrični par $(A, B)$ ispituju je li on pozitivno definitan, u smislu da postoji realan broj $\lambda_0$ takav da je matrica $A-\lambda_0B$ pozitivno definitna. Skup svih takvih $\lambda_0$ čini otvoreni interval koji zovemo definitan interval, a bilo koji takav $\lambda_0$ zovemo definitan pomak. Najjednostavniji algoritmi ispitivanja koje predlažemo temelje se na ispitivanju glavnih podmatrica reda 1 ili 2. Također razvijamo efikasniji algoritam ispitivanja potprostora uz pretpostavku indefinitnosti matrice B. Taj se algoritam temelji na iterativnom ispitivanju malih gusto popunjenih komprimiranih parova koji nastaju korištenjem test-potprostora malih dimenzija, a predlažemo i ubrzanje samog algoritma. Algoritam ispitivanja potprostora posebno je pogodan za velike rijetko popunjene vrpčaste matrične parove, a može se primijeniti u ispitivanju hiperbolnosti kvadratnog svojstvenog problema. U drugom dijelu ove disertacije za dani pozitivno definitni matrični par $(A, B)$ reda $n$ s indefinitnom matricom $B$ konstruiramo nove algoritme minimizacije traga funkcije $f(X)=X^HAX$ uz uvjet $X^HBX=diag(I_{k_+}, -I_{k_-})$ gdje je $X \in \mathbb{C}^{n \times (k_++k_-)}, 1 \leq k_+ \leq n_+, 1 \leq k_- \leq n_-$ i $(n_+, n_-, n_0)$ inercija matrice $B$. Predlažemo opći indefinitni algoritam, te razvijamo efikasne algoritme prekondicioniranih gradijentnih iteracija koje smo nazvali indefinitna $m$-shema. Stoga metode indefinitne $m$-sheme za dani pozitivno definitni par i jedan ili dva definitna pomaka (koji se mogu dobiti algoritmom ispitivanja potprostora) istovremeno računaju manji broj unutarnjih svojstvenih vrijednosti oko definitnog intervala i pridružene svojstvene vektore. Također, dajemo ideje kako računati manji broj svojstvenih vrijednosti oko bilo kojeg broja unutar rubova spektra, a izvan definitnog intervala, i pridruženih svojstvenih vektora, danog pozitivno definitnog matričnog para koristeći pozitivno definitnu matricu prekondicioniranja. Algoritmi su posebno pogodni za velike rijetko popunjene matrične parove. Nizom numeričkih eksperimenata pokazujemo efikasnost samih algoritama ispitivanja i algoritama računanja unutarnjih svojstvenih vrijednosti i pridruženih svojstvenih vektora. Efikasnost naših metoda uspoređujemo s nekim postojećim metodama.The generalized eigenvalue problem (GEP) for given matrices $A, B \in \mathbb{C}^{n \times n}$ is to find scalars $\lambda$ and nonzero vectors $x \in \mathbb{C}^n$ such that $Ax = \lambda Bx$ (1). The pair $(\lambda, x)$ is called an eigenpair, $\lambda$ is an eigenvalue and $x$ corresponding eigenvector. GEP (1) where A and B are both Hermitian, or real symmetric, occurs in many applications of mathematics. Very important case is when B (and A) is positive definite (appearing, e.g., in the finite element discretization of self-adjoint and elliptic PDE-eigenvalue problem [25]). Another very important case is when B (and A) is indefinite, but the matrix pair (A, B) is definite, meaning, there exist real numbers $\alpha, \beta$ such that the matrix $\alpha A + \beta B$ is positive definite (appearing, e.g., in mechanics [83] and computational quantum chemistry [4]). Many theoretical properties (variational principles, perturbation theory, etc.) and eigenvalue solvers for Hermitian matrix are extended to definite matrix pairs [64, 79, 83]. A Hermitian matrix pair (A, B) is called positive (negative) definite if there exists a real $\lambda_0$ such that $A- \lambda_0 B$ is positive ( negative) definite. The set of all such $\lambda_0$ is an open interval called the definiteness interval [83], and any such $\lambda_0$ will be called definitizing shift. In the first part of this thesis we propose new algorithms for detecting definite Hermitian matrix pairs (A, B). The most simple algorithms we propose are based on testing the main submatrices of order 1 or 2. These algorithms do not have to give a final answer about (in)definiteness of the given pair, so we develop a more efficient subspace algorithm assuming B is indefinite. Our subspace algorithm for detecting definiteness is based on iterative testing of small full compressed matrix pairs formed using test-subspaces of small dimensions. It is generalization of the method of coordinate relaxation proposed in [36, Section 3.6]. We also propose acceleration of the subspace algorithm in a way that certain linear systems must be solved in every or in some iteration steps. If the matrix pair is definite, the subspace algorithm detects if it is positive or negative definite and returns one definitizing shift. The subspace algorithm is particulary suited for large, sparse and banded matrix pairs, and can be used in testing hyperbolicity of a Hermitian quadratic matrix polynomial. Numerical experiments are given which illustrate efficiency of several variants of our subspace algorithm and comparison is made with an arc algorithm [19, 17, 29]. In the second part of this thesis we are interested in solving partial positive definite GEP (1) where B (and A) is indefinite (both A and B can be singular). Specifically, we are interested in iterative algorithms which will compute a small number of eigenvalues closest to the definiteness interval and corresponding eigenvectors. These algorithms are based on trace minimization property [41, 49]: find the minimum of the trace of the function: $f(X)=X^HAX$ such that $X^HBX=diag(I_{k_+}, -I_{k_-})$ where $X \in \mathbb{C}^{n \times (k_++k_-)}, 1 \leq k_+ \leq n_+, 1 \leq k_- \leq n_-$ and $(n_+, n_-, n_0)$ is the inertia of B. The class of algorithms we propose will be preconditioned gradient type iteration, suitable for large and sparse matrices, previously studied for the case with A and/or B are known to be positive definite (for a survey of preconditioned iterations see [3, 39]). In the recent paper [42] an indefinite variants of LOBPCG algorithm [40] were suggested. The authors of [42] were not aware of any other preconditioned eigenvalue solver tailored to definite matrix pairs with indefinite matrices. In this thesis we propose some new preconditioned eigenvalue solvers suitable for this case, which include truncated and extended versions of indefinite LOBPCG from [42]. Our algorithms use one or two definitizing shifts. For the truncated versions of indefinite LOBPCG, which we call indefinite BPSD/A, we derive a sharp convergence estimates. Since for the LOBPCG type algorithms there are still no sharp convergence estimates, estimates derived for BPSD/A type methods serve as an upper (non-sharp) convergence estimates. We also devise some possibilities of using our algorithms to compute a modest number of eigenvalues around any spectral gap of a definite matrix pair (A, B). Numerical experiments are given which illustrate efficiency and some limitations of our algorithms