Angles between subspaces and their tangents

Principal angles between subspaces (PABS) (also called canonical angles) serve as a classical tool in mathematics, statistics, and applications, e.g., data mining. Traditionally, PABS are introduced via their cosines. The cosines and sines of PABS are commonly defined using the singular value decomposition. We utilize the same idea for the tangents, i.e., explicitly construct matrices, such that their singular values are equal to the tangents of PABS, using several approaches: orthonormal and non-orthonormal bases for subspaces, as well as projectors. Such a construction has applications, e.g., in analysis of convergence of subspace iterations for eigenvalue problems.Comment: 15 pages, 1 figure, 2 tables. Accepted to Journal of Numerical Mathematic

Observations on degenerate saddle point problems

We investigate degenerate saddle point problems, which can be viewed as limit cases of standard mixed formulations of symmetric problems with large jumps in coefficients. We prove that they are well-posed in a standard norm despite the degeneracy. By wellposedness we mean a stable dependence of the solution on the right-hand side. A known approach of splitting the saddle point problem into separate equations for the primary unknown and for the Lagrange multiplier is used. We revisit the traditional Ladygenskaya--Babu\v{s}ka--Brezzi (LBB) or inf--sup condition as well as the standard coercivity condition, and analyze how they are affected by the degeneracy of the corresponding bilinear forms. We suggest and discuss generalized conditions that cover the degenerate case. The LBB or inf--sup condition is necessary and sufficient for wellposedness of the problem with respect to the Lagrange multiplier under some assumptions. The generalized coercivity condition is necessary and sufficient for wellposedness of the problem with respect to the primary unknown under some other assumptions. We connect the generalized coercivity condition to the positiveness of the minimum gap of relevant subspaces, and propose several equivalent expressions for the minimum gap. Our results provide a foundation for research on uniform wellposedness of mixed formulations of symmetric problems with large jumps in coefficients in a standard norm, independent of the jumps. Such problems appear, e.g., in numerical simulations of composite materials made of components with contrasting properties.Comment: 8 page

Gradient flow approach to geometric convergence analysis of preconditioned eigensolvers

Preconditioned eigenvalue solvers (eigensolvers) are gaining popularity, but their convergence theory remains sparse and complex. We consider the simplest preconditioned eigensolver--the gradient iterative method with a fixed step size--for symmetric generalized eigenvalue problems, where we use the gradient of the Rayleigh quotient as an optimization direction. A sharp convergence rate bound for this method has been obtained in 2001--2003. It still remains the only known such bound for any of the methods in this class. While the bound is short and simple, its proof is not. We extend the bound to Hermitian matrices in the complex space and present a new self-contained and significantly shorter proof using novel geometric ideas.Comment: 8 pages, 2 figures. Accepted to SIAM J. Matrix Anal. (SIMAX

Absolute value preconditioning for symmetric indefinite linear systems

We introduce a novel strategy for constructing symmetric positive definite (SPD) preconditioners for linear systems with symmetric indefinite matrices. The strategy, called absolute value preconditioning, is motivated by the observation that the preconditioned minimal residual method with the inverse of the absolute value of the matrix as a preconditioner converges to the exact solution of the system in at most two steps. Neither the exact absolute value of the matrix nor its exact inverse are computationally feasible to construct in general. However, we provide a practical example of an SPD preconditioner that is based on the suggested approach. In this example we consider a model problem with a shifted discrete negative Laplacian, and suggest a geometric multigrid (MG) preconditioner, where the inverse of the matrix absolute value appears only on the coarse grid, while operations on finer grids are based on the Laplacian. Our numerical tests demonstrate practical effectiveness of the new MG preconditioner, which leads to a robust iterative scheme with minimalist memory requirements

Angle-free cluster robust Ritz value bounds for restarted block eigensolvers

Convergence rates of block iterations for solving eigenvalue problems typically measure errors of Ritz values approximating eigenvalues. The errors of the Ritz values are commonly bounded in terms of principal angles between the initial or iterative subspace and the invariant subspace associated with the target eigenvalues. Such bounds thus cannot be applied repeatedly as needed for restarted block eigensolvers, since the left- and right-hand sides of the bounds use different terms. They must be combined with additional bounds which could cause an overestimation. Alternative repeatable bounds that are angle-free and depend only on the errors of the Ritz values have been pioneered for Hermitian eigenvalue problems in doi:10.1515/rnam.1987.2.5.371 but only for a single extreme Ritz value. We extend this result to all Ritz values and achieve robustness for clustered eigenvalues by utilizing nonconsecutive eigenvalues. Our new bounds cover the restarted block Lanczos method and its modifications with shift-and-invert and deflation, and are numerically advantageous.Comment: 24 pages, 4 figure

Bounds for the Rayleigh quotient and the spectrum of self-adjoint operators

The absolute change in the Rayleigh quotient (RQ) is bounded in this paper in terms of the norm of the residual and the change in the vector. If $x$ is an eigenvector of a self-adjoint bounded operator $A$ in a Hilbert space, then the RQ of the vector $x$, denoted by $\rho(x)$, is an exact eigenvalue of $A$. In this case, the absolute change of the RQ $|\rho(x)-\rho(y)|$ becomes the absolute error in an eigenvalue $\rho(x)$ of $A$ approximated by the RQ $\rho(y)$ on a given vector $y.$ There are three traditional kinds of bounds of the eigenvalue error: a priori bounds via the angle between vectors $x$ and $y$; a posteriori bounds via the norm of the residual $Ay-\rho(y)y$ of vector $y$; mixed type bounds using both the angle and the norm of the residual. We propose a unifying approach to prove known bounds of the spectrum, analyze their sharpness, and derive new sharper bounds. The proof approach is based on novel RQ vector perturbation identities.Comment: 13 page