PACF: A precision-adjustable computational framework for solving singular values

Singular value decomposition (SVD) plays a significant role in matrix analysis, and the differential quotient difference with shifts (DQDS) algorithm is an important technique for solving singular values of upper bidiagonal matrices. However, ill-conditioned matrices and large-scale matrices may cause inaccurate results or long computation times when solving singular values. At the same time, it is difficult for users to effectively find the desired solution according to their needs. In this paper, we design a precision-adjustable computational framework for solving singular values, named PACF. In our framework, the same solution algorithm contains three options: original mode, high-precision mode, and mixed-precision mode. The first algorithm is the original version of the algorithm. The second algorithm is a reliable numerical algorithm we designed using Error-free transformation (EFT) technology. The last algorithm is an efficient numerical algorithm we developed using the mixed-precision idea. Our PACF can add different solving algorithms for different types of matrices, which are universal and extensible. Users can choose different algorithms to solve singular values according to different needs. This paper implements the high-precision DQDS and mixed-precision DQDS algorithms and conducts extensive experiments on a supercomputing platform to demonstrate that our algorithm is reliable and efficient. Besides, we introduce the error analysis of the inner loop of the DQDS and HDQDS algorithms

On the optimality and sharpness of Laguerre's lower bound on the smallest eigenvalue of a symmetric positive definite matrix

summary:Lower bounds on the smallest eigenvalue of a symmetric positive definite matrix $A\in \mathbb {R}^{m\times m}$ play an important role in condition number estimation and in iterative methods for singular value computation. In particular, the bounds based on ${\rm Tr}(A^{-1})$ and ${\rm Tr}(A^{-2})$ have attracted attention recently, because they can be computed in $O(m)$ operations when $A$ is tridiagonal. In this paper, we focus on these bounds and investigate their properties in detail. First, we consider the problem of finding the optimal bound that can be computed solely from ${\rm Tr}(A^{-1})$ and ${\rm Tr}(A^{-2})$ and show that the so called Laguerre's lower bound is the optimal one in terms of sharpness. Next, we study the gap between the Laguerre bound and the smallest eigenvalue. We characterize the situation in which the gap becomes largest in terms of the eigenvalue distribution of $A$ and show that the gap becomes smallest when $\{{\rm Tr}(A^{-1})\}^2/{\rm Tr}(A^{-2})$ approaches 1 or $m$. These results will be useful, for example, in designing efficient shift strategies for singular value computation algorithms

Differential qd algorithm with shifts for rank-structured matrices

Although QR iterations dominate in eigenvalue computations, there are several important cases when alternative LR-type algorithms may be preferable. In particular, in the symmetric tridiagonal case where differential qd algorithm with shifts (dqds) proposed by Fernando and Parlett enjoys often faster convergence while preserving high relative accuracy (that is not guaranteed in QR algorithm). In eigenvalue computations for rank-structured matrices QR algorithm is also a popular choice since, in the symmetric case, the rank structure is preserved. In the unsymmetric case, however, QR algorithm destroys the rank structure and, hence, LR-type algorithms come to play once again. In the current paper we discover several variants of qd algorithms for quasiseparable matrices. Remarkably, one of them, when applied to Hessenberg matrices becomes a direct generalization of dqds algorithm for tridiagonal matrices. Therefore, it can be applied to such important matrices as companion and confederate, and provides an alternative algorithm for finding roots of a polynomial represented in the basis of orthogonal polynomials. Results of preliminary numerical experiments are presented

Perturbation splitting for more accurate eigenvalues

Let $T$ be a symmetric tridiagonal matrix with entries and eigenvalues of different magnitudes. For some $T$, small entrywise relative perturbations induce small errors in the eigenvalues, independently of the size of the entries of the matrix; this is certainly true when the perturbed matrix can be written as $\widetilde{T}=X^{T}TX$ with small $||X^{T}X-I||$. Even if it is not possible to express in this way the perturbations in every entry of $T$, much can be gained by doing so for as many as possible entries of larger magnitude. We propose a technique which consists of splitting multiplicative and additive perturbations to produce new error bounds which, for some matrices, are much sharper than the usual ones. Such bounds may be useful in the development of improved software for the tridiagonal eigenvalue problem, and we describe their role in the context of a mixed precision bisection-like procedure. Using the very same idea of splitting perturbations (multiplicative and additive), we show that when $T$ defines well its eigenvalues, the numerical values of the pivots in the usual decomposition $T-\lambda I=LDL^{T}$ may be used to compute approximations with high relative precision.Fundação para a Ciência e Tecnologia (FCT) - POCI 201

Restructuring the Tridiagonal and Bidiagonal QR Algorithms for Performance

We show how both the tridiagonal and bidiagonal QR algorithms can be restructured so that they be- come rich in operations that can achieve near-peak performance on a modern processor. The key is a novel, cache-friendly algorithm for applying multiple sets of Givens rotations to the eigenvector/singular vector matrix. This algorithm is then implemented with optimizations that (1) leverage vector instruction units to increase floating-point throughput, and (2) fuse multiple rotations to decrease the total number of memory operations. We demonstrate the merits of these new QR algorithms for computing the Hermitian eigenvalue decomposition (EVD) and singular value decomposition (SVD) of dense matrices when all eigen- vectors/singular vectors are computed. The approach yields vastly improved performance relative to the traditional QR algorithms for these problems and is competitive with two commonly used alternatives— Cuppen’s Divide and Conquer algorithm and the Method of Multiple Relatively Robust Representations— while inheriting the more modest O(n) workspace requirements of the original QR algorithms. Since the computations performed by the restructured algorithms remain essentially identical to those performed by the original methods, robust numerical properties are preserved

MRRR-based Eigensolvers for Multi-core Processors and Supercomputers

The real symmetric tridiagonal eigenproblem is of outstanding importance in numerical computations; it arises frequently as part of eigensolvers for standard and generalized dense Hermitian eigenproblems that are based on a reduction to tridiagonal form. For its solution, the algorithm of Multiple Relatively Robust Representations (MRRR or MR3 in short) - introduced in the late 1990s - is among the fastest methods. To compute k eigenpairs of a real n-by-n tridiagonal T, MRRR only requires O(kn) arithmetic operations; in contrast, all the other practical methods require O(k^2 n) or O(n^3) operations in the worst case. This thesis centers around the performance and accuracy of MRRR.Comment: PhD thesi