A GPU-based hyperbolic SVD algorithm

A one-sided Jacobi hyperbolic singular value decomposition (HSVD) algorithm, using a massively parallel graphics processing unit (GPU), is developed. The algorithm also serves as the final stage of solving a symmetric indefinite eigenvalue problem. Numerical testing demonstrates the gains in speed and accuracy over sequential and MPI-parallelized variants of similar Jacobi-type HSVD algorithms. Finally, possibilities of hybrid CPU--GPU parallelism are discussed.Comment: Accepted for publication in BIT Numerical Mathematic

Convergence Analysis of Extended LOBPCG for Computing Extreme Eigenvalues

This paper is concerned with the convergence analysis of an extended variation of the locally optimal preconditioned conjugate gradient method (LOBPCG) for the extreme eigenvalue of a Hermitian matrix polynomial which admits some extended form of Rayleigh quotient. This work is a generalization of the analysis by Ovtchinnikov (SIAM J. Numer. Anal., 46(5):2567-2592, 2008). As instances, the algorithms for definite matrix pairs and hyperbolic quadratic matrix polynomials are shown to be globally convergent and to have an asymptotically local convergence rate. Also, numerical examples are given to illustrate the convergence.Comment: 21 pages, 2 figure

Laplacian Mixture Modeling for Network Analysis and Unsupervised Learning on Graphs

Laplacian mixture models identify overlapping regions of influence in unlabeled graph and network data in a scalable and computationally efficient way, yielding useful low-dimensional representations. By combining Laplacian eigenspace and finite mixture modeling methods, they provide probabilistic or fuzzy dimensionality reductions or domain decompositions for a variety of input data types, including mixture distributions, feature vectors, and graphs or networks. Provable optimal recovery using the algorithm is analytically shown for a nontrivial class of cluster graphs. Heuristic approximations for scalable high-performance implementations are described and empirically tested. Connections to PageRank and community detection in network analysis demonstrate the wide applicability of this approach. The origins of fuzzy spectral methods, beginning with generalized heat or diffusion equations in physics, are reviewed and summarized. Comparisons to other dimensionality reduction and clustering methods for challenging unsupervised machine learning problems are also discussed.Comment: 13 figures, 35 reference

Inertia-based spectrum slicing for symmetric quadratic eigenvalue problems

[EN] In the quadratic eigenvalue problem (QEP) with all coefficient matrices symmetric, there can be complex eigenvalues. However, some applications need to compute real eigenvalues only. We propose a Lanczos-based method for computing all real eigenvalues contained in a given interval of large-scale symmetric QEPs. The method uses matrix inertias of the quadratic polynomial evaluated at different shift values. In this way, for hyperbolic problems, it is possible to make sure that all eigenvalues in the interval have been computed. We also discuss the general nonhyperbolic case. Our implementation is memory-efficient by representing the computed pseudo-Lanczos basis in a compact tensor product representation. We show results of computational experiments with a parallel implementation in the SLEPc library.Agencia Estatal de Investigacion, Grant/Award Number: TIN2016-75985-PCampos, C.; Román Moltó, JE. (2020). Inertia-based spectrum slicing for symmetric quadratic eigenvalue problems. Numerical Linear Algebra with Applications. 27(4):1-17. https://doi.org/10.1002/nla.2293S117274Tisseur, F., & Meerbergen, K. (2001). The Quadratic Eigenvalue Problem. SIAM Review, 43(2), 235-286. doi:10.1137/s0036144500381988Veselić, K. (2011). Damped Oscillations of Linear Systems. Lecture Notes in Mathematics. doi:10.1007/978-3-642-21335-9Grimes, R. G., Lewis, J. G., & Simon, H. D. (1994). A Shifted Block Lanczos Algorithm for Solving Sparse Symmetric Generalized Eigenproblems. SIAM Journal on Matrix Analysis and Applications, 15(1), 228-272. doi:10.1137/s0895479888151111Campos, C., & Roman, J. E. (2012). Strategies for spectrum slicing based on restarted Lanczos methods. Numerical Algorithms, 60(2), 279-295. doi:10.1007/s11075-012-9564-zLi, R., Xi, Y., Vecharynski, E., Yang, C., & Saad, Y. (2016). A Thick-Restart Lanczos Algorithm with Polynomial Filtering for Hermitian Eigenvalue Problems. SIAM Journal on Scientific Computing, 38(4), A2512-A2534. doi:10.1137/15m1054493Guo, C.-H., Higham, N. J., & Tisseur, F. (2010). An Improved Arc Algorithm for Detecting Definite Hermitian Pairs. SIAM Journal on Matrix Analysis and Applications, 31(3), 1131-1151. doi:10.1137/08074218xNiendorf, V., & Voss, H. (2010). Detecting hyperbolic and definite matrix polynomials. Linear Algebra and its Applications, 432(4), 1017-1035. doi:10.1016/j.laa.2009.10.014NakatsukasaY NoferiniV. Inertia laws and localization of real eigenvalues for generalized indefinite eigenvalue problems;2017. Preprint arXiv:1711.00495.Parlett, B. N., & Chen, H. C. (1990). Use of indefinite pencils for computing damped natural modes. Linear Algebra and its Applications, 140, 53-88. doi:10.1016/0024-3795(90)90222-xCampos, C., & Roman, J. E. (2016). Restarted Q-Arnoldi-type methods exploiting symmetry in quadratic eigenvalue problems. BIT Numerical Mathematics, 56(4), 1213-1236. doi:10.1007/s10543-016-0601-5Guo, C.-H., & Lancaster, P. (2005). Algorithms for hyperbolic quadratic eigenvalue problems. Mathematics of Computation, 74(252), 1777-1792. doi:10.1090/s0025-5718-05-01748-5Li, H., & Cai, Y. (2015). Solving the real eigenvalues of hermitian quadratic eigenvalue problems via bisection. The Electronic Journal of Linear Algebra, 30, 721-743. doi:10.13001/1081-3810.1979RomanJE CamposC RomeroE andTomasA. SLEPc users manual. DSIC‐II/24/02–Revision 3.9. D. Sistemes Informàtics i Computació Universitat Politècnica de València;2018.Hernandez, V., Roman, J. E., & Vidal, V. (2005). SLEPc. ACM Transactions on Mathematical Software, 31(3), 351-362. doi:10.1145/1089014.1089019Guo, J.-S., Lin, W.-W., & Wang, C.-S. (1995). Numerical solutions for large sparse quadratic eigenvalue problems. Linear Algebra and its Applications, 225, 57-89. doi:10.1016/0024-3795(93)00318-tSleijpen, G. L. G., Booten, A. G. L., Fokkema, D. R., & van der Vorst, H. A. (1996). Jacobi-davidson type methods for generalized eigenproblems and polynomial eigenproblems. BIT Numerical Mathematics, 36(3), 595-633. doi:10.1007/bf01731936Bai, Z., & Su, Y. (2005). SOAR: A Second-order Arnoldi Method for the Solution of the Quadratic Eigenvalue Problem. SIAM Journal on Matrix Analysis and Applications, 26(3), 640-659. doi:10.1137/s0895479803438523Güttel, S., & Tisseur, F. (2017). The nonlinear eigenvalue problem. Acta Numerica, 26, 1-94. doi:10.1017/s0962492917000034Yang, L., Sun, Y., & Gong, F. (2018). The inexact residual iteration method for quadratic eigenvalue problem and the analysis of convergence. Journal of Computational and Applied Mathematics, 332, 45-55. doi:10.1016/j.cam.2017.10.003Keçeli, M., Corsetti, F., Campos, C., Roman, J. E., Zhang, H., Vázquez-Mayagoitia, Á., … Wagner, A. F. (2018). SIESTA-SIPs: Massively parallel spectrum-slicing eigensolver for an ab initio molecular dynamics package. Journal of Computational Chemistry, 39(22), 1806-1814. doi:10.1002/jcc.25350Voss, H., Werner, B., & Hadeler, K. P. (1982). A minimax principle for nonlinear eigenvalue problems with applications to nonoverdamped systems. Mathematical Methods in the Applied Sciences, 4(1), 415-424. doi:10.1002/mma.1670040126Higham, N. J., Mackey, D. S., & Tisseur, F. (2009). Definite Matrix Polynomials and their Linearization by Definite Pencils. SIAM Journal on Matrix Analysis and Applications, 31(2), 478-502. doi:10.1137/080721406Al-Ammari, M., & Tisseur, F. (2012). Hermitian matrix polynomials with real eigenvalues of definite type. Part I: Classification. Linear Algebra and its Applications, 436(10), 3954-3973. doi:10.1016/j.laa.2010.08.035Gohberg, I., Lancaster, P., & Rodman, L. (1980). Spectral Analysis of Selfadjoint Matrix Polynomials. The Annals of Mathematics, 112(1), 33. doi:10.2307/1971320RozložnÍk, M., Okulicka-DŁużewska, F., & Smoktunowicz, A. (2015). Cholesky-Like Factorization of Symmetric Indefinite Matrices and Orthogonalization with Respect to Bilinear Forms. SIAM Journal on Matrix Analysis and Applications, 36(2), 727-751. doi:10.1137/130947003Lu, D., Su, Y., & Bai, Z. (2016). Stability Analysis of the Two-level Orthogonal Arnoldi Procedure. SIAM Journal on Matrix Analysis and Applications, 37(1), 195-214. doi:10.1137/151005142Campos, C., & Roman, J. E. (2016). Parallel Krylov Solvers for the Polynomial Eigenvalue Problem in SLEPc. SIAM Journal on Scientific Computing, 38(5), S385-S411. doi:10.1137/15m1022458Higham, N. J., Mackey, D. S., Mackey, N., & Tisseur, F. (2007). Symmetric Linearizations for Matrix Polynomials. SIAM Journal on Matrix Analysis and Applications, 29(1), 143-159. doi:10.1137/050646202BalayS AbhyankarS AdamsM et al. PETSc users manual. ANL‐95/11 ‐ Revision 3.10. Argonne National Laboratory;2018.Betcke, T., Higham, N. J., Mehrmann, V., Schröder, C., & Tisseur, F. (2013). NLEVP. ACM Transactions on Mathematical Software, 39(2), 1-28. doi:10.1145/2427023.2427024Assink, J., Waxler, R., & Velea, D. (2017). A wide-angle high Mach number modal expansion for infrasound propagation. The Journal of the Acoustical Society of America, 141(3), 1781-1792. doi:10.1121/1.497757

A Matrix Hyperbolic Cosine Algorithm and Applications

In this paper, we generalize Spencer's hyperbolic cosine algorithm to the matrix-valued setting. We apply the proposed algorithm to several problems by analyzing its computational efficiency under two special cases of matrices; one in which the matrices have a group structure and an other in which they have rank-one. As an application of the former case, we present a deterministic algorithm that, given the multiplication table of a finite group of size $n$, it constructs an expanding Cayley graph of logarithmic degree in near-optimal O(n^2 log^3 n) time. For the latter case, we present a fast deterministic algorithm for spectral sparsification of positive semi-definite matrices, which implies an improved deterministic algorithm for spectral graph sparsification of dense graphs. In addition, we give an elementary connection between spectral sparsification of positive semi-definite matrices and element-wise matrix sparsification. As a consequence, we obtain improved element-wise sparsification algorithms for diagonally dominant-like matrices.Comment: 16 pages, simplified proof and corrected acknowledging of prior work in (current) Section

Nonlinear theory for coalescing characteristics in multiphase Whitham modulation theory

The multiphase Whitham modulation equations with $N$ phases have $2N$ characteristics which may be of hyperbolic or elliptic type. In this paper a nonlinear theory is developed for coalescence, where two characteristics change from hyperbolic to elliptic via collision. Firstly, a linear theory develops the structure of colliding characteristics involving the topological sign of characteristics and multiple Jordan chains, and secondly a nonlinear modulation theory is developed for transitions. The nonlinear theory shows that coalescing characteristics morph the Whitham equations into an asymptotically valid geometric form of the two-way Boussinesq equation. That is, coalescing characteristics generate dispersion, nonlinearity and complex wave fields. For illustration, the theory is applied to coalescing characteristics associated with the modulation of two-phase travelling-wave solutions of coupled nonlinear Schr\"odinger equations, highlighting how collisions can be identified and the relevant dispersive dynamics constructed.Comment: 40 pages, 2 figure