Modified successive overrelaxation (SOR) type methods for M-matrices

The SOR is a basic iterative method for solution of the linear system =. Such systems can easily be solved using direct methods such as Gaussian elimination. However, when the coefficient matrix is large and sparse, iterative methods such as the SOR become indispensable. A new preconditioner for speeding up the convergence of the SOR iterative method for solving the linear system = is proposed. Arising from the preconditioner, two new preconditioned iterative techniques of the SOR method are developed. The preconditioned iterations are applied to the linear system whose coefficient matrix is an −matrix. Convergence of the preconditioned iterations is established through standard procedures. Numerical examples and results comparison are in conformity with the analytic results. More so, it is established that the spectral radii of the proposed preconditioned SOR 1 and 2 are less than that of the classical SOR, which implies faster convergence

Parallel Newton Method for High-Speed Viscous Separated Flowfields. G.U. Aero Report 9210

This paper presents a new technique to parallelize Newton method for the locally conical approximate, laminar Navier-Stokes solutions on a distributed memory parallel computer. The method uses Newton's method for nonlinear systems of equations to find steady-state solutions. The parallelization is based on a parallel iterative solver for large sparse non-symmetric linear system. The method of distributed storage of the matrix data results in the corresponding geometric domain decomposition. The large sparse Jacobian matrix is then generated distributively in each subdomain. Since the numerical algorithms on the global domain are unchanged, the convergence and the accuracy of the original sequential scheme are maintained, and no inner boundary condition is needed

Newton-like Methods for Fast High Resolution Simulation of Hypersonic Viscous Flow. G.U. Aero Report 9228

Two Newton-like methods, i.e. the sparse finite difference Newton method and the sparse quasi-Newton method, are applied to the Navier-Stokes solutions of hypersonic flows using the Osher flux difference splitting high resolution scheme. The resulting large block structured sparse linear system is solved by a new multilevel iterative solver, the ct-GMRES method, which includes a preconditioner and a damping factor. The algorithm is demonstrated to provide fast, accurate solutions of the hypersonic flow over a cone at high angle of attack. Being parallelisable on distributed memory multiprocessors and having an ability to tackle highly non-linear problems, it has great promise in tackling more complex practical air vehicle configurations. As a by-product of using the GMRES method, in which Hessenberg matrices are generated, the eigenvalues of the linear system can be estimated using the Amoldi method. The spectra produced provide some insight into the behaviour of the GMRES method for different linear systems corresponding to different preconditioning and damping

An efficient GPU version of the preconditioned GMRES method

[EN] In a large number of scientific applications, the solution of sparse linear systems is the stage that concentrates most of the computational effort. This situation has motivated the study and development of several iterative solvers, among which preconditioned Krylov subspace methods occupy a place of privilege. In a previous effort, we developed a GPU-aware version of the GMRES method included in ILUPACK, a package of solvers distinguished by its inverse-based multilevel ILU preconditioner. In this work, we study the performance of our previous proposal and integrate several enhancements in order to mitigate its principal bottlenecks. The numerical evaluation shows that our novel proposal can reach important run-time reductions.Aliaga, JI.; Dufrechou, E.; Ezzatti, P.; Quintana-Orti, ES. (2019). An efficient GPU version of the preconditioned GMRES method. The Journal of Supercomputing. 75(3):1455-1469. https://doi.org/10.1007/s11227-018-2658-1S14551469753Aliaga JI, Badia RM, Barreda M, Bollhöfer M, Dufrechou E, Ezzatti P, Quintana-Ortí ES (2016) Exploiting task and data parallelism in ILUPACK’s preconditioned CG solver on NUMA architectures and many-core accelerators. Parallel Comput 54:97–107Aliaga JI, Bollhöfer M, Dufrechou E, Ezzatti P, Quintana-Ortí ES (2016) A data-parallel ILUPACK for sparse general and symmetric indefinite linear systems. In: Lecture Notes in Computer Science, 14th Int. Workshop on Algorithms, Models and Tools for Parallel Computing on Heterogeneous Platforms—HeteroPar’16. SpringerAliaga JI, Bollhöfer M, Martín AF, Quintana-Ortí ES (2011) Exploiting thread-level parallelism in the iterative solution of sparse linear systems. Parallel Comput 37(3):183–202Aliaga JI, Bollhöfer M, Martín AF, Quintana-Ortí ES (2012) Parallelization of multilevel ILU preconditioners on distributed-memory multiprocessors. Appl Parallel Sci Comput LNCS 7133:162–172Aliaga JI, Dufrechou E, Ezzatti P, Quintana-Ortí ES (2018) Accelerating a preconditioned GMRES method in massively parallel processors. In: CMMSE 2018: Proceedings of the 18th International Conference on Mathematical Methods in Science and Engineering (2018)Bollhöfer M, Grote MJ, Schenk O (2009) Algebraic multilevel preconditioner for the Helmholtz equation in heterogeneous media. SIAM J Sci Comput 31(5):3781–3805Bollhöfer M, Saad Y (2006) Multilevel preconditioners constructed from inverse-based ILUs. SIAM J Sci Comput 27(5):1627–1650Dufrechou E, Ezzatti P (2018) A new GPU algorithm to compute a level set-based analysis for the parallel solution of sparse triangular systems. In: 2018 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2018, Canada, 2018. IEEE Computer SocietyDufrechou E, Ezzatti P (2018) Solving sparse triangular linear systems in modern GPUs: a synchronization-free algorithm. In: 2018 26th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP), pp 196–203. https://doi.org/10.1109/PDP2018.2018.00034Eijkhout V (1992) LAPACK working note 50: distributed sparse data structures for linear algebra operations. Tech. rep., Knoxville, TN, USAGolub GH, Van Loan CF (2013) Matrix computationsHe K, Tan SXD, Zhao H, Liu XX, Wang H, Shi G (2016) Parallel GMRES solver for fast analysis of large linear dynamic systems on GPU platforms. Integration 52:10–22 http://www.sciencedirect.com/science/article/pii/S016792601500084XLiu W, Li A, Hogg JD, Duff IS, Vinter B (2017) Fast synchronization-free algorithms for parallel sparse triangular solves with multiple right-hand sides. Concurr Comput 29(21)Saad Y (2003) Iterative methods for sparse linear systems, 2nd edn. SIAM, PhiladelphiaSchenk O, Wächter A, Weiser M (2008) Inertia revealing preconditioning for large-scale nonconvex constrained optimization. SIAM J Sci Comput 31(2):939–96

On an integrated Krylov-ADI solver for large-scale Lyapunov equations

One of the most computationally expensive steps of the low-rank ADI method for large-scale Lyapunov equations is the solution of a shifted linear system at each iteration. We propose the use of the extended Krylov subspace method for this task. In particular, we illustrate how a single approximation space can be constructed to solve all the shifted linear systems needed to achieve a prescribed accuracy in terms of Lyapunov residual norm. Moreover, we show how to fully merge the two iterative procedures in order to obtain a novel, efcient implementation of the low-rank ADI method, for an important class of equations. Many state-of-the-art algorithms for the shift computation can be easily incorporated into our new scheme, as well. Several numerical results illustrate the potential of our novel procedure when compared to an implementation of the low-rank ADI method based on sparse direct solvers for the shifted linear systems

Preconditioning for standard and two-sided Krylov subspace methods

This thesis is concerned with the solution of large nonsymmetric sparse linear systems. The main focus is on iterative solution methods and preconditioning. Assuming the linear system has a special structure, a minimal residual method called TSMRES, based on a generalization of a Krylov subspace, is presented and its convergence properties studied. In numerical experiments it is shown that there are cases where the convergence speed of TSMRES is faster than that of GMRES and vice versa. The numerical implementation of TSMRES is studied and a new numerically stable formulation is presented. In addition it is shown that preconditioning general linear systems for TSMRES by splittings is feasible in some cases. The direct solution of sparse linear systems of the Hessenberg type is also studied. Finally, a new approach to compute a factorized approximate inverse of a matrix suitable for preconditioning is presented

Volume 19 (2) 2013

New generation of General Purpose Graphic Processing Unit (GPGPU) cards with their large computation power allow to approach difficult tasks from Radio Frequency Integrated Circuits (RFICs) modeling area. Using different electromagnetic modeling methods, the Finite Element Method (FEM) and the Finite Integration Technique (FIT), to model Radio Frequency Integrated Circuit (RFIC) devices, large linear equations systems have to be solved. This paper presents the benefits of using Graphic Processing Unit (GPU) computations for solving such systems which are characterized by sparse complex matrices. CUSP is a GPU generic parallel algorithms library for sparse linear algebra and graph computations based on Compute Unified Device Architecture (CUDA). The code is calling iterative methods available in CUSP in order to solve those complex linear equation systems. The tests were performed on various Central Processing Units (CPU) and GPU hardware configurations. The results of these tests show that using GPU computations for solving the linear equations systems, the electromagnetic modeling process of RFIC devices can be accelerated and at the same time a high level of computation accuracy is maintained. Tests were carried out on matrices obtained for an integrated inductor designed for RFICs, and for Micro Stripe (MS) designed for Photonics Integrated Circuit (PIC).Pozna

Generalized preconditioning strategies

Over the past decade Professor David J. Evans [1968] has suggested the use of ‘Preconditioning’ in iterative methods for solving large, sparse systems of linear equations, which arise from the finite difference approximations to the partial differential equations. Since then, certain aspects on preconditioning have appeared in the literature and a whole new theory constructed. The versatility of the preconditioning concept is shown by the stimulating exploration of new numerical algorithms and methods of their realization. The aim of this thesis is to emphasise in the theory we use and develop together with the practice we state. This study led to a new form of preconditioning, which has not yet appeared in the literature. Specifically, we consider the conditioning matrix factorized into two rectangular matrices, so as to develop a new preconditioned iterative method and its related properties as well. It requires the selection of two parameters to be applied, a preconditioning parameter at its optimal value and an acceleration parameter in such a fashion that a simultaneous displacement method is applicable. [Continues.

Linear Stability Analysis Using Lyapunov Inverse Iteration

In this dissertation, we develop robust and efficient methods for linear stability analysis of large-scale dynamical systems, with emphasis on the incompressible Navier-Stokes equations. Linear stability analysis is a widely used approach for studying whether a steady state of a dynamical system is sensitive to small perturbations. The main mathematical tool that we consider in this dissertation is Lyapunov inverse iteration, a recently developed iterative method for computing the eigenvalue with smallest modulus of a special eigenvalue problem that can be specified in the form of a Lyapunov equation. It has the following "inner-outer" structure: the outer iteration is the eigenvalue computation and the inner iteration is solving a large-scale Lyapunov equation. This method has two applications in linear stability analysis: it can be used to estimate the critical value of a physical parameter at which the steady state becomes unstable (i.e., sensitive to small perturbations), and it can also be applied to compute a few rightmost eigenvalues of the Jacobian matrix. We present numerical performance of Lyapunov inverse iteration in both applications, analyze its convergence in the second application, and propose strategies of implementing it efficiently for each application. In previous work, Lyapunov inverse iteration has been used to estimate the critical parameter value at which a parameterized path of steady states loses stability. We refine this method by proposing an adaptive stopping criterion for the Lyapunov solve (inner iteration) that depends on the accuracy of the eigenvalue computation (outer iteration). The use of such a criterion achieves dramatic savings in computational cost and does not affect the convergence of the target eigenvalue. The method of previous work has the limitation that it can only be used at a stable point in the neighborhood of the critical point. We further show that Lyapunov inverse iteration can also be used to generate a few rightmost eigenvalues of the Jacobian matrix at any stable point. These eigenvalues are crucial in linear stability analysis, and existing approaches for computing them are not robust. A convergence analysis of this method leads to a way of implementing it that only entails one Lyapunov solve. In addition, we explore the utility of various Lyapunov solvers in both applications of Lyapunov inverse iteration. We observe that different Lyapunov solvers should be used for the Lyapunov equations arising from the two applications. Applying a Lyapunov solver entails solving a number of large and sparse linear systems. We explore the use of sparse iterative methods for this task and construct a new variant of the Lyapunov solver that significantly reduces the costs of the sparse linear solves