Numerically Stable Recurrence Relations for the Communication Hiding Pipelined Conjugate Gradient Method

Pipelined Krylov subspace methods (also referred to as communication-hiding methods) have been proposed in the literature as a scalable alternative to classic Krylov subspace algorithms for iteratively computing the solution to a large linear system in parallel. For symmetric and positive definite system matrices the pipelined Conjugate Gradient method outperforms its classic Conjugate Gradient counterpart on large scale distributed memory hardware by overlapping global communication with essential computations like the matrix-vector product, thus hiding global communication. A well-known drawback of the pipelining technique is the (possibly significant) loss of numerical stability. In this work a numerically stable variant of the pipelined Conjugate Gradient algorithm is presented that avoids the propagation of local rounding errors in the finite precision recurrence relations that construct the Krylov subspace basis. The multi-term recurrence relation for the basis vector is replaced by two-term recurrences, improving stability without increasing the overall computational cost of the algorithm. The proposed modification ensures that the pipelined Conjugate Gradient method is able to attain a highly accurate solution independently of the pipeline length. Numerical experiments demonstrate a combination of excellent parallel performance and improved maximal attainable accuracy for the new pipelined Conjugate Gradient algorithm. This work thus resolves one of the major practical restrictions for the useability of pipelined Krylov subspace methods.Comment: 15 pages, 5 figures, 1 table, 2 algorithm

Robust and scalable 3-D geo-electromagnetic modelling approach using the finite element method

We present a robust and scalable solver for time-harmonic Maxwell's equations for problems with large conductivity contrasts, wide range of frequencies, stretched grids and locally refined meshes. The solver is part of the fully distributed adaptive 3-D electromagnetic modelling scheme which employs the finite element method and unstructured non-conforming hexahedral meshes for spatial discretization using the open-source software deal.II. We use the complex-valued electric field formulation and split it into two real-valued equations for which we utilize an optimal block-diagonal pre-conditioner. Application of this pre-conditioner requires the solution of two smaller real-valued symmetric problems. We solve them by using either a direct solver or the conjugate gradient method pre-conditioned with the recently introduced auxiliary space technique. The auxiliary space pre-conditioner reformulates the original problem in form of several simpler ones, which are then solved using highly efficient algebraic multigrid methods. In this paper, we consider the magnetotelluric case and verify our numerical scheme by using COMMEMI 3-D models. Afterwards, we run a series of numerical experiments and demonstrate that the solver converges in a small number of iterations for a wide frequency range and variable problem sizes. The number of iterations is independent of the problem size, but exhibits a mild dependency on frequency. To test the stability of the method on locally refined meshes, we have implemented a residual-based a posteriori error estimator and compared it with uniform mesh refinement for problems up to 200 million unknowns. We test the scalability of the most time consuming parts of our code and show that they fulfill the strong scaling assumption as long as each MPI process possesses enough degrees of freedom to alleviate communication overburden. Finally, we refer back to a direct solver-based pre-conditioner and analyse its complexity in time. The results show that for multiple right-hand sides the direct solver-based pre-conditioner can still be faster for problems of medium size. On the other hand, it also shows non-linear growth in memory, whereas the auxiliary space method increases only linearly.ISSN:0956-540XISSN:1365-246