The observed and expected continued growth in the number of nodes in
large-scale parallel computers gives rise to two major challenges: global
communication operations are becoming major bottlenecks due to their limited
scalability, and the likelihood of node failures is increasing. We study an
approach for addressing these challenges in the context of solving large sparse
linear systems. In particular, we focus on the pipelined preconditioned
conjugate gradient (PPCG) method, which has been shown to successfully deal
with the first of these challenges. In this paper, we address the second
challenge. We present extensions to the PPCG solver and two of its variants
which make them resilient against the failure of a compute node while fully
preserving their communication-hiding properties and thus their scalability.
The basic idea is to efficiently communicate a few redundant copies of local
vector elements to neighboring nodes with very little overhead. In case a node
fails, these redundant copies are gathered at a replacement node, which can
then accurately reconstruct the lost parts of the solver's state. After that,
the parallel solver can continue as in the failure-free scenario. Experimental
evaluations of our approach illustrate on average very low runtime overheads
compared to the standard non-resilient algorithms. This shows that scalable
algorithmic resilience can be achieved at low extra cost.Comment: 12 pages, 2 figures, 2 table