A cooperative conjugate gradient method for linear systems permitting multithread implementation of low complexity

Abstract

This paper proposes a generalization of the conjugate gradient (CG) method used to solve the equation $Ax=b$ for a symmetric positive definite matrix $A$ of large size $n$. The generalization consists of permitting the scalar control parameters (= stepsizes in gradient and conjugate gradient directions) to be replaced by matrices, so that multiple descent and conjugate directions are updated simultaneously. Implementation involves the use of multiple agents or threads and is referred to as cooperative CG (cCG), in which the cooperation between agents resides in the fact that the calculation of each entry of the control parameter matrix now involves information that comes from the other agents. For a sufficiently large dimension $n$, the use of an optimal number of cores gives the result that the multithread implementation has worst case complexity $O(n^{2+1/3})$ in exact arithmetic. Numerical experiments, that illustrate the interest of theoretical results, are carried out on a multicore computer.Comment: Expanded version of manuscript submitted to the IEEE-CDC 2012 (Conference on Decision and Control