Preconditioning issues in the numerical solution of nonlinear equations and nonlinear least squares

Second order methods for optimization call for the solution of sequences of linear systems. In this survey we will discuss several issues related to the preconditioning of such sequences. Covered topics include both techniques for building updates of factorized preconditioners and quasi-Newton approaches. Sequences of unsymmetric linear systems arising in Newton-Krylov methods will be considered as well as symmetric positive definite sequences arising in the solution of nonlinear least-squares by Truncated Gauss-Newton methods

Preconditioning of linear least squares by robust incomplete factorization for implicitly held normal equations

The efficient solution of the normal equations corresponding to a large sparse linear least squares problem can be extremely challenging. Robust incomplete factorization (RIF) preconditioners represent one approach that has the important feature of computing an incomplete LLT factorization of the normal equations matrix without having to form the normal matrix itself. The right-looking implementation of Benzi and T˚uma has been used in a number of studies but experience as shown that in some cases it can be computationally slow and its memory requirements are not known a priori. Here a new left-looking variant is presented that employs a symbolic preprocessing step to replace the potentially expensive searching through entries of the normal matrix. This involves a directed acyclic graph (DAG) that is computed as the computation proceeds. An inexpensive but effective pruning algorithm is proposed to limit the number of edges in the DAG. Problems arising from practical applications are used to compare the performance of the right-looking approach with a left-looking implementation that computes the normal matrix explicitly and our new implicit DAG-based left-looking variant

Numerically Stable Recurrence Relations for the Communication Hiding Pipelined Conjugate Gradient Method

Pipelined Krylov subspace methods (also referred to as communication-hiding methods) have been proposed in the literature as a scalable alternative to classic Krylov subspace algorithms for iteratively computing the solution to a large linear system in parallel. For symmetric and positive definite system matrices the pipelined Conjugate Gradient method outperforms its classic Conjugate Gradient counterpart on large scale distributed memory hardware by overlapping global communication with essential computations like the matrix-vector product, thus hiding global communication. A well-known drawback of the pipelining technique is the (possibly significant) loss of numerical stability. In this work a numerically stable variant of the pipelined Conjugate Gradient algorithm is presented that avoids the propagation of local rounding errors in the finite precision recurrence relations that construct the Krylov subspace basis. The multi-term recurrence relation for the basis vector is replaced by two-term recurrences, improving stability without increasing the overall computational cost of the algorithm. The proposed modification ensures that the pipelined Conjugate Gradient method is able to attain a highly accurate solution independently of the pipeline length. Numerical experiments demonstrate a combination of excellent parallel performance and improved maximal attainable accuracy for the new pipelined Conjugate Gradient algorithm. This work thus resolves one of the major practical restrictions for the useability of pipelined Krylov subspace methods.Comment: 15 pages, 5 figures, 1 table, 2 algorithm

Spectral two-level preconditioners for sequences of linear systems

De nombreuses simulations numériques nécessitent la résolution d'une série de systèmes linéaires impliquant une même matrice mais des second-membres différents. Des méthodes efficaces pour ce type de problèmes cherchent à tirer bénéfice des résolutions précédentes pour accélérer les résolutions restantes. Deux grandes classes se distinguent dans la façon de procéder: la première vise à réutiliser une partie du sous-espace de Krylov, la deuxième à construire une mise à jour du préconditionneur à partir de vecteurs approximant un espace invariant. Dans cette thèse, nous nous sommes intéressés à cette dernière approche en cherchant à améliorer le préconditionneur d'origine. Dans une première partie, une seule mise à jour du préconditionneur est considérée pour tous les systèmes. Cette mise à jour consiste en une correction spectrale de rang faible qui permet de translater de un la position des plus petites valeurs propres en module de la matrice du système préconditionné de départ. Des expérimentations numériques sont réalisées en utilisant la méthode GMRES couplée à un préconditionneur de type inverse approchée. L'information spectrale est obtenue par un solveur de valeurs propres lors d'une phase préliminaire au calcul. Dans une deuxième partie, on autorise une possible mise à jour entre chaque système. Une correction spectrale incrémentale est proposée. Des expérimentations numériques sont réalisées en utilisant la méthode GMRES-DR, d'une part parce qu'elle est efficace en tant que solveur linéaire, et d'autre part parce qu'elle permet une bonne approximation des petites valeurs propres au cours de la résolution linéaire. Des stratégies sont développées afin de sélectionner l'information spectrale la plus pertinente. Ces approches ont été validées sur des problèmes de grande taille issus de simulations industrielles en électromagnétisme. Dans ce but, elles ont été implantées dans un code parallèle développé par EADS-CCR. ABSTRACT : Many numerical simulations in scientific and engineering applications require the solution of a set of large linear systems involving the same coefficient matrix but different right-hand sides. Efficient methods for tackling this problem attempt to benefit from the previously solved right-hand sides for the solution of the next ones. This goal can be achieved either by recycling Krylov subspaces or by building preconditioner updates based on near invariant subspace information. In this thesis, we focus our attention on this last approach that attempts to improve a selected preconditioner. In the first part, we consider only one update of the preconditioner for all the systems. This update consists of a spectral low-rank correction that shifts by one the smallest eigenvalues in magnitude of the matrix of the original preconditioned system. We perform experiments in the context of the GMRES method preconditioned by an approximate inverse preconditioner. The spectral information is computed by an eigensolver in a preprocessing phase. In the second part, we consider an update of the preconditioner between each system. An incremental spectral correction of the preconditioner is proposed. We perform experiments using the GMRES-DR method, thanks to its efficiency as a linear solver and its ability to recover reliable approximations of the desired eigenpairs at run time. Suitable strategies are investigated for selecting reliable eigenpairs. The efficiency of the proposed approaches is in particular assessed for the solution of large and challenging problems in electromagnetic applications. For this purpose, they have been implemented in a parallel industrial code developed by EADS-CCR