Nonlinear CG-like iterative methods

AbstractA nonlinear conjugate gradient method has been introduced and analyzed by J.W. Daniel. This method applies to nonlinear operators with symmetric Jacobians. Orthomin(1) is an iterative method which applies to nonsymmetric and definite linear systems. In this article we generalize Orthomin(1) to a method which applies directly to nonlinear operator equations. Each iteration of the new method requires the solution of a scalar nonlinear equation. Under conditions that the Hessian is uniformly bounded away from zero and the Jacobian is uniformly positive definite the new method is proved to converge to a globally unique solution. Error bounds and local convergence results are also obtained. Numerical experiments on solving nonlinear operator equations arising in the discretization of nonlinear elliptic partial differential equations are presented

On the simplification of generalized conjugate-gradient methods for nonsymmetrizable linear systems

AbstractThe conjugate-gradient (CC) method, developed by Hestenes and Stiefel in 1952, can be effectively used to solve the linear system Au = b when A is symmetrixable in the sense that ZA and Z are symmetric and positive definite (SPD) for some Z. A number of generalizations of the CG method have been proposed by the authors and by others for handling the nonsymmetrizable case. For many problems the amount of computer memory and computational effort required may be so large as to make the procedures not feasible. Truncated schemes are often used, but in some cases the truncated methods may not converge even though the nontruncated schemes converge. However, it is well known that if A is symmetric, the generalized CG schemes can be greatly simplified, even though A is not SPD, so that the truncated schemes are equivalent to the nontruncated schemes. In the present paper it is shown that such a simplification can occur if a nonsingular matrix H is available such that HA = ATH. (Of course, if A = AT, then H can be taken to be the identity matrix.) It is also shown that such an H always exists; however, it may not be practical to compute H. These results are used to derive three variations of the Lanczos method for solving nonsymmetrizable systems. Two of the forms are well known, but the third appears to be new. An argument is given for choosing the third form over the other two

Generalized conjugate-gradient acceleration of nonsymmetrizable iterative methods

AbstractConjugate-gradient acceleration provides a powerful tool for speeding up the convergence of a symmetrizable basic iterative method for solving a large system of linear algebraic equations with a sparse matrix. The object of this paper is to describe three generalizations of conjugate-gradient acceleration which are designed to speed up the convergence of basic iterative methods which are not necessarily symmetrizable. The application of the procedures to some commonly used basic iterative methods is described

A preconditioned Newton-Krylov method for computing steady-state pulse solutions of mode-locked lasers

Thesis (S.M.)--Massachusetts Institute of Technology, Computation for Design and Optimization Program, 2008.Includes bibliographical references (p. 47-48).We solve the periodic boundary value problem for a mode-locked laser cavity using a specially preconditioned matrix-implicit Newton-Krylov solver. Solutions are obtained at least an order of magnitude faster than with dynamic simulation, the standard method. Our method is demonstrated experimentally on a one-dimensional temporal model of an eight femtosecond mode-locked laser operating in the dispersion-managed soliton regime. Our solver is applicable to finding the steady-state solution of any nonlinear optical cavity with moderate self phase modulation, such as those of solid state lasers, and requires only a model for the round-trip action of the cavity. We conclude by proposing avenues of future work to improve the method's convergence and expand its applicability to lasers with higher degrees of cavity nonlinearity. Our approach can be extended to spatio-temporal cavity models, potentially allowing for the first feasible simulation of the full dynamics of Kerr-lens mode locking.by Jonathan R. Birge.S.M

Linear iterative solvers for implicit ODE methods

The numerical solution of stiff initial value problems, which lead to the problem of solving large systems of mildly nonlinear equations are considered. For many problems derived from engineering and science, a solution is possible only with methods derived from iterative linear equation solvers. A common approach to solving the nonlinear equations is to employ an approximate solution obtained from an explicit method. The error is examined to determine how it is distributed among the stiff and non-stiff components, which bears on the choice of an iterative method. The conclusion is that error is (roughly) uniformly distributed, a fact that suggests the Chebyshev method (and the accompanying Manteuffel adaptive parameter algorithm). This method is described, also commenting on Richardson's method and its advantages for large problems. Richardson's method and the Chebyshev method with the Mantueffel algorithm are applied to the solution of the nonlinear equations by Newton's method

Breakdowns in the implementation of the Lánczos method for solving linear systems

AbstractThe Lánczos method for solving systems of linear equations is based on formal orthogonal polynomials. Its implementation is realized via some recurrence relationships between polynomials of a family of orthogonal polynomials or between those of two adjacent families of orthogonal polynomials. A division by zero can occur in such recurrence relations, thus causing a breakdown in the algorithm which has to be stopped. In this paper, two types of breakdowns are discussed. The true breakdowns which are due to the nonexistence of some polynomials and the ghost breakdowns which are due to the recurrence relationship used. Among all the recurrence relationships which can be used and all the algorithms for implementing the Lánczos method which came out from them, the only reliable algorithm is Lánczos/Orthodir which can only suffer from true breakdowns. It is shown how to avoid true breakdowns in this algorithm. Other algorithms are also discussed and the case of near-breakdown is treated. The same treatment applies to other methods related to Lánczos'

NLTGCR: A class of Nonlinear Acceleration Procedures based on Conjugate Residuals

This paper develops a new class of nonlinear acceleration algorithms based on extending conjugate residual-type procedures from linear to nonlinear equations. The main algorithm has strong similarities with Anderson acceleration as well as with inexact Newton methods - depending on which variant is implemented. We prove theoretically and verify experimentally, on a variety of problems from simulation experiments to deep learning applications, that our method is a powerful accelerated iterative algorithm.Comment: Under Revie