In the context of solving nonlinear equations, the term  affine invariance  was introduced to describe the fact that when a function F: Rn → Rn is transformed to G = AF ,where A is an invertible matrix, then the equation F(x) = 0 has the same solutions as G(x) = 0, and the Newton iterates Xk+1 = Xk-F\u27(Xk)-1F(Xk) remain unchanged when F is replaced by G. The idea was that this property of Newton\u27s method should be reflected in its convergence analysis and practical implementation, not only on aesthetic grounds but also because the resulting algorithms would likely be less sensitive to scaling, conditioning, and other numerical issues

Ypma, Tjalling

English

Western Washington University

Western Washington UniversityWestern CEDARMathematics College of Science and Engineering2005Review of: Newton Methods for NonlinearProblems: Affine Invariance and AdaptiveAlgorithms, by P. DeuflhardTjalling YpmaWestern Washington University, tjalling.ypma@wwu.eduFollow this and additional works at: https://cedar.wwu.edu/math_facpubsPart of the Mathematics CommonsThis Book Review is brought to you for free and open access by the College of Science and Engineering at Western CEDAR. It has been accepted forinclusion in Mathematics by an authorized administrator of Western CEDAR. For more information, please contact westerncedar@wwu.edu.Recommended CitationYpma, Tjalling, "Review of: Newton Methods for Nonlinear Problems: Affine Invariance and Adaptive Algorithms, by P. Deuflhard"(2005). Mathematics. 94.https://cedar.wwu.edu/math_facpubs/94  Society for Industrial and Applied Mathematics is collaborating with JSTOR to digitize, preserve and extend access to SIAM Review.http://www.jstor.orgSociety for Industrial and Applied MathematicsReview Author(s): Tjalling Ypma Review by: Tjalling Ypma Source:   SIAM Review, Vol. 47, No. 2 (Jun., 2005), pp. 401-403Published by:  Society for Industrial and Applied MathematicsStable URL:  http://www.jstor.org/stable/20453655Accessed: 21-10-2015 17:10 UTCYour use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at http://www.jstor.org/page/ info/about/policies/terms.jspJSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org.This content downloaded from 140.160.178.72 on Wed, 21 Oct 2015 17:10:22 UTCAll use subject to JSTOR Terms and ConditionsBOOK REVIEWS 401 written as the phase plane system (4) df _ 9 d< a'(f)' dg _ (a + b(f ))g + c(f - a,'(f ) cf) Then the corresponding first-order ODE for integral curves is (5) dg9=f + b' (f) - c(f )a(f ) df g Finally, integration yields the integral equa tion for g(f) that lies at the heart of this book, (6) g(f) = af + b(f) - j c(r)(r) dr. The authors provide the necessary back ground for the analysis of integral equations of this form. The benefits of this interesting line of attack are well illustrated through the many examples of (1) that would be prob lematic studied otherwise. What follows is an extensive presentation of proofs of results on existence, uniqueness, boundedness, so lutions with bounded support, and depen dence on the wavespeed in terms of proper ties of a(u), b(u), c(u). The book has a rea sonable balance between proofs and more specific applications and examples. It also provides physical and historical contexts for the problems with extensive bibliographic notes. The most global results for (1) are presented in the earlier chapters; later on, things are subdivided into more specific re sults for the cases of convection-diffusion and reaction-diffusion problems. The book concludes with a chapter detailing special cases of (1) with known closed-form explicit solutions. I found this book to be a very nice addi tion to Birkhauser's Progress in Nonlinear Differential Equations series, and I believe it will be a very useful resource for any one studying traveling waves in nonlinear parabolic equations. Its focus is limited, but within its scope, its presentation is encyclo pedic. It has no results on stability, rates of convergence and long time asymptotic be havior, or problems in multiple dimensions, but the results on traveling waves it does present can be used as the building blocks for further research in all of these areas. Hence I see this book as a valuable refer ence for most problems relating to PDEs such as (1). REFERENCES [1] B. H. GILDING AND R. KERSNER, The char acterization of reaction-convection diffusion processes by travelling waves, J. Differential Equations, 124 (1996), pp. 27-79. [2] A. A. SAMARSKII, V. A. GALAKTIONOV, S. P. KURDYUMOV, AND A. P. MIKHAILOV, Blow-up in Quasilinear Parabolic Equations, de Gruyter Exp. Math. 19, Walter de Gruyter, Berlin, 1995. [3] A. I. VOLPERT, V. A. VOLPERT, AND V. A. VOLPERT, Traveling Wave Solutions of Parabolic Systems, Transl. Math. Monogr. 140, AMS, Providence, RI, 1994. THOMAS P. WITELSKI Duke University Newton Methods for Nonlinear Prob lems: Affine Invariance and Adaptive Al gorithms. By P Deuflhard. Springer-Verlag, Berlin, 2004. $99.00. xii+424 pp., hardcover. ISBN 3-540-21099-7. In the context of solving nonlinear equa tions, the term "affine invariance" was in troduced to describe the fact that when a function F: Rn -+ R' is transformed to G = AF , where A is an invertible matrix, then the equation F(x) = 0 has the same solutions as G(x) = 0, and the Newton iter atesXk+1 = Xk-Ft (Xk)-1F(Xk) remain un changed when F is replaced by G. The idea was that this property of Newton's method should be reflected in its convergence anal ysis and practical implementation, not only on aesthetic grounds but also because the resulting algorithms would likely be less sensitive to scaling, conditioning, and other numerical issues. This simple idea has been expanded by Peter Deuflhard and his coworkers to em brace at least four different forms of in variance, pertaining to three different con texts. The invariance mentioned above is now termed affine covariance, while meth ods and analyses for the same problem This content downloaded from 140.160.178.72 on Wed, 21 Oct 2015 17:10:22 UTCAll use subject to JSTOR Terms and Conditions402 BOOK REVIEWS class that are independent of the choice of B under the transformation x = By and G(y) = F(By) are termed affine con travariant. For the class of unconstrained optimization problems min f (x), setting x = By and g(y) = f(By) results in a func tion g whose Hessian matrix g" (y) is related to the Hessian matrix f" (x) of f through the relation g"(y) = BTf"(x)B; hence invari ance under this transformation is termed affine conjugacy. Finally, for the dynam ical system x' = F(x), setting x = By gives y' = G(y) = B-1F(By), for which G'(y) = B-1F'(x)B; the corresponding in variance is thus termed affine similarity. This book focuses on providing conver gence analyses of various standard numeri cal methods for these problem classes, with both the formulation of the methods and the analysis couched in the appropriate in variant format. This analysis is then used to develop invariant computational estimates of quantities which can be used to mon itor the convergence behavior of the cor responding algorithm. A typical example is the analysis of Newton's method itself. First an affine covariant form of the clas sic Newton-Mysovskikh theorem is proved, characterizing the convergence through an invariant majorizing sequence. Then prop erties of some invariant computable esti mates of terms of this majorizing sequence are derived. These quantities are then in corporated as convergence monitors and termination criteria in an affine covariant implementation of Newton's method. The book is broad in scope, covering a wide range of topics generally associated with the solution of nonlinear equations. This includes the special situations aris ing in the context of stiff differential equa tions and boundary value problems. For example, the extensive portion of the book that centers on the use of Newton's method and its variants for solving nonlinear equa tions includes an analysis of inexact Newton methods and quasi-Newton methods, while damped Newton methods, trust region al gorithms, and continuation methods are among the more widely convergent classes of methods analyzed. The analysis extends to the invariant computational control of the iterative linear equation solvers (such as GMRES and PCG) used within the context of the nonlinear equation solver. The word "adaptive" in the title of the book refers to the invariant adjustment of steplengths and grids in the context of solving differen tial equations. Arguably the scope of the book is too wide, with the sheer volume of material demanding sometimes very terse presentations and cross-referencing in the quest for brevity, at the expense of clarity and readability. The invariant (re)formulations of various standard numerical methods, and the corre sponding (re)statements of the related con vergence theorems, are an attractive feature of this book, not least because they pro vide alternative perspectives on an other wise classic landscape. These features also sometimes permit derivations of standard results that are somewhat more general and transparent than usual. The analysis of in variant computable convergence quantities is also of considerable value, and many of the results are quite readily implementable. An interesting feature of the analysis is what the author calls "bit counting lemmas," in which properties of these computable quan tities are derived on the basis of the relative accuracy of a priori or a posteriori estimates of other underlying theoretical quantities. The term "bit counting" refers to the fact that these analyses cover even the case when there is only one correct bit in the estimate. This book is certainly not an easy read. Much of it is inevitably devoted to the often tedious pursuit of algebraic inequalities. While the algebra is sometimes enlivened by some deft trickery, there is cause to be grateful that the painful manipulation has been done by someone else. A considerable amount of detective work, and occasionally frustration, is involved in piecing together some of the arguments, since the notation is not always clearly defined and the many backward references are sometimes obscure. Occasionally awkward phrasing makes some statements hard to interpret, and the book is not error-free. To a large extent this book is an organized compendium and synthesis of the work of the author and his students and coworkers over the last 30 years. The references go back to the author's doctoral dissertation dating from 1972, and a significant per centage of the 200-odd references relate to This content downloaded from 140.160.178.72 on Wed, 21 Oct 2015 17:10:22 UTCAll use subject to JSTOR Terms and ConditionsBOOK REVIEWS 403 publications of the author and co-workers. A number of these publications are tech nical reports or doctoral theses, which this book thus makes more accessible and places within a broader context. Unfortunately this book will not free one from the need to consult these references: as the complex ity of the material increases, so does the sketchiness of the discussion, and a good deal of the last part of the book is only partially intelligible without access to the original publications. A considerable amount of background knowledge is required to appreciate much of the content of this book. Such a back ground would certainly have to include at least a passing acquaintance with most of the content of the introductory texts by Kelley [3, 4, 5], including some knowledge of iterative linear equation solvers based on Krylov subspaces. A stronger background would include the classic text of Dennis and Schnabel [1], Greenbaum's introduc tion to Krylov subspace methods [2], and some elements of the standard reference by Ortega and Rheinboldt [6]. To the extent that this book can be regarded as a text it is restricted to an advanced audience, or at least an audience with an experienced guide. The relatively few exercises at the end of each chapter are generally not for the faint-hearted. The book is thus primarily a research monograph, but it is also useful as a reference from which one can scavenge good ideas and interesting theoretical ap proaches, as well as providing readily imple mentable techniques with firm theoretical foundations to replace the ad hoc devices so often incorporated into numerical algo rithms. It is a pity that these virtues are not supported by a clearer exposition. REFERENCES [1] J. E. DENNIS, JR., AND R. B. SCHN ABEL, Numerical Methods for Uncon strained Optimization and Nonlinear Equations, Prentice-Hall, Englewood Cliffs, NJ, 1983; reprinted as Classics Appl. Math. 16, SIAM, Philadelphia, 1996. [2] A. GREENBAUM, Iterative Methods for Solv ing Linear Systems, SIAM, Philadel phia, 1997. [3] C. T. KELLEY, Iterative Methods for Lin ear and Nonlinear Equations, SIAM, Philadelphia, 1995. [4] C. T. KELLEY, Iterative Methods for Opti mization, SIAM, Philadelphia, 1999. [5] C. T. KELLEY, Solving Nonlinear Equa tions with Newton's Method, SIAM, Philadelphia, 2003. [6] J. M. ORTEGA AND W. C. RHEINBOLDT, Iterative Solution of Nonlinear Equa tions in Several Variables, Academic Press, New York, 1970; reprinted as Classics Appl. Math. 30, SIAM, Philadelphia, 2000. TJALLING YPMA Western Washington University This content downloaded from 140.160.178.72 on Wed, 21 Oct 2015 17:10:22 UTCAll use subject to JSTOR Terms and Conditions

Review of: Newton Methods for Nonlinear Problems: Affine Invariance and Adaptive Algorithms, by P. Deuflhard

https://cedar.wwu.edu/cgi/viewcontent.cgi?article=1092&amp;context=math_facpubs

Review of: Newton Methods for Nonlinear Problems: Affine Invariance and Adaptive Algorithms, by P. Deuflhard

Abstract

Similar works

Full text

Available Versions

Western Washington University