Analysis of nonsymmetric matrix iterations based on eigenvalues can be misleading. In this paper, we discuss sixteen theorems involving $\epsilon$-pseudospectra that each generalize a familiar eigenvalue theorem and may provide more descriptive information in some cases. Our organizing principle is that each pseudospectral theorem reduces precisely to the corresponding eigenvalue theorem when $\epsilon$ = 0.\ud
\ud
This work was supported by UK Engineering and Physical Sciences Research Council Grant GR/M12414

Embree, Mark

Trefethen, Lloyd N.

Hungarian

Analysis of nonsymmetric matrix iterations based on eigenvalues can be misleading. In this paper, we discuss sixteen theorems involving $\epsilon$-pseudospectra that each generalize a familiar eigenvalue theorem and may provide more descriptive information in some cases. Our organizing principle is that each pseudospectral theorem reduces precisely to the corresponding eigenvalue theorem when $\epsilon$ = 0.  This work was supported by UK Engineering and Physical Sciences Research Council Grant GR/M12414

ORA - Oxford University Research Archive

Generalizing Eigenvalue Theorems to Pseudospectra Theorems

Oxford University Research Archive

GENERALIZING EIGENVALUE THEOREMSTO PSEUDOSPECTRA THEOREMSMARK EMBREEy AND LLOYD N. TREFETHENyAbstrat. Analysis of nonsymmetri matrix iterations based on eigenvalues an be misleading.In this paper, we disuss sixteen theorems involving "-pseudospetra that eah generalize a familiareigenvalue theorem and may provide more desriptive information in some ases. Our organizingpriniple is that eah pseudospetral theorem redues preisely to the orresponding eigenvalue the-orem when " = 0.Key words. Eigenvalues, pseudospetra, matrix iterationsAMS subjet lassiations. 15A18, 15A60, 65F10, 65F151. Introdution. Though we speak of linear algebra, iterative methods belongto the realm of linear analysis. Convergene of errors or residuals to zero is theonern, and this proess has meaning beause the algebrai problem is embedded ina normed spae. Exept for questions onerning nite termination, the appropriatetools for analyzing onvergene are not the tools of algebra, suh as eigenvalues, whihare basis-independent, but those of analysis, suh as singular values, whih are denedvia norms and neessarily hange with the basis.In this paper we onsider the partiular tools of linear analysis known as pseu-dospetra, whih were invented to give information about matries that lak a well-onditioned basis of eigenvetors. For simpliity our norm k  k will always be thevetor 2-norm and the matrix 2-norm that it indues. With this hoie of norm, thematries of interest are those that are far from normal, in the sense that their eigen-vetors, if a omplete set exists, are far from orthogonal. Many of our results an beextended to other norms, and also to operators as well as matries, but we will notdisuss these generalizations.Throughout the artile, A is an N N matrix, and (A) denotes its spetrum,i.e., its set of eigenvalues, a subset of the omplex plane C . The pseudospetra of Aare nested subsets of C that expand to ll the plane as "!1.Definition. For eah "  0, the "-pseudospetrum "(A) of A is the set ofnumbers z 2 C satisfying any of the equivalent onditions:(i) k(z  A) 1k  " 1;(ii) min(z  A)  ";(iii) kAu  zuk  " for some vetor u with kuk = 1;(iv) z is an eigenvalue of A+E for some matrix E with kEk  ".Here min denotes the smallest singular value, and we employ the onvention thatk(z  A) 1k =1 for z 2 (A).Pseudospetra were introdued as early as 1975 [11℄ and beame a popular toolduring the 1990s. We will not give detailed referenes here but refer the reader to [20℄and [21℄ for examples, to [22℄ for algorithms and a list of appliations, and to [23℄ forhistory. For extensive online information about pseudospetra, inluding examplesand a bibliography of papers by many authors, see the Web site [4℄.This brief artile is devoted to a simple idea:yOxford University Computing Laboratory, Wolfson Building, Parks Road, Oxford OX1 3QD,United Kingdom (embreeomlab.ox.a.uk, LNTomlab.ox.a.uk). This work was supported byUK Engineering and Physial Sienes Researh Counil Grant GR/M12414.12 MARK EMBREE AND LLOYD N. TREFETHENMany theorems about eigenvalues are speial ases " = 0 of theorems about"-pseudospetra.Our whole ontent onsists of the presentation of sixteen examples of theorems of thiskind. These theorems are for the most part neither mathematially deep nor evennew, though in some ases they have not been stated in the language of pseudospetrabefore. Nevertheless, for pratial appliations involving highly non-normal matries,they may sometimes be more useful than their eigenvalue speial ases. This will tendto be so in situations where the eigenvalues of A are misleading, lling a region of Csmaller than where A atually \lives." For an example illustrating the limitations ofeigenvalue analysis for Krylov subspae methods for linear systems, see [7℄. Considerthe following simpler but extreme example. If A is nilpotent, with AK = 0 for someK  1, then (A) = f0g. Some suh matries will have norms kAkk that diminishsteadily toward 0 as k ! K, while for others, there may be no redution until k = K orgreat transient growth before the eventual deay. Eigenvalues alone annot distinguishbetween these behaviors, but pseudospetra an.Our presentation will adhere to a xed pattern. In eah ase, we rst list atheorem about eigenvalues, without proof, that is either elementary or well known.We follow this with a generalized theorem for pseudospetra together with an outlineof a proof. Some pointers to the literature are inluded along the way, but we do notaim to be exhaustive, as it is often hard with this essentially elementary material totrak down the rst appearane of a result in print.We hope that this artile may provide a useful ompendium for those onernedwith non-normal matries and assoiated iterations, but we emphasize that this ol-letion does not inlude all potentially useful theorems involving pseudospetra. Byonning our attention to theorems that redue for " = 0 to valid statements abouteigenvalues, we exlude some of the subtler estimates that may be obtained frompseudospetra, notably those based on ontour integrals. One example is the KreissMatrix Theorem, whih ontains a onstant eN that does not redue leanly to 1 as" ! 0 [10, 17℄. Another is the bound on a polynomial norm kp(A)k, of immediaterelevane to iterations suh as GMRES, that an be obtained by integrating p(z)over the boundary ontour(s) of "(A) [19℄. For new results omparing suh ontourintegral tehniques to other approahes, see [6℄.2. Sixteen Theorems. Our rst theorem states that ill-onditioning is equiv-alent to the existene of a pseudoeigenvalue near the origin. The result has beenattributed to Gastinel (see [18, pp. 120, 133℄).Theorem 1. A is singular () 0 2 (A):Theorem 1". kA 1k  " 1 () 0 2 "(A):Proof. Immediate from the denitions.Pseudospetra possess the satisfying property that every onneted omponent ofthe "-pseudospetrum must ontain at least one eigenvalue. This property forms thebasis for the following result.Theorem 2. A has N distint eigenvalues =) A is diagonalizable.Theorem 2". "(A) has N distint omponents =) A is diagonalizable.Proof. The funtion log k(z A) 1k is subharmoni, whih implies that it satisesa maximum priniple, and therefore so does the resolvent norm k(z   A) 1k. Thisimplies that eah omponent of "(A) ontains at least one eigenvalue of A.PSEUDOSPECTRA THEOREMS 3Gallestey has developed an algorithm for omputing pseudospetra based on themaximum priniple property used in the above proof [5℄. A simpler exlusion algo-rithm, reently proposed by Gallopoulos, is based upon the next result [2℄.Theorem 3. k(z  A) 1k  1dist(z;(A)) .Theorem 3". k(z  A) 1k  1dist(z;"(A)) + " .Proof. A perturbation of A of norm dist(z;"(A))+" ould make z an eigenvalue.Gallopoulos's algorithm utilizes Theorem 3" rewritten in the formdist(z;"(A))  1k(z  A) 1k   ":In our next theorem, S is an arbitrary nonsingular matrix and (S) is its onditionnumber, (S)  kSkkS 1k. Though the theorem is stated as an inlusion in onediretion only, it applies in the other diretion too, and in that sense Theorem 4maintains our usual pattern of being the speial ase " = 0 of Theorem 4". The resultdemonstrates that pseudospetra are invariant under unitary transformations, andalso reets the extent to whih an ill-onditioned similarity transformation an alterpseudospetra. When SBS 1 is a diagonalization of A, Theorem 4" is equivalent tothe most familiar version of the Bauer{Fike Theorem [1℄.Theorem 4. A = SBS 1 =) (A) = (B).Theorem 4". A = SBS 1 =) "(A)  (S)"(B).Proof. Sine (z   A) 1 = S(z   B) 1S 1, k(z   A) 1k  (S)k(z   B) 1k.Therefore if k(z  A) 1k  " 1, then k(z  B) 1k  ((S)") 1.The following theorem makes use of the idea of the \average pseudoeigenvalue"of a matrix, mean"2"(A)". Of ourse, this quantity needs to be dened. We ouldbe very spei and make use of, say, Haar measure (isotropy in C N ) on the spaeof N  N matries, but for the purposes of this theorem it is enough to say thatmean"2"(A)" is the mean of the eigenvalues of A + E averaged over any xeddistribution on the matries E with kEk  " with the property that eah matrixentry eij has mean 0.Theorem 5. tr(A) = N mean2(A).Theorem 5". tr(A) = N mean"2"(A)".Proof. The theorem looks deep but is elementary. All we need to do is onsidertraes of perturbed matries. Sine eah ejj has mean 0 by assumption, so does theirsum, and thus tr(A) = meankEk"tr(A+E) = N mean"2"(A)".Our next pair of results require a denition of the ondition number A()of a set  = (A) depending on A with respet to perturbations of A. If 1and 2 are generi sets, let d(1;2) denote the Hausdor distane, d(1;2) =maxfmaxs21 d(s;2);maxs22 d(s;1)g, where d(s;) is the usual distane of a point s to aset . Then A()  lim sup"!0 " 1 supkEk=" d (A+E);(A):4 MARK EMBREE AND LLOYD N. TREFETHENTheorem 6. (A) depends ontinuously on A, with ondition number 1 if A isnormal.Theorem 6". "(A) depends ontinuously on A, with ondition number 1 if Ais normal.Proof. This follows from the denitions of "(A) and the ondition number,together with the ontinuity of eigenvalues with respet to perturbations of the entries.Eigenvalues an hange dramatially with small perturbations, a warning thatanalysis based on them an be misleading. The following theorem hints that pseu-dospetra may be more robust.Theorem 7. (A+E)  kEk(A).Theorem 7". "(A+E)  "+kEk(A).Proof. If z 2 "(A + E), then there exists a matrix F with kFk  " suh that(A+E + F )u = zu for some u 6= 0. Sine kE + Fk  "+ kEk, z 2 "+kEk(A).We now turn to the problems of estimating the behavior of a matrix from itsspetra and pseudospetra.Theorem 8.  2 (A) =) kAk  jj.Theorem 8". " 2 "(A) =) kAk  j"j   ".Proof. If " 2 "(A), then Au = "u + "v for some vetors u; v 2 C withkuk = kvk = 1. It follows that kAuk  j"j   ".The onvergene analysis of stationary iterative methods is based on the behaviorof powers of the iteration matrix. It has long been known that transient growth anour even when the spetral radius of the iteration matrix is less than one (see,e.g., [24℄). The following two theorems use pseudospetra to desribe this transientgrowth. The rst is the \easy half of the Kreiss matrix theorem," that is, the half ofthat theorem that does not depend on N and whose proof is elementary [10℄.Theorem 9. max2(A) jj > 1 =) supk>0 kAkk =1.Theorem 9". max"2"(A) j"j > 1 + C" =) supk0 kAkk > C.Proof. Sine kA0k = 1, the result is trivial for C < 1, so assume C  1. Ifmax2(A) jj > 1, then the onlusion ertainly holds, so assume max2(A) jj  1,in whih ase we have the onvergent series representation(z  A) 1 = z 1(I + z 1A+ z 2A2 +   );valid for all z with jzj > 1. We now argue the ontrapositive. If kAkk  C for allk  0, then k(z  A) 1k  jz 1jC1  jz 1j = Cjzj   1for any z with jzj > 1. This implies that "(A) is ontained in the disk about theorigin of radius 1 + C", i.e., max"2"(A) j"j  1 + C".Theorem 10.  2 (A) =) kAkk  jjk for all k.Theorem 10". " 2 "(A) =) kAkk  j"jk   k"kAkk 11  k"=kAk for all k suh thatk" < kAk.PSEUDOSPECTRA THEOREMS 5Proof. Pik E suh that kEk  " and " 2 (A+E). Then k(A+ E)kk  j"jk,whih implieskAkk  j"jk   k"kAkk 1  k2"2kAkk 2      j"jk   k"kAkk 1  1 + k"=kAk+ (k")2=kAk2 +    :Provided k" < kAk, the series in this last equation onverges, givingkAkk  j"jk   k"kAkk 11  k"=kAk :Theorem 9" has an exat analogue for ontinuous time (see [12, 13℄).Theorem 11. max2(A)Re > 0 =) supt>0 ketAk =1.Theorem 11". max"2"(A)Re" > C" =) supt>0 ketAk > C.Proof. As in the proof of Theorem 9", the onlusion is immediate if C < 1 ormax2(A)Re > 0, so we assume that C  1 and max2(A)Re  0 and use theLaplae transform identity (z  A) 1 = Z 10 e ztetAdt;valid for Rez > 0. Again arguing the ontrapositive, we note that if ketAk  C forall t > 0, then k(z   A) 1k  C=Rez for z with Rez > 0, implying that "(A) isontained in the half-plane dened by Rez  C".Our next result is a pseudospetral generalization of Gershgorin's theorem, whihwe believe to be new. It implies that if "(A) ontains points distant from (A) forsuÆiently small ", then the eigenvalue bounds given by Gershgorin's theorem willbe more desriptive of the pseudospetra than the spetrum. Coupling this withTheorems 9" and 10", one sees that Gershgorin eigenvalue bounds an potentiallysuggest a better desription of transient behavior of iterative matrix proesses thanwould be obtained from the eigenvalues themselves. For these theorems, dene dj =ajj and rj = Pk 6=j jajk j, and for any omplex number z and real number r  0, letD(z; r) denote the losed disk about z of radius r.Theorem 12. (A)  [j D(dj ; rj).Theorem 12". "(A)  [j D(dj ; rj + "N).Proof. A perturbation E of A with kEk  " must satisfy jeij j  " for eah i; j.Therefore A+ E satises the same Gershgorin bounds as A, exept with the enterof eah disk moved by at most " and the radius of eah disk inreased by at most(N   1)".The next result onerns the numerial range or eld of values, whih we denoteby W (A). In the ontext of iterative methods, the theorem indiates how analysisbased on the eld of values (see, e.g., [3℄) relates to pseudospetral analysis. We6 MARK EMBREE AND LLOYD N. TREFETHENwrite onv(S) for the onvex hull in C of a set S  C . The notation \S n "-border"also requires some explanation. By this we mean the set of points z 2 C suh thatD(z; ")  S. Perhaps Reddy, Shmid, and Henningson were the rst to formulate thisresult in the language of pseudospetra [13, Thm. 2.2℄.Theorem 13. W (A)  onv((A)).Theorem 13". W (A)  onv("(A)) n "-border.Proof. This result follows from a familiar result in funtional analysis: thatW (A)is the intersetions of all onvex sets S that satisfy the onditionk(z  A) 1k  1dist(z; S) :See, for example, Kato [9, p. 268℄.The spetral mapping theorem (see, e.g., [9, p. 45℄) is a jewel in the rown ofeigenvalue theorems; it is theoretially appealing and pratially relevant, formingthe basis for rational transformation tehniques for omputing eigenvalues. The nu-merial range obeys a similar, though one-sided, mapping theorem [8℄. Theorems 13and 13" suggest that a similar result might hold for pseudospetra. Our next theoremis a modest step in this diretion, a preise mapping theorem for linear transforma-tions [23, Thm. 2.4℄.Theorem 14. (+ A) = + (A) for ;  2 C .Theorem 14". "jj(+ A) = + "(A) for ;  2 C .Proof. The result is trivial when  = 0. Otherwise, note thatjj k(z   ( + A)) 1k = k( 1(z   )  A) 1k:For Theorems 15 and 16, let V denote an Nk retangular matrix with orthonor-mal olumns, for some k  N , as might be obtained in Arnoldi or subspae iteration,and let H denote a k  k square matrix. In the Arnoldi iteration, H would haveHessenberg form, but this is not neessary for these theorems. First, we assume thatthe olumns of V exatly span an invariant subspae of A. The resulting theoremforms the basis for algorithms that ompute pseudospetra by projeting A onto aarefully hosen invariant subspae [13, 22℄.Theorem 15. AV = VH =) (H)  (A).Theorem 15". AV = VH =) "(H)  "(A).Proof. If kHu  zuk  " for some u 2 C N with kuk = 1, then kV Hu  V zuk  "too, and this implies kAV u  zV uk  ".Pratial algorithms suh as the impliitly restarted Arnoldi method [16℄ or sub-spae iteration (see, e.g., [14, xV.1℄) rarely yield an exat basis for the invariantsubspae. Rather, the olumns of V form an orthonormal basis for some approximateinvariant subspae of A. Let H denote the generalized Rayleigh quotient this basisforms, H  V AV . With this notation, eigenvalue Theorem 15 has an alternative,more pratial pseudospetral generalization.Theorem 16. AV = VH =) (H)  (A).Theorem 16". AV = VH +R =) (H)  "(A) for " = kRk.Proof. Consider the square matrix E =  RV . Then (A + E)V = AV   R =VH , so by Theorem 15, the eigenvalues of H are eigenvalues of A + E and hene"-pseudoeigenvalues of A for " = k RV k = kRk.PSEUDOSPECTRA THEOREMS 7For an Arnoldi fatorization with k basis vetors, V 2 C nk, Theorem 16" reduesto a well known result: " = kRk = jhk+1;k j, where hk+1;k is the (k+1; k) entry in theextended upper Hessenberg matrix (see, e.g., [15, Lem. 2.1℄).Aknowledgment. We thank Anne Greenbaum for arefully reading an earlierversion of this manusript and pointing out several orretions.REFERENCES[1℄ F. L. Bauer and C. T. Fike, Norms and exlusion theorems, Numer. Math., 2 (1960), pp. 137{141.[2℄ C. Bekas, E. Gallopoulos, and V. Simonini, On the omputational eetiveness of trans-fer funtion approximations to the matrix pseudospetrum. Presentation at the CopperMountain Conferene on Iterative Methods, April 2000.[3℄ M. Eiermann, Fields of values and iterative methods, Lin. Alg. Applis., 180 (1993), pp. 167{197.[4℄ M. Embree, Psuedospetra of matries and operators. Web site to be launhed, 2000.[5℄ E. Gallestey, Computing spetral value sets using the subharmoniity of the norm of rationalmatries, BIT, 38 (1998), pp. 22{33.[6℄ A. Greenbaum, Using the Cauhy integral formula and the partial frations deomposition ofthe resolvent to estimate kf(A)k. Manusript, 2000.[7℄ A. Greenbaum, V. Ptak, and Z. Strakos, Any noninreasing onvergene urve is possiblefor GMRES, SIAM J. Matrix Anal. Appl., 17 (1996), pp. 465{469.[8℄ T. Kato, Some mapping theorems for the numerial range, Pro. Japan Aad., 41 (1965),pp. 652{655.[9℄ , Perturbation Theory for Linear Operators, Springer-Verlag, New York, 2nd ed., 1980.[10℄ H.-O. Kreiss, Uber die Stabilitatsdenition fur Dierenzengleihungen die partielle Dieren-tialgleihungen approximieren, BIT, 2 (1962), pp. 153{181.[11℄ H. J. Landau, On Szeg}o's eigenvalue distribution theory and non-Hermitian kernels,J. d'Analyse Math., 28 (1975), pp. 335{357.[12℄ A. Pazy, Seimgroups of Linear Operators and Appliations to Partial Dierential Equations,Springer-Verlag, New York, 1983.[13℄ S. C. Reddy, P. J. Shmid, and D. S. Henningson, Pseudospetra of the Orr{Sommerfeldoperator, SIAM J. Appl. Math., 53 (1993), pp. 15{47.[14℄ Y. Saad, Numerial Methods for Large Eigenvalue Problems, Manhester University Press,Manhester, 1992.[15℄ V. Simonini and E. Gallopoulos, Transfer funtions and resolvent norm approximation oflarge matries, Elet. Trans. Numer. Anal., 7 (1998), pp. 190{201.[16℄ D. C. Sorensen, Impliit appliation of polynomial lters in a k-step Arnoldi method, SIAMJ. Matrix Anal. Appl., 13 (1992), pp. 357{385.[17℄ M. N. Spijker, On a onjeture of LeVeque and Trefethen related to the Kreiss matrix theorem,BIT, 31 (1991), pp. 551{555.[18℄ G. W. Stewart and J. Sun, Matrix Perturbation Theory, Aademi Press, Boston, 1990.[19℄ L. N. Trefethen, Approximation theory and numerial linear algebra, in Algorithms for Ap-proximation II, J. C. Mason and M. G. Cox, eds., London, 1990, Chapman and Hall.[20℄ , Pseudospetra of matries, in Numerial Analysis 1991, D. F. GriÆths and G. A. Wat-son, eds., Harlow, Essex, UK, 1992, Longman Sienti and Tehnial, pp. 234{266.[21℄ , Pseudospetra of linear operators, SIAM Review, 39 (1997), pp. 383{406.[22℄ , Computation of pseudospetra, Ata Numeria, (1999), pp. 247{295.[23℄ , Spetra and pseudospetra: The behavior of non-normal matries and operators, inThe Graduate Student's Guide to Numerial Analysis, M. Ainsworth, J. Levesley, andM. Marletta, eds., Springer-Verlag, Berlin, 1999, pp. 217{250.[24℄ R. S. Varga, Matrix Iterative Analysis, Prentie{Hall, Englewood Clis, N.J., 1962.

English

Analysis of nonsymmetric matrix iterations based on eigenvalues can be misleading. In this paper, we discuss sixteen theorems involving $\epsilon$-pseudospectra that each generalize a familiar eigenvalue theorem and may provide more descriptive information in some cases. Our organizing principle is that each pseudospectral theorem reduces precisely to the corresponding eigenvalue theorem when $\epsilon$ = 0.

This work was supported by UK Engineering and Physical Sciences Research Council Grant GR/M12414

Generalizing Eigenvalue Theorems to Pseudospectra Theorems

Abstract

Similar works

Full text

Available Versions

ORA - Oxford University Research Archive

Oxford University Research Archive

Oxford University Research Archive (ORA)

Mathematical Institute Eprints Archive