The standard theorem for stochastic matrices with positive entries is generalized to matrices with no sign restriction on the entries. The condition that column sums be equal to 1 is kept, but the positivity condition is replaced by a condition on the distances between columns

Jewett, Robert I.

Ćurgus, Branko

Western Washington University

Western Washington UniversityWestern CEDARMathematics College of Science and Engineering1-2015Somewhat Stochastic MatricesBranko ĆurgusWestern Washington University, branko.curgus@wwu.eduRobert I. JewettWestern Washington UniversityFollow this and additional works at: https://cedar.wwu.edu/math_facpubsPart of the Mathematics CommonsThis Article is brought to you for free and open access by the College of Science and Engineering at Western CEDAR. It has been accepted for inclusionin Mathematics by an authorized administrator of Western CEDAR. For more information, please contact westerncedar@wwu.edu.Recommended CitationĆurgus, Branko and Jewett, Robert I., "Somewhat Stochastic Matrices" (2015). Mathematics. 61.https://cedar.wwu.edu/math_facpubs/61Somewhat Stochastic MatricesBranko Ćurgus and Robert I. JewettAbstract. The standard theorem for stochastic matrices with positive entries is generalizedto matrices with no sign restriction on the entries. The condition that column sums be equalto 1 is kept, but the positivity condition is replaced by a condition on the distances betweencolumns.1. INTRODUCTION. The notion of a Markov chain is ubiquitous in linear algebraand probability books. In linear algebra, a Markov chain is a sequence {xk} of vec-tors defined recursively by a specified vector x0, a square matrix P , and the recursionxk = Pxk−1 for k = 1, 2, . . . . That is, xk = Pkx0. Natural probabilistic restrictions areimposed on x0 and P . It is assumed that x0 is a probability vector, that is, its entriesare nonnegative and add up to 1. It is assumed that P is a stochastic matrix, that is, itis a square matrix whose columns are probability vectors. The original version of themain theorem about Markov chains appears in Markov’s paper [2]. In the language oflinear algebra, it reads:Suppose that P is a stochastic matrix with all positive entries. Then there ex-ists a unique probability vector q such that Pq = q. If {xk} is a Markov chaindetermined by P, then it converges to q.More generally, the same conclusion holds for a stochastic matrix P for whichPs has all positive entries for some positive integer s. All elementary linear algebratextbooks that we examined state this theorem. None give a complete proof. Partialproofs or intuitive explanations of the theorem’s validity are always based on knowl-edge about the matrix’s eigenvalues and eigenvectors. This argument becomes sophis-ticated when the matrix is not diagonalizable.What these proofs leave obscure is a certain contractive property of a stochasticmatrix already observed by Markov. Of course, this contractive property is exploredin research papers and some advanced books. However, the relative simplicity of theunderlying idea gets lost in the technical details of an advanced setting. We feel thatthis contractive property deserves to be popularized. We use it here to provide a directproof of a theorem, which is more general than the one stated above.We consider real square matrices A whose columns add up to 1. Such a matrixwe call a somewhat stochastic matrix. The probabilistic condition that all entries benonnegative is dropped. Instead of assuming that all entries of As are positive, we makean assumption about distances between the columns of As . This assumption leads to acontractive property of a matrix that yields convergence. This and other definitions aregiven next.2. DEFINITIONS. All numbers in this article are real, except in Example 3. Allmatrices are square and will be denoted by uppercase letters. All vectors, except forhttp://dx.doi.org/10.4169/amer.math.monthly.122.01.36MSC: Primary 15B51, Secondary 15B48; 60J1036 c© THE MATHEMATICAL ASSOCIATION OF AMERICA [Monthly 1221, are column vectors and will be denoted by bold lowercase letters. All entries of therow vector 1 are equal to 1,1 = [1 1 · · · 1].This row vector helps to express conditions that were already mentioned and will ap-pear repeatedly. The equation 1A = 1 says that all column sums of A are equal to 1.And 1x = 1 says that the entries of a column vector x sum to 1, while 1x = 0 saysthat the entries of a column vector x sum to 0. We use the standard notation N for theset of positive integers and R for the set of real numbers.For a vector x with entries x1, . . . , xn , we set‖x‖ :=n∑j=1|x j |.Notice that the distance ‖x − y‖ between vectors x and y associated with this norm isthe n-dimensional version of the Manhattan distance.Consider an n × n matrix A with columns a1, . . . , an and entries ai j . For the purposeof the next two definitions, we think of the columns of a matrix as points in Rn . In thisway, the concept of a diameter of a set is applied to a matrix as follows:diam A = max1≤i, j≤n‖ai − a j‖.Next, we definevar A = 12diam A = max1≤i, j≤n12n∑l=1|ali − al j |.We call this quantity the column variation of a matrix A. The idea of using that quantityis due to Markov [2, Section 5]. In [2], for fixed i, j the quantity 12∑nl=1 |ali − al j | isnot given explicitly as half the sum of the absolute values of the real numbers ali − al jbut rather as the sum of the positive terms in this list. Since Markov considered onlystochastic matrices, for which the sum of all terms in this list is 0, the quantity he usedcoincides with the variation. For more on Markov’s work, see [4]. The column and rowvariation appear in research literature under various names; see [1, Section 3.3].Recalling the original theorem about a Markov chain stated in our first paragraph,we will show that the inequality‖Pkx0 − q‖ ≤ (var P)k‖x0 − q‖ (1)holds for all k. Furthermore, for a stochastic matrix P with all positive entries, it turnsout that var P < 1. This strict contractive property of a stochastic matrix with all pos-itive entries implies convergence of the Markov chain {Pkx0}.3. THE COLUMN VARIATION OF A MATRIX. The first step toward a proofof inequality (1) is the proposition that follows. To repeat, our results do not requireentries to be nonnegative. Not only that, but in this proposition A could be a rectangularmatrix with complex entries. However, the assumption that the entries of the vector yare real is essential, as is shown in Example 3.January 2015] SOMEWHAT STOCHASTIC MATRICES 37Proposition. Let A be an n × n real matrix and let y be an n × 1 vector with realentries such that 1y = 0. Then‖Ay‖ ≤ (var A)‖y‖. (2)Proof. We will use the common notation for the positive and negative part of a realnumber t : t+ = max{t, 0} and t− = max{−t, 0}. Clearly, t+, t− ≥ 0, and t = t+ − t−and |t | = t+ + t−.Let A be an n × n matrix with columns a1, . . . , an and let y ∈ Rn be such that1y = 0.The inequality (2) is obvious if y = 0. Assume that y = 0. Set z = (2/‖y‖)y. Then(2) is equivalent to‖Az‖ ≤ (var A)‖z‖ with ‖z‖ = 2. (3)Clearly 1z = 0. Let z1, . . . , zn be the entries of z. Then we have2 = ‖z‖ =n∑j=1|z j | =n∑j=1(z+j + z−j) = n∑j=1z+j +n∑j=1z−jand0 =n∑j=1z j =n∑j=1(z+j − z−j) = n∑j=1z+j −n∑j=1z−j .From the last two displayed relations, we deduce thatn∑j=1z+k =n∑j=1z−k = 1. (4)Using again the notation introduced at the beginning of the proof, we getAz =n∑j=1z j a j =n∑j=1z+j a j −n∑i=1z−i ai . (5)Since Az is represented in (5) as a difference of two convex combinations of thecolumns of A, the inequality ‖Az‖ ≤ diam A follows from the geometrically clearfact that a set has the same diameter as its convex hull. However, we use (4) and (5) tocontinue with an algebraic argument:Az =n∑j=1(n∑i=1z−i)z+j a j −n∑i=1⎛⎝ n∑j=1z+j⎞⎠ z−i ai=n∑j=1n∑i=1z+j z−i (a j − ai ).38 c© THE MATHEMATICAL ASSOCIATION OF AMERICA [Monthly 122Consequently,‖Az‖ ≤n∑j=1n∑i=1z+j z−i∥∥a j − ai∥∥ (by the triangle inequality and z+j , z−j ≥ 0)≤ (diam A)n∑k=1z+kn∑j=1z−j (by definition of diam A)= 2(var A) (by (4) and definition of var A)= (var A)‖z‖ (since ‖z‖ = 2).This completes the proof of (3) and the theorem is proved.4. POWERS OF SOMEWHAT STOCHASTIC MATRICES. Let A be a some-what stochastic matrix, that is, A is square and real and 1A = 1. The equalities1Ak = (1A)Ak−1 = 1Ak−1 = · · · = (1A)A = 1A = 1show that any power Ak of A is somewhat stochastic. Furthermore, if 1y = 0, then1Aky = 1y = 0 for all positive integers k. This property of a somewhat stochasticmatrix A allows us to repeatedly apply the proposition to powers of A. Assuming that1y = 0 and, for the sake of simplicity, setting c = var A, we have‖Aky‖ = ‖A(Ak−1y)‖ ≤ c‖Ak−1y‖ ≤ · · · ≤ ck−1‖Ay‖ ≤ ck‖y‖. (6)Now we are ready to state and prove the main result.Theorem. Let A be an n × n somewhat stochastic matrix. Assume that there existss ∈ N such that var(As) < 1. Then(a) there exists a unique q ∈ Rn such that Aq = q and 1q = 1,(b) if x is such that 1x = 1, then the sequence {Akx} converges to q as k tends to+∞.Proof. The assumption that 1A = 1 means that 1 is an eigenvalue of A	, the transposeof A. Since A	 and A have the same eigenvalues, there exists a real nonzero vector vsuch that Av = v.Let s be a positive integer such that var(As) < 1 and set c = var(As). Clearly,Asv = v. If 1v = 0, then the proposition yields‖v‖ = ‖Asv‖ ≤ c‖v‖ < ‖v‖.This is a contradiction. Therefore, 1v = 0. Setting q = (1v)−1v provides a vectorwhose existence is claimed in (a). To verify uniqueness, let p be another such vector.Then 1(p − q) = 0, As(p − q) = p − q, and, by the proposition,‖p − q‖ = ‖As(p − q)‖ ≤ c‖p − q‖.Consequently, p − q = 0 since 0 ≤ c < 1.January 2015] SOMEWHAT STOCHASTIC MATRICES 39Let k ∈ N be such that k > s and assume that 1y = 0. By the division algorithm,there exist unique integers j and r such that k = s j + r and r ∈ {0, . . . , s − 1}. Herej > (k/s) − 1 > 0. Now we apply (6) to the matrix As and vector Ar y. We obtain‖Aky‖ = ‖(As) j Ar y‖ ≤ c j‖Ar y‖.Consequently, for all k > s, we have‖Aky‖ ≤ c(k/s)−1 max0≤r<s‖Ar y‖. (7)Let x ∈ Rn be such that 1x = 1. Then 1(x − q) = 0. Substituting y = x − q in (7)yields∥∥Akx − q∥∥ = ∥∥Ak(x − q)∥∥ ≤ c(k/s)−1 max1≤r<s‖Ar (x − q)‖.Now, since 0 ≤ c < 1, we get Akx → q as k → +∞. This proves (b) and completesthe proof.The standard theorem about Markov chains is obtained as a corollary to ourtheorem.Corollary. Let P be an n × n stochastic matrix. Assume that there exists s ∈ N suchthat all entries of Ps are positive. Then(a) there exists a unique probability vector q ∈ Rn such that Pq = q,(b) if x is a probability vector, then the sequence {Pkx} converges to q as k tendsto +∞.Proof. To apply the theorem, we will prove that var(Ps) < 1. For i, j ∈ {1, . . . , n},denote by bi j the entries of Ps which, by assumption, are positive. Next, notice thatfor positive numbers a and b we have |a − b| < a + b. Therefore, for arbitrary i, j wehaven∑l=1|bli − bl j | <n∑l=1(bli + bl j ) = 2.This proves that the distance between arbitrary columns of Ps is less then 2. Con-sequently, diam(Ps) < 2, and hence, var(Ps) < 1. Now we apply the theorem forconvergence. The proofs of the remaining claims are standard.The theorem can be restated in terms of the powers of A. This follows from thefollowing equivalency.Let A be an arbitrary square matrix and let q be a vector such that 1q = 1. Denoteby Q the square matrix, each of whose columns is equal to q, that is, Q = q1. Thenthe following two statements are equivalent:(i) If x is such that 1x = 1, then the sequence {Akx} converges to q as k tends to+∞;(ii) the powers Ak tend to Q as k tends to +∞.40 c© THE MATHEMATICAL ASSOCIATION OF AMERICA [Monthly 122Assume (i) and let e1, . . . , en be the vectors of the standard basis of Rn . Then the j thcolumn of Ak is Ake j . By (i), Ake j converges to q as k tends to +∞. This proves (ii).Now assume (ii) and let x be a vector with 1x = 1. Then Akx converges to Qx =(q1)x = q(1x) = q. This proves (i).In fact, Q is a projection onto the span of q. To see this, calculate Q2 = (q1)(q1) =q(1q)1 = q11 = q1 = Q and Qx = q(1x) = (1x)q for any x ∈ Rn .5. EXAMPLES. We conclude the article with three examples.Example 1. The matrixA = 15⎡⎣ 0 2 −4−1 −1 06 4 9⎤⎦is somewhat stochastic. The largest distance between two columns is between the sec-ond and the third, and it equals 12/5. Therefore, var A = 6/5 > 1. ButA2 = 125⎡⎣−26 −18 −361 −1 450 44 57⎤⎦and var(A2) = 18/25 < 1. Hence, the theorem applies, and q = 13[−2 1 8]	.Example 2. Consider the following three kinds of stochastic matrices:A =⎡⎣1 + +0 + +0 0 +⎤⎦ , B =⎡⎣+ + 0+ 0 +0 + +⎤⎦ , and C =⎡⎣0 + 00 0 11 + 0⎤⎦ .Here we use + for positive numbers.Since A is upper triangular, all its powers are upper triangular, so no power of A hasall positive entries. Thus, the standard theorem does not apply. However, directly fromthe definition it follows that var A < 1, so the theorem applies. Also, q = [1 0 0]	.The matrix B is not positive but var B < 1, so our theorem applies. Also, the stan-dard theorem applies here as well since B2 is positive.The first five powers of C are⎡⎣0 + 00 0 11 + 0⎤⎦ ,⎡⎣0 0 +1 + 00 + +⎤⎦ ,⎡⎣+ + 00 + ++ + +⎤⎦ ,⎡⎣0 + ++ + ++ + +⎤⎦ , and⎡⎣+ + ++ + ++ + +⎤⎦ .The variation of the first two matrices is 1, while var(C3) < 1. The first positive powerof C is C5.Example 3. In this example, we consider matrices with complex entries. Let ω =(−1 + i√3)/2. Then 1, ω, and ω are the cube roots of unity. So, 1 + ω + ω =0, ωω = 1, ω2 = ω, and ω2 = ω.January 2015] SOMEWHAT STOCHASTIC MATRICES 41Consider one vector and two matrices:v =⎡⎣1ωω⎤⎦ , A = 13⎡⎣1 1 11 1 11 1 1⎤⎦ , and B = 13⎡⎣1 ω ωω 1 ωω ω 1⎤⎦ .We calculate Av = 0, Bv = v, and var B = √3/2. Since ‖Bv‖ > √3/2‖v‖, the ma-trix B and the vector v provide an example showing that the conclusion of the propo-sition may not hold with complex entries.A linear combination of A and B shows that the restriction to real numbers cannotbe dropped in the theorem. Let γ be a complex number and set C = A + γ B. SinceA2 = A, AB = BA = 0 and B2 = B, we have Ck = A + γ k B.The matrix A is stochastic with variation 0, while 1B = 0 and var B = √3/2.Hence, 1C = 1, that is, C is somewhat stochastic with complex entries. Also,var C = var(γ B) = |γ |√3/2.Therefore, if 1 < |γ | < 2/√3, then var C < 1, but the sequence {Ck} diverges, as wecan see from the formula for Ck .Finally, we mention that the vector v together with the vectors u = [1 1 1]	 andw = [1 ω ω]	 form an orthogonal basis for the complex inner product space C3,that A is the orthogonal projection onto the span of u, and that B is the orthogonalprojection onto the span of v.REFERENCES1. I. Ipsen, T. Selee, Ergodicity coefficients defined by vector norms. SIAM J. Matrix Anal. Appl. 32 (2011)153–200.2. A. A. Markov, Extension of the law of large numbers to dependent quantities (in Russian), Izvestiia Fiz.-Matem. Obsch. Kazan Univ., (2nd Ser.), 15 (1906) 135–156; also in [3, pp. 339–361]; English translation[5, Section 11].3. A. A. Markov, Selected Works. Theory of Numbers. Theory of Probability. (Russian) Izdat. Akad. NaukSSSR, Leningrad, 1951.4. E. Seneta, Markov and the Creation of Markov Chains. Edited by A. N. Langville and W. J. Stewart. MAM2006. Markov Anniversary Meeting, Boson Books, Raleigh, NC, 2006, 1–20.5. Probability and Statistics. Russian Papers. Edited by O. B. Sheynin. Selected and translated by OscarSheynin. NG Verlag, Berlin, 2004, also item 5 http://www.sheynin.de/download.html.BRANKO ĆURGUS received his Ph.D. in mathematics in 1985 from the University of Sarajevo in the formerYugoslavia. His Ph.D. research was done under the advisement of Prof. Heinz Langer at the Technical Univer-sity Dresden in the former German Democratic Republic. Since 1987, he has been enjoying life, teaching, andresearching mathematics at Western Washington University in Bellingham, Washington.Department of Mathematics, Western Washington University, Bellingham, Washington 98225, USAcurgus@wwu.eduROBERT I. JEWETT has done research in combinatorics and analysis. He taught mathematics for manyyears at Western Washington University. He is now retired and lives in Bellingham.Department of Mathematics, Western Washington University, Bellingham, Washington 98225, USA42 c© THE MATHEMATICAL ASSOCIATION OF AMERICA [Monthly 122

Somewhat Stochastic Matrices

https://cedar.wwu.edu/cgi/viewcontent.cgi?article=1060&context=math_facpubs

Somewhat Stochastic Matrices

Abstract

Similar works

Full text

Available Versions

Western Washington University