© 2017, Springer International Publishing AG. We introduce a family of orders on the set S+n of positive-definite matrices of dimension n derived from the homogeneous geometry of S+n induced by the natural transitive action of the general linear group GL(n). The orders are induced by affine-invariant cone fields, which arise naturally from a local analysis of the orders that are compatible with the homogeneous structure of S+n. We then revisit the well-known Löwner-Heinz theorem and provide an extension of this classical result derived using differential positivity with respect to affine-invariant cone fields.ER

Mostajeran, C

Sepulchre, R

Apollo (Cambridge)

Affine-Invariant Orders on the Set ofPositive-Definite MatricesCyrus Mostajeran and Rodolphe SepulchreDepartment of Engineering, University of Cambridge, Cambridge CB2 1PZ, UnitedKingdom,csm54@cam.ac.ukAbstract. We introduce a family of orders on the set S+n of positive-definite matrices of dimension n derived from the homogeneous geometryof S+n induced by the natural transitive action of the general linear groupGL(n). The orders are induced by affine-invariant cone fields, which arisenaturally from a local analysis of the orders that are compatible with thehomogeneous structure of S+n . We then revisit the well-known Lo¨wner-Heinz theorem and provide an extension of this classical result derivedusing differential positivity with respect to affine-invariant cone fields.1 IntroductionThe question of how one can order the elements of a space in a consistent andwell-defined manner is of fundamental importance to many areas of appliedmathematics, including the theory of monotone functions and matrix means inwhich the notion of order plays a defining role [9, 6, 1, 8] . These concepts playan important role in a wide variety of applications across information geometrywhere one is interested in performing statistical analysis on sets of matrices. Insuch applications, the choice of order relation is often taken for granted. Thischoice, however, is of crucial significance since a function that is not monotonewith respect to one order, may be monotone with respect to another, in whichcase powerful results from monotonicity theory would become relevant.In this paper, we outline an approach to systematically generate orders onhomogeneous spaces, which form a class of nonlinear spaces that are ubiquitousin many applications in information engineering and control theory. A homoge-neous space is a manifold on which a Lie group acts transitively, in the sensethat any point on the manifold can be mapped onto any other point by an el-ement of a group of transformations that act on the space. The geometry ofhomogeneous spaces, coupled with the observation that cone fields induce conalorders on continuous spaces [7], forms the basis for the approach taken in thispaper. The aim is to systematically generate cone fields that are invariant withrespect to the homogeneous geometry, thereby defining families of conal ordersbuilt upon the underlying symmetries of the space.The focus of this paper is on ordering the elements of the set of symmetricpositive-definite matrices S+n of dimension n. Positive definite matrices arise innumerous applications, including as covariance matrices in statistics and com-puter vision, as variables in convex and semidefinite programming, as unknownsin fundamental problems in systems and control theory, as kernels in machinelearning, and as diffusion tensors in medical imaging. The space S+n forms asmooth manifold that can be viewed as a homogeneous space admitting a tran-sitive action by the general linear group GL(n), which endows the space with anaffine-invariant geometry as reviewed in Section 2. In Section 3, this geometryis used to construct affine-invariant cone fields and new partial orders on S+n .In Section 4, we discuss how differential positivity [5] can be used to study andcharacterize monotonicity on S+n with respect to the invariant orders introducedin this paper. We also state a generalized version of the celebrated Lo¨wner-Heinztheorem [9, 6] of operator monotonicity theory derived using this approach.2 Homogeneous geometry of S+nThe set S+n of symmetric positive definite matrices of dimension n has the struc-ture of a homogeneous space with a transitive GL(n)-action. This follows bynoting that any Σ ∈ S+n admits a Cholesky decomposition Σ = AAT for someA ∈ GL(n). The Cauchy polar decomposition of the invertible matrix A yieldsa unique decomposition A = PQ of A into an orthogonal matrix Q ∈ O(n) anda symmetric positive-definite matrix P ∈ Sn+. Now note that if Σ has Choleskydecomposition Σ = AAT and A has a Cauchy polar decomposition A = PQ,then Σ = PQQTP = P 2. That is, Σ is invariant with respect to the orthogonalpart Q of the polar decomposition. Therefore, we can identify any Σ ∈ S+n withthe equivalence class [Σ1/2] = Σ1/2 · O(n) in the quotient space GL(n)/O(n),where Σ1/2 denotes the unique positive definite square root of Σ. That is,S+n∼= GL(n)/O(n). (1)The identification in (1) can also be made by noting the transitive action ofGL(n) on S+n defined byτA : Σ 7→ AΣAT ∀A ∈ GL(n), ∀Σ ∈ S+n . (2)This action is said to be almost effective in the sense that ±I are the onlyelements of GL(n) that fix every Σ ∈ S+n . The isotropy group of this action atΣ = I is precisely O(n), since τQ : I 7→ QIQT = I if and only if Q ∈ O(n).Once again, if Σ ∈ S+n has Cholesky decomposition Σ = AAT and A has polardecomposition A = PQ, then τA(I) = AIAT = P 2 = Σ.A homogeneous space G/H is said to be reductive if there exists a subspacem of g such that g = h ⊕ m and Ad(H)m ⊆ m. Recall that the Lie algebragl(n) of GL(n) consists of the set Rn×n of all real n × n matrices equippedwith the Lie bracket [X,Y ] = XY − Y X, while the Lie algebra of O(n) iso(n) = {X ∈ Rn×n : XT = −X}. Since any matrix X ∈ Rn×n has a uniquedecomposition X = 12 (X−XT )+ 12 (X+XT ), as a sum of an antisymmetric partand a symmetric part, we have gl(n) = o ⊕ m, where m = {X ∈ Rn×n : XT =X}. Furthermore, since AdQ(S) = QSQ−1 = QSQT is a symmetric matrix foreach S ∈ m, we have AdO(n) = {QSQ−1 : Q ∈ O(n), S ∈ m} ⊆ m. Hence,S+n = GL(n)/O(n) is indeed a reductive homogeneous space with reductivedecomposition gl(n) = o(n)⊕m.The tangent space ToS+n of S+n at the base-point o = [I] = I ·O(n) is identifiedwith m. For each Σ ∈ S+n , the action τΣ1/2 : S+n → S+n induces the vector spaceisomorphism dτΣ1/2 |I : TIS+n → TΣS+n given bydτΣ1/2∣∣IX = Σ1/2XΣ1/2, ∀X ∈ m. (3)The map (3) can be used to extend structures defined in ToS+n to structuresdefined on the tangent bundle TS+n through affine-invariance, provided that thestructures in ToS+n are AdO(n)-invariant. The AdO(n)-invariance is required toensure that the extension to TS+n is unique and thus well-defined. For instance,any homogeneous Riemannian metric on S+n∼= GL(n)/O(n) is determined by anAdO(n)-invariant inner product on m. Any such inner product induces a normthat is rotationally invariant and so can only depend on the scalar invariantstr(Xk) where k ≥ 1 and X ∈ m. Moreover, as the inner product is a quadraticfunction, ‖X‖2 must be a linear combination of (tr(X))2 and tr(X2). Thus, anyAdO(n)-invariant inner product on m must be a scalar multiple of〈X,Y 〉m = tr(XY ) + µ tr(X) tr(Y ), (4)where µ is a scalar parameter with µ > −1/n to ensure positive-definiteness [12].Therefore, the corresponding affine-invariant Riemannian metrics are generatedby (3) and given by〈X,Y 〉Σ = 〈Σ−1/2XΣ−1/2, Σ−1/2Y Σ−1/2〉m= tr(Σ−1XΣ−1Y ) + µ tr(Σ−1X) tr(Σ−1Y ), (5)for Σ ∈ S+n and X,Y ∈ TΣS+n . In the case µ = 0, (5) yields the most commonlyused ‘natural’ Riemannian metric on S+n , which corresponds to the Fisher in-formation metric for the multivariate normal distribution [4, 13], and has beenwidely used in applications such as tensor computing in medical imaging.3 Affine-invariant orders3.1 Affine-invariant cone fieldsA cone field K on S+n smoothly assigns a cone K(Σ) ⊂ TΣS+n to each pointΣ ∈ S+n . We say that K is affine-invariant or homogeneous with respect to thequotient geometry S+n∼= GL(n)/O(n) if(dτΣ1/22 Σ−1/21∣∣Σ1)K(Σ1) = K(Σ2), (6)for all Σ1, Σ2 ∈ S+n . The procedure we will use for constructing affine-invariantcone fields on S+n is similar to the approach taken for generating the affine-invariant Riemannian metrics in Section 2. We begin by defining a cone K(I) atI that is AdO(n)-invariant:X ∈ K(I)⇔ AdQX = dτQX = QXQT ∈ K(I), ∀Q ∈ O(n). (7)Using such a cone, we generate a cone field viaK(Σ) = dτΣ1/2∣∣IK(I) = {X ∈ TΣS+n : Σ−1/2XΣ−1/2 ∈ K(I)}. (8)The AdO(n)-invariance condition (7) is satisfied if K(I) has a spectral characteri-zation; that is, we can check to see if any given X ∈ TIS+n ∼= m lies in K(I) usingonly properties of X that are characterized by its spectrum. For instance, tr(X)and tr(X2) are both properties of X that are spectrally characterized and indeedAdO(n)-invariant. Furthermore, quadratic AdO(n)-invariant cones are defined byinequalities on suitable linear combinations of (tr(X))2 and tr(X2).Proposition 1 For any choice of parameter µ ∈ (0, n), the setK(I) = {X ∈ TIS+n : (tr(X))2 − µ tr(X2) ≥ 0, tr(X) ≥ 0}, (9)defines an AdO(n)-invariant cone in TIS+n = {X ∈ Rn×n : XT = X}.Proof. AdO(n)-invariance is clear since tr(X2) = tr(QXQTQXQT ) and tr(X) =tr(QXQT ) for all Q ∈ O(n). To prove that (9) is a cone, first note that 0 ∈ K(I)and for λ > 0, X ∈ K(I), we have λX ∈ K(I) since tr(λX) = λ tr(X) ≥ 0 and(tr(λX))2 − µ tr((λX)2) = λ2[(tr(X))2 − µ tr(X2)] ≥ 0. (10)To show convexity, let X1, X2 ∈ K(I). Now tr(X1 +X2) = tr(X1) + tr(X2) ≥ 0,and(tr(X1 +X2))2 − µ tr((X1 +X2)2) = [(tr(X1))2 − µ tr(X21 )]+ [(tr(X2))2 − µ tr(X22 )] + 2[tr(X1) tr(X2)− µ tr(X1X2)] ≥ 0, (11)since tr(X1X2) ≤ (tr(X21 ))12 (tr(X22 ))12 ≤ 1√µ tr(X1) 1√µ tr(X2), where the firstinequality follows by Cauchy-Schwarz. Finally, we need to show that K(I) ispointed. If X ∈ K(I) and −X ∈ K(I), then tr(−X) = − tr(X) = 0. Thus,(tr(X))2 − µ tr(X2) = −µ tr(X2) ≥ 0, which is possible if and only if all of theeigenvalues of X are zero; i.e., if and only if X = 0. uunionsqThe parameter µ controls the opening angle of the cone. If µ = 0, then (9)defines the half-space tr(X) ≥ 0. As µ increases, the opening angle of the conebecomes smaller and for µ = n (9) collapses to a ray. For any fixed µ ∈ (0, n),we obtain a unique well-defined affine-invariant cone field given byK(Σ) = {X ∈ TΣS+n : (tr(Σ−1X))2 − µ tr(Σ−1XΣ−1X) ≥ 0, tr(Σ−1X) ≥ 0}.(12)It should be noted that of course not all AdO(n)-invariant cones at I arequadratic. Indeed, it is possible to construct polyhedral AdO(n)-invariant conesthat arise as the intersections of a collection of spectrally defined half-spacesin TIS+n . The clearest example of such a construction is the cone of positive-semidefinite matrices in TIS+n , which of course itself has a spectral characteri-zation K(I) = {X ∈ TIS+n : λi(X) ≥ 0, i = 1, . . . , n}, where (λi(X)) denote then real eigenvalues of the symmetric matrix X.3.2 Visualization of affine-invariant cone fields on S+2It is well-known that the set of positive semi-definite matrices of dimension nforms a cone in the space of symmetric n× n matrices. Moreover, S+n forms theinterior of this cone. A concrete visualization of this identification can be madein the n = 2 case, as shown in Figure 1. The set S+2 can be identified with theinterior of K = {(x, y, z) ∈ R3 : z2 − x2 − y2 ≥ 0, z ≥ 0}, through the mapφ : S+2 → K given byφ :(a bb c)7→ (x, y, z) =(√2b,1√2(a− c), 1√2(a+ c)). (13)xyzz2 − x2 − y2 ≥ 0z ≥ 0φ : S+2 → K = {(x, y, z) ∈ R3 : z2 − x2 − y2 ≥ 0, z ≥ 0}S+2 ={(a bb c): ac− b2 > 0, a+ c > 0}Fig. 1. Identification of S+2 with the interior of the closed, convex, pointed cone K ={(x, y, z) ∈ R3 : z2 − x2 − y2 ≥ 0, z ≥ 0} in R3.Inverting φ, we find that a = 1√2(z+y), b = 1√2x, c = 1√2(z−y). Note that thepoint (x, y, z) = (0, 0,√2) corresponds to the identity matrix I ∈ S+2 . We seekto arrive at a visual representation of the affine-invariant cone fields generatedfrom the AdO(n)-invariant cones (9) for different choices of the parameter µ. Thedefining inequalities tr(X) ≥ 0 and (tr(X))2 − µ tr(X2) ≥ 0 in TIS+2 take theformsδz ≥ 0, and(2µ− 1)δz2 − δx2 − δy2 ≥ 0, (14)respectively, where (δx, δy, δz) ∈ T(0,0,√2)K ∼= TIS+2 . Clearly the translation-invariant cone fields generated from this cone are given by the same equationsas in (14) for (δx, δy, δz) ∈ T(x,y,z)K ∼= Tφ−1(x,y,z)S+2 .To obtain the affine-invariant cone fields, note that at Σ = φ−1(x, y, z) ∈ S+2 ,the inequality tr(Σ−1X) ≥ 0 takes the formtr[(c −b−b a)(δa δbδb δc)]= c δa− 2b δb+ a δc ≥ 0 (15)⇔ z δz − x δx− y δy ≥ 0. (16)Similarly, the inequality (tr(Σ−1X))2 − µ tr(Σ−1XΣ−1X) ≥ 0 is given by2(x δx+ y δy − z δz)2−µ [(z2 + x2 − y2)δx2 + (z2 − x2 − y2)δy2+ (x2 + y2 + z2)δz2 + 4xy δxδy − 4xz δxδz − 4yz δyδz] ≥ 0, (17)where (δx, δy, δz) ∈ T(x,y,z)K ∼= TΣS+2 . In the case µ = 1, this reduces to( 2µ − 1)δz2− δx2− δy2 ≥ 0. That is, for µ = 1 the quadratic cone field generatedby affine-invariance coincides with the corresponding translation-invariant conefield. Generally, however, affine-invariant and translation-invariant cone fields donot agree, as depicted in Figure 2. Each of the different cone fields in Figure 2induces a distinct partial order on S+n .µ > 1 µ = 1 µ < 1(a)(b)Fig. 2. Cone fields on S+2 : (a) Quadratic affine-invariant cone fields for different choicesof the parameter µ ∈ (0, 2). (b) The corresponding translation-invariant cone fields.3.3 The Lo¨wner orderThe Lo¨wner order is the partial order ≥L on S+n defined byA ≥L B ⇔ A−B ≥L O, (18)where the inequality on the right denotes that A−B is positive semi-definite [2].The definition in (18) is based on translations and the ‘flat’ geometry of S+n . Itis clear that the Lo¨wner order is translation invariant in the sense that A ≥L Bimplies that A+C ≥L B+C for all A,B,C ∈ S+n . From the perspective of conalorders, the Lo¨wner order is the partial order induced by the cone field generatedby translations of the cone of positive semi-definite matrices at TIS+n .In the previous section, we gave an explicit construction showing that thecone field generated through translations of the cone of positive semi-definite ma-trices at TIS+n coincides with the cone field generated through affine-invariancein the n = 2 case. We will now show that this is a general result which holds forall n. First note that the cone at TIS+n can be expressed asK(I) = {X ∈ TIS+n : uTXu ≥ 0 ∀u ∈ Rn, uTXu = 0⇒ u = 0}, (19)and the resulting translation-invariant cone field is simply given byKT (Σ) = {X ∈ TΣS+n : uTXu ≥ 0 ∀u ∈ Rn, uTXu = 0⇒ u = 0}. (20)The corresponding affine-invariant cone field is given byKA(Σ) = {X ∈ TΣS+n : uTΣ−1/2XΣ−1/2u ≥ 0 ∀u ∈ Rn,uTΣ−1/2XΣ−1/2u = 0⇒ u = 0}, (21)which is seen to be equal to KT by introducing the invertible transformationu¯ = Σ−1/2u in (21). Thus we see that the Lo¨wner order enjoys the special statusof being both affine-invariant and translation-invariant, even though its classicaldefinition is based on the ‘flat’ or translational geometry on S+n .4 Monotonicity on S+nLet f be a map of S+n into itself. We say that f is monotone with respect to apartial order ≥ on S+n if f(Σ1) ≥ f(Σ2) whenever Σ1 ≥ Σ2. Such functions wereintroduced by Lo¨wner in his seminal paper [9] on operator monotone functions.Since then operator monotone functions have been studied extensively and foundapplications to many fields including electrical engineering, network theory, andquantum information theory [3, 10]. One of the most fundamental results inoperator theory is the Lo¨wner-Heinz theorem [9, 6] stated below.Theorem 1 (Lo¨wner-Heinz) If Σ1 ≥L Σ2 in S+n and r ∈ [0, 1], thenΣr1 ≥L Σr2 . (22)Furthermore, if n ≥ 2 and r > 1, then Σ1 ≥L Σ2 6⇒ Σr1 ≥L Σr2 .There are several different proofs of the Lo¨wner-Heinz theorem. See [2, 11,9, 6], for instance. Most of these proofs are based on analytic methods, such asintegral representations from complex analysis. Instead we employ a geometricapproach to study monotonicity based on a differential analysis of the system.One of the advantages of such an approach is that it is immediately applicableto all of the conal orders considered in this paper, while providing a deepergeometric insight into the behavior of the map under consideration. Recall thata smooth map f : S+n → S+n is said to be differentially positive with respect toa cone field K on S+n ifδΣ ∈ K(Σ) ⇒ df |Σ(δΣ) ∈ K(f(Σ)), (23)where df |Σ : TΣS+n → Tf(Σ)S+n denotes the differential of f at Σ. Assumingthat ≥K is a partial order induced by K, then f is monotone with respect to ≥Kif and only if it is differentially positive with respect to K. Applying this to thefamily of affine-invariant cone fields in (12), we arrive at the following extensionto the Lo¨wner-Heinz theorem.Theorem 2 (Generalized Lo¨wner-Heinz) For any of the quadratic affine-invariant orders (12) parameterized by µ, and r ∈ [0, 1], the map f(Σ) = Σr ismonotone on S+n .This result suggests that the monotonicity of the map f : Σ 7→ Σr for r ∈ (0, 1)is intimately connected to the affine-invariant geometry of S+n and not its trans-lational geometry. The proof of Theorem 2 has been omitted from this abstractdue to length limitations. A more detailed treatment of the topics discussed here,alongside new results and a proof of Theorem 2 will be provided in a subsequentjournal paper.5 ConclusionThe choice of partial order is a key part of studying monotonicity of functionsthat is often taken for granted. Invariant cone fields provide a geometric approachto systematically construct ‘natural’ orders by connecting the geometry of thestate space to the search for orders. Coupled with differential positivity, invariantcone fields provide an insightful and powerful method for studying monotonicity,as shown in the case of S+n .References1. Ando, T.: Concavity of certain maps on positive definite matrices and applicationsto hadamard products. Linear Algebra and its Applications 26, 203 – 241 (1979)2. Bhatia, R.: Positive Definite Matrices. Princeton University Press (2007)3. Bhatia, R.: Matrix analysis, vol. 169. Springer Science & Business Media (2013)4. Burbea, J., Rao, C.: Entropy differential metric, distance and divergence measuresin probability spaces: A unified approach. Journal of Multivariate Analysis 12(4),575 – 596 (1982)5. Forni, F., Sepulchre, R.: Differentially positive systems. IEEE Transactions onAutomatic Control 61(2), 346–359 (2016)6. Heinz, E.: Beitrge zur strungstheorie der spektralzerlegung. Mathematische An-nalen 123, 415–438 (1951)7. Hilgert, J., Hofmann, K.H., Lawson, J.: Lie groups, convex cones, and semi-groups.Oxford University Press (1989)8. Kubo, F., Ando, T.: Means of positive linear operators. Mathematische Annalen246, 205–224 (1979)9. Lo¨wner, K.: U¨ber monotone matrixfunktionen. Mathematische Zeitschrift 38, 177–216 (1934)10. Nielsen, M.A., Chuang, I.: Quantum computation and quantum information (2002)11. Pedersen, G.K.: Some operator monotone functions. Proceedings of the AmericanMathematical Society 36(1), 309–310 (1972)12. Pennec, X.: Statistical computing on manifolds for computational anatomy. Ph.D.thesis, Universite´ Nice Sophia Antipolis (2006)13. Skovgaard, L.T.: A Riemannian geometry of the multivariate normal model. Scan-dinavian Journal of Statistics 11(4), 211–223 (1984)

Affine-invariant orders on the set of positive-definite matrices

https://www.repository.cam.ac.uk/bitstream/handle/1810/294314/%2382-CyrusMostajeran.pdf?sequence=3&isAllowed=y

Affine-invariant orders on the set of positive-definite matrices

Abstract

Similar works

Full text

Available Versions

Apollo (Cambridge)