The article establishes relationships between the matrix derivatives of F with respect to X as introduced by von Rosen (1988), Kollo and von Rosen (2000) and the Magnus-Neudecker (1999) matrix derivative. The usual transformations apply and the Moore-Penrose inverse of the duplication matrix is used. Both X and F have the same dimension

Neudecker, Heinz

The article establishes relationships between the matrix derivatives of Fwith respect to Xas introduced by von Rosen (1988), Kollo and von Rosen (2000) and the Magnus-Neudecker (1999) matrix derivative.
The usual transformations apply and the Moore-Penrose inverse of the duplication matrix is used. Both Xand F have the same dimension.Peer Reviewe

UPCommons. Portal del coneixement obert de la UPC

Statistics & Operations Research TransactionsSORT 27 (2) July-December 2003, 153-164Statistics &Operations ResearchTransactionsOn two matrix derivativesby Kollo and von RosenHeinz Neudecker∗University of Amsterdam, The NetherlandsAbstractThe article establishes relationships between the matrix derivatives of F with respect to X as introducedby von Rosen (1988), Kollo and von Rosen (2000) and the Magnus-Neudecker (1999) matrix derivative.The usual transformations apply and the Moore-Penrose inverse of the duplication matrix is used. BothX and F have the same dimension.MSC: primary 15A69, secondary 62H10Keywords: Matrix derivatives and differentials, vectorization, commutation and duplication matrices1 IntroductionVon Rosen (1988) and Kollo and von Rosen (2000) study moments of the invertedWishart distribution. For finding specific expressions they use two types of matrixderivatives. Unfortunately these are not easily accessible to the uninitiated reader. Thetwo are obviously related, both being matrix representations of the Fre´chet derivative.There is, however, a more accessible representation, namely the Magnus-Neudeckermatrix derivative. In this article we shall link the three representations and considersome illustrative applications, most of these taken from the two quoted articles.∗Address for correspondence: Oosterstraat, 13. 1741 GH Schagen. The Netherlands. Postal address: NL 1741 GH13. E-mail: heinz@fee.uva.nlReceived: April 2002Accepted: June 2003154 On two matrix derivatives by Kollo and von Rosen2 The three matrix derivativesVon Rosen (1988) defined the matrix derivative∂F∂X=∑i jklεkl∂ fi j∂xklEi j ⊗ Ekl (1)whereεkl =1 if k = l12if k , l,Ei j = eie′j.where ei is the ith column of the identity matrix Ip, F =(fi j)and X = (xkl) are symmetricmatrices of dimension p, and i, j, k, l = 1, . . . , p. F = F(X) is a function of X.Kollo and von Rosen (2000) defined the matrix derivativedFdX=∑i jklεkl∂ fi j∂xkl(vec Ei j)(vec Ekl)′ . (2)It is clear thatKp2, p2 vecdFdX= Cp2vec∂F∂X(3)with Cp2= Ip ⊗Kpp ⊗ Ip, K denoting a commutation matrix. We used property (ii) of theAppendix.Thereforevec∂F∂X=(Kpp ⊗ Kpp)Cp2vecdFdX, (4)becauseKp2, p2 = Cp2(Kpp ⊗ Kpp)Cp2.See, e.g. Ghazal and Neudecker (2000) for properties of Cp2. Some will be reported inthe Appendix.In Section 3 we shall establish the identitydFdX= Dp∂ f∂x′D+p (5)where∂ f∂x′is the Magnus-Neudecker matrix derivative and D+p is the Moore-Penroseinverse of the duplication matrix Dp. Further f = D+p vec F and x = D+p vec X.Equivalently f = v(F) and x = v(X).The combination of (4) and (5) would enable us to express ∂F∂Xin∂ f∂x′, in vectorizedform. As this is not so fruitful, we shall not do it. We prefer to use (4) and subsequentlydevectorize the resulting vec ∂F∂X.Heinz Neudecker 1553 The link betweendFdXand∂ f∂ x′The first thing to do is to establishLemma 1Kppd Fd X=d Fd X,d Fd XKpp =dFdX. (6)Proof. We have Kpp(e j ⊗ ei) = ei ⊗ e j. HenceKpp∂ fi j∂xkl(e j ⊗ ei)=∂ fi j∂xkl(ei ⊗ e j)=∂ f ji∂xkl(ei ⊗ e j),andKpp∑i j∂ fi j∂xkl(e j ⊗ ei)=∑i j∂ f ji∂xkl(ei ⊗ e j)=∑i j∂ fi j∂xkl(e j ⊗ ei)by interchanging the indices i and j. Hence KppdFdX=dFdX. Similarly we find thatdFdXKpp =dFdX. ¤Having found these basic properties ofdFdXwe shall proveLemma 2dFdXd vec X = d vec F.156 On two matrix derivatives by Kollo and von RosenProof. Using the definition ofdFdXwe getdFdXd vec X =∑i jklεkl∂ fi j∂ xkl(e j ⊗ ei)(el ⊗ ek)′d vec X=∑i jklεkl∂ fi j∂ xkl(e j e′l ⊗ ei e′k)d vec X=∑i jklεkl∂ fi j∂ xkld vec ei e′k X el e′j=∑i jklεkl(∂ fi j∂ xkld xkl)vec ei e′j=∑i jklεkl∂ vec fi j ei e′j∂ xkld xkl=∑klεkl∂ vec F∂ xkld xkl = d vec F. ¤Having established this result we shall proveTheorem 3Dp∂ f∂ x′=dFdXDp.Proof. We rewrite the result of lemma 2 asdFdXDp dx = Dp d f = Dp∂ f∂x′dx.Omitting the arbitrary dx we obtain the result. ¤The basic result follows as a corollary, namely.Corollary 4dFdX= Dp∂ f∂ x′D+p .Proof. Postmultiplication of the result of theorem 3 by D+p yields, by virtue of lemma1,Dp∂ f∂ x′D+p =dFdXDp D+p =12dFdX(Ip2 + Kpp)=dFdX.¤Heinz Neudecker 1574 Some applications ofdFdXWe shall examine three cases.(1) Kollo & von Rosen (2000) finddFdX=12(Ip2 + Kpp),for F = X (their result 2.9).This can be derived succinctly in the following way. Differentiation of F yieldsdF = dX, which leads to d vec F = d vec X and subsequently to d f = dx. HencedFdX= Dp D+p =12(Ip2 + Kpp)by corollary 4.¤(2)dFdX= − 12(Ip2 + Kpp) (X−1 ⊗ X−1)for F = X−1.Proceeding as before we getdF = −X−1(dX)X−1,d vec F = −(X−1 ⊗ X−1)d vec X,d f = −D+p(X−1 ⊗ X−1)Dp dx and∂ f∂x′= −D+p(X−1 ⊗ X−1)Dp.Corollary 4 yields thendFdX= −Dp D+p(X−1 ⊗ X−1)Dp D+p = −14(Ip2 + Kpp) (X−1 ⊗ X−1) (Ip2 + Kpp)== − 14(Ip2 + Kpp)2 (X−1 ⊗ X−1)= − 12(Ip2 + Kpp) (X−1 ⊗ X−1).¤(3)dFdX= 12(Ip2 + Kpp) (Ip ⊗ X + X ⊗ Ip)for F = X2.We getdF = (dX)X + X dX,d vec F =(Ip ⊗ X + X ⊗ Ip)d vec X,d f = D+p(Ip ⊗ X + X ⊗ Ip)Dp d x.158 On two matrix derivatives by Kollo and von RosenHence∂ f∂x′= D+p(Ip ⊗ X + X ⊗ Ip)Dp,dFdX= Dp D+p(Ip ⊗ X + X ⊗ Ip)Dp D+p == 14(Ip2 + Kpp) (Ip ⊗ X + X ⊗ Ip) (Ip2 + Kpp)== 14(Ip2 + Kpp)2 (Ip ⊗ X + X ⊗ Ip)== 12(Ip2 + Kpp) (Ip ⊗ X + X ⊗ Ip).¤In the following section we shall give some applications of∂F∂X.5 Some applications of∂F∂X(1)∂X∂X= 12(vec Ip) (vec Ip)′+ 12Kpp.See von Rosen (1988, Lemma 2.1, d, i).Derivation: UsedXdX= 12(Ip2 + Kpp).See section 4 (1). It is easy to see that Cp2vec Ip2 = Cp2vec(Ip ⊗ Ip)= vec Ip⊗vec Ip.Further Cp2vecKpp = vecKpp. For these properties we refer to (ii) and (iii) in theAppendix.Hencevec∂X∂X= 12(Kpp ⊗ Kpp) (vec Ip ⊗ vec Ip + vecKpp)= 12(vec Ip ⊗ vec Ip + vecKpp)= 12vec[(vec Ip) (vec Ip)′+ Kpp],from which (1) follows.¤(2)∂X−1∂X= − 12(vec X−1) (vec X−1)′−12Kpp(X−1 ⊗ X−1).See von Rosen (1988, Lemma 2.2i).Heinz Neudecker 159Derivation: UsedX−1dX= − 12(Ip2 + Kpp) (X−1 ⊗ X−1).See section 4 (2). Thenvec∂X−1∂X= − 12(Kpp ⊗ Kpp)Cp2vec[(Ip2 + Kpp) (X−1 ⊗ X−1)]= − 12(Kpp ⊗ Kpp) (vec X−1 ⊗ vec X−1)= − 12(Kpp ⊗ Kpp)vecKpp(X−1 ⊗ X−1)= − 12vec X−1 ⊗ vec X−1 − 12vecKpp(X−1 ⊗ X−1)= − 12vec[(vec X−1) (vec X−1)′]−12vecKpp(X−1 ⊗ X−1),from which (2) follows.We also used property 18 in Neudecker (2000). For a simple proof see (iv) in theAppendix.¤(3)∂X2∂X= 12Kpp(Ip ⊗ X + X ⊗ Ip)+ 12(vec X)(vec Ip)′+ 12(vec Ip)(vec X)′.Derivation: UsedX2dX= 12(Ip2 + Kpp) (Ip ⊗ X + X ⊗ Ip).See section 4 (3). Thenvec∂X2∂X= 12(Kpp ⊗ Kpp)Cp2vec[(Ip2 + Kpp) (Ip ⊗ X + X ⊗ Ip)]= 12(Kpp ⊗ Kpp) (vec Ip ⊗ vec X + vec X ⊗ vec Ip)+ 12(Kpp ⊗ Kpp)vec[Kpp(Ip ⊗ X + X ⊗ Ip)]= 12(vec Ip ⊗ vec X + vec X ⊗ vec Ip)+ 12vec[Kpp(Ip ⊗ X + X ⊗ Ip)]= 12vec[(vec X)(vec Ip)′+(vec Ip)(vec X)′]+ 12vec[Kpp(Ip ⊗ X + X ⊗ Ip)].Devectorization yields (3).¤(4)∂F−1∂X= −(F−1 ⊗ Ip) ∂F∂X(F−1 ⊗ Ip), where F = F(X).See von Rosen (1988, Lemma 2.1, c, iii).160 On two matrix derivatives by Kollo and von RosenDerivation: It is known that dF−1 = −F−1(dF)F−1, hencedv(F−1)= D+p d vec F−1 = −D+p(F−1 ⊗ F−1)d vec F= −D+p(F−1 ⊗ F−1)Dp d v(F) = −D+p(F−1 ⊗ F−1)Dp∂ f∂x′dxand finally∂v(F−1)∂x′= −D+p(F−1 ⊗ F−1)Dp∂ f∂x′.By Corollary 4 and Lemma 1 we havedF−1dX= −Dp D+p(F−1 ⊗ F−1)Dp∂ f∂x′D+p= − 12(Ip2 + Kpp) (F−1 ⊗ F−1) dFdX= − 12(F−1 ⊗ F−1) (Ip2 + Kpp) dFdX= −(F−1 ⊗ F−1) dFdX.Application of Section 2 (4) yieldsvec∂F−1∂X= −(Kpp ⊗ Kpp)Cp2vec(F−1 ⊗ F−1) dFdX= −(Kpp ⊗ Kpp)Cp2(Ip2 ⊗ F−1⊗ F−1)vecdFdX= −(Kpp ⊗ Kpp)Cp2(Ip ⊗ Ip ⊗ F−1⊗ F−1)vecdFdX= −(Kpp ⊗ Kpp) (Ip ⊗ F−1⊗ Ip ⊗ F−1)Cp2vecdFdX= −(F−1 ⊗ Ip ⊗ F−1⊗ Ip) (Kpp ⊗ Kpp)Cp2vecdFdX= −(F−1 ⊗ Ip ⊗ F−1⊗ Ip)vec∂F∂X= −vec[(F−1 ⊗ Ip) ∂F∂X(F−1 ⊗ Ip)].Hence∂F−1∂X= −(F−1 ⊗ Ip) ∂F∂X(F−1 ⊗ Ip).¤Heinz Neudecker 1616 AppendixThe following matrix properties have been used in this article. The first five involveCp2= Ip ⊗ Kpp ⊗ Ip.(i) Cp2(A ⊗ B ⊗C ⊗ D)Cp2= A ⊗C ⊗ B ⊗ D, for (p × p) A, B,C,D.(ii) Cp2vec (A ⊗ B) = vec A ⊗ vec B, for (p × p) A and B.Proof.Cp2vec (A ⊗ B) =∑i j(Ip ⊗ Ei j ⊗ E ji ⊗ Ip)vec (A ⊗ B)=∑i jvec[(E ji ⊗ Ip)(A ⊗ B)(Ip ⊗ E ji)]=∑i jvec(E ji A ⊗ BE ji)=∑i jvec(e j Ai. ⊗ B. j e′i)=∑i jvec[(e j ⊗ B. j) (Ai. ⊗ e′i)]== vec∑jvec B. j e′j∑ivec ei Ai.′= vec[(vec B) (vec A)′]= vec A ⊗ vec B.As usual Ai. is the ith row of A, A. j is the jth column of A.¤(iii) Cp2vecKpp = vecKpp.Proof.Cp2vecKpp =∑i jCp2vec(Ei j ⊗ E ji)=∑i j(vec Ei j ⊗ vec E ji)= vec∑i j(vec E ji) (vec Ei j)′= vec∑i j(ei ⊗ e j) (e′j ⊗ e′i)= vec∑i j(ei e′j ⊗ e j e′i)= vec∑i j(Ei j ⊗ E ji)= vecKpp.¤(iv) Cp2vecKpp (A ⊗ B) = vecKpp (A ⊗ B′) for (p × p) A and B.162 On two matrix derivatives by Kollo and von RosenProof.Cp2vecKpp (A ⊗ B) = Cp2(A′ ⊗ B′ ⊗ Ip ⊗ Ip)vecKpp=(A′ ⊗ Ip ⊗ B′⊗ Ip)Cp2vecKpp =(A′ ⊗ Ip ⊗ B′⊗ Ip)vecKpp= vec(B′ ⊗ Ip)Kpp(A ⊗ Ip)= vecKpp(Ip ⊗ B′) (A ⊗ Ip)= vecKpp (A ⊗ B′) .¤(v) Kp2, p2 = Cp2(Kpp ⊗ Kpp)Cp2.Proof. Consider the(p2, p2)matrixQ =∑kl(Ekl ⊗ Qkl)or equivalentlyQ = [Qkl] (k, l = 1, . . . , p)with (p × p) matrix Qkl.Then Kp2, p2 vecQ = vecQ′ andCp2(Kpp ⊗ Kpp)Cp2vecQ =∑i jstCp2(Ei j ⊗ E ji ⊗ Est ⊗ Ets)Cp2vecQ=∑i jstkl(Ei j ⊗ Est ⊗ E ji ⊗ Ets)vec (Ekl ⊗ Qkl)= vec∑i jstkl(E ji ⊗ Ets)(Ekl ⊗ Qkl)(E ji ⊗ Ets)= vec∑i jstkl(E ji Ekl E ji ⊗ Ets Qkl Ets)= vec∑i jst(E ji ⊗ Ets Qi j Ets)= vec∑i jst(Qi j)st(Ei j ⊗ Est)′= vec∑i j(Ei j ⊗ Qi j)′= vecQ′,where(Qi j)stis the (s, t) element of Qi j.¤We further used the standard properties:(vi) Kpp vec A = vec A′ for (p × p) matrix A.Heinz Neudecker 163(vii) Dp D+p =12(Ip2 + Kpp), D+p Dp = Ip∗ with 2p∗ = p(p + 1).(viii) vec A BC = (C′ ⊗ A) vec B for compatible matrices A, B and C.7 ReferencesGhazal, G. A. and Neudecker, H. (2000). On second-order and fourth-order moments of jointly distributedrandom matrices: a survey. Linear Algebra and its Applications, 321, 61-93.Kollo, T. and von Rosen, D. (2000). Distribution and density approximation of the covariance matrix in thegrowth curve model. Statistics, 35, 1-22.Magnus, J. R. and Neudecker, H. (1980). The elimination matrix: some lemmas and applications. SIAMJournal on Algebraic and Discrete Methods, 1, 422-449.Magnus, J. R. and Neudecker, H. (1999). Matrix Differential Calculus with Applications in Statistics andEconometrics, revised edition. John Wiley, Chichester, England.Neudecker, H. (2000). On expected values of fourth-degree matrix products of a multinormal matrixvariate. New Trends in Probability and Statistics. Proceedings of the 6th Tartu Conference onMultivariate Statistics. TEV Vilnius/Utrecht.Neudecker, H. and Wansbeek, T.(1983). Some results on commutation matrices with statistical applications. Canadian Journal ofStatistics, 11, 221-31.von Rosen, D. (1988). Moments for the inverted Wishart distribution. Scandinavian Journal of Statistics,15, 97-109.ResumAquest article estableix les relacions entre las derivades matricials de F respecte de X introduı¨des pervon Rosen (1988), Kollo i von Rosen (2000) i les derivades matricials de Magnus i Neudecker (1999).Les operacions vectorials de duplicacio´ i transformacio´ en vectors so´n les usuals i les inverses de lasmatrius duplicades so´n les de Moore-Penrose. Ambdues X i F = F(X) tenen la mateixa dimensio´.MSC: primary 15A69, secondary 62H10Paraules clau: Derivades i diferencials de matrius, vectoritzacio´, commutacio´ i duplicacio´ de matriusHola

On two matrix derivatives by Kollo and von Rosen

Diposit Digital de Documents de la UAB

The article establishes relationships between the matrix derivatives of Fwith respect to Xas introduced by von Rosen (1988), Kollo and von Rosen (2000) and the Magnus-Neudecker (1999) matrix derivative.

The usual transformations apply and the Moore-Penrose inverse of the duplication matrix is used. Both Xand F have the same dimension.Peer Reviewe

UPCommons

Revistes Catalanes amb Accés Obert

Statistics & Operations Research Transactions
SORT 27 (2) July-December 2003, 153-164
Statistics &
Operations Research
Transactions
On two matrix derivatives
by Kollo and von Rosen
Heinz Neudecker∗
University of Amsterdam, The Netherlands
Abstract
The article establishes relationships between the matrix derivatives of F with respect to X as introduced
by von Rosen (1988), Kollo and von Rosen (2000) and the Magnus-Neudecker (1999) matrix derivative.
The usual transformations apply and the Moore-Penrose inverse of the duplication matrix is used. Both
X and F have the same dimension.
MSC: primary 15A69, secondary 62H10
Keywords: Matrix derivatives and differentials, vectorization, commutation and duplication matrices
1 Introduction
Von Rosen (1988) and Kollo and von Rosen (2000) study moments of the inverted
Wishart distribution. For finding specific expressions they use two types of matrix
derivatives. Unfortunately these are not easily accessible to the uninitiated reader. The
two are obviously related, both being matrix representations of the Fre´chet derivative.
There is, however, a more accessible representation, namely the Magnus-Neudecker
matrix derivative. In this article we shall link the three representations and consider
some illustrative applications, most of these taken from the two quoted articles.
∗Address for correspondence: Oosterstraat, 13. 1741 GH Schagen. The Netherlands. Postal address: NL 1741 GH
13. E-mail: heinz@fee.uva.nl
Received: April 2002
Accepted: June 2003
154 On two matrix derivatives by Kollo and von Rosen
2 The three matrix derivatives
Von Rosen (1988) defined the matrix derivative
∂F
∂X
=
∑
i jkl
εkl
∂ fi j
∂xkl
Ei j ⊗ Ekl (1)
where
εkl =

1 if k = l
1
2
if k , l,
Ei j = eie
′
j.
where ei is the i
th column of the identity matrix Ip, F =
(
fi j
)
and X = (xkl) are symmetric
matrices of dimension p, and i, j, k, l = 1, . . . , p. F = F(X) is a function of X.
Kollo and von Rosen (2000) defined the matrix derivative
dF
dX
=
∑
i jkl
εkl
∂ fi j
∂xkl
(
vec Ei j
)
(vec Ekl)
′ . (2)
It is clear that
Kp2, p2 vec
dF
dX
= C
p
2
vec
∂F
∂X
(3)
with C
p
2
= Ip ⊗Kpp ⊗ Ip, K denoting a commutation matrix. We used property (ii) of the
Appendix.
Therefore
vec
∂F
∂X
=
(
Kpp ⊗ Kpp
)
C
p
2
vec
dF
dX
, (4)
because
Kp2, p2 = C
p
2
(
Kpp ⊗ Kpp
)
C
p
2
.
See, e.g. Ghazal and Neudecker (2000) for properties of C
p
2
. Some will be reported in
the Appendix.
In Section 3 we shall establish the identity
dF
dX
= Dp
∂ f
∂x′
D+p (5)
where
∂ f
∂x′
is the Magnus-Neudecker matrix derivative and D+p is the Moore-Penrose
inverse of the duplication matrix Dp. Further f = D
+
p vec F and x = D
+
p vec X.
Equivalently f = v(F) and x = v(X).
The combination of (4) and (5) would enable us to express ∂F
∂X
in
∂ f
∂x′
, in vectorized
form. As this is not so fruitful, we shall not do it. We prefer to use (4) and subsequently
devectorize the resulting vec ∂F
∂X
.
Heinz Neudecker 155
3 The link between
dF
dX
and
∂ f
∂ x′
The first thing to do is to establish
Lemma 1
Kpp
d F
d X
=
d F
d X
,
d F
d X
Kpp =
dF
dX
. (6)
Proof. We have Kpp(e j ⊗ ei) = ei ⊗ e j. Hence
Kpp
∂ fi j
∂xkl
(
e j ⊗ ei
)
=
∂ fi j
∂xkl
(
ei ⊗ e j
)
=
∂ f ji
∂xkl
(
ei ⊗ e j
)
,
and
Kpp
∑
i j
∂ fi j
∂xkl
(
e j ⊗ ei
)
=
∑
i j
∂ f ji
∂xkl
(
ei ⊗ e j
)
=
∑
i j
∂ fi j
∂xkl
(
e j ⊗ ei
)
by interchanging the indices i and j. Hence Kpp
dF
dX
=
dF
dX
. Similarly we find that
dF
dX
Kpp =
dF
dX
. ¤
Having found these basic properties of
dF
dX
we shall prove
Lemma 2
dF
dX
d vec X = d vec F.
156 On two matrix derivatives by Kollo and von Rosen
Proof. Using the definition of
dF
dX
we get
dF
dX
d vec X =
∑
i jkl
εkl
∂ fi j
∂ xkl
(e j ⊗ ei)(el ⊗ ek)
′d vec X
=
∑
i jkl
εkl
∂ fi j
∂ xkl
(
e j e
′
l ⊗ ei e
′
k
)
d vec X
=
∑
i jkl
εkl
∂ fi j
∂ xkl
d vec ei e
′
k X el e
′
j
=
∑
i jkl
εkl
(
∂ fi j
∂ xkl
d xkl
)
vec ei e
′
j
=
∑
i jkl
εkl
∂ vec fi j ei e
′
j
∂ xkl
d xkl
=
∑
kl
εkl
∂ vec F
∂ xkl
d xkl = d vec F. ¤
Having established this result we shall prove
Theorem 3
Dp
∂ f
∂ x′
=
dF
dX
Dp.
Proof. We rewrite the result of lemma 2 as
dF
dX
Dp dx = Dp d f = Dp
∂ f
∂x′
dx.
Omitting the arbitrary dx we obtain the result. ¤
The basic result follows as a corollary, namely.
Corollary 4
dF
dX
= Dp
∂ f
∂ x′
D+p .
Proof. Postmultiplication of the result of theorem 3 by D+p yields, by virtue of lemma
1,
Dp
∂ f
∂ x′
D+p =
dF
dX
Dp D
+
p =
1
2
dF
dX
(
Ip2 + Kpp
)
=
dF
dX
.
¤
Heinz Neudecker 157
4 Some applications of
dF
dX
We shall examine three cases.
(1) Kollo & von Rosen (2000) find
dF
dX
=
1
2
(
Ip2 + Kpp
)
,
for F = X (their result 2.9).
This can be derived succinctly in the following way. Differentiation of F yields
dF = dX, which leads to d vec F = d vec X and subsequently to d f = dx. Hence
dF
dX
= Dp D
+
p =
1
2
(
Ip2 + Kpp
)
by corollary 4.
¤
(2)
dF
dX
= − 1
2
(
Ip2 + Kpp
) (
X−1 ⊗ X−1
)
for F = X−1.
Proceeding as before we get
dF = −X−1(dX)X−1,
d vec F = −
(
X−1 ⊗ X−1
)
d vec X,
d f = −D+p
(
X−1 ⊗ X−1
)
Dp dx and
∂ f
∂x′
= −D+p
(
X−1 ⊗ X−1
)
Dp.
Corollary 4 yields then
dF
dX
= −Dp D
+
p
(
X−1 ⊗ X−1
)
Dp D
+
p = −
1
4
(
Ip2 + Kpp
) (
X−1 ⊗ X−1
) (
Ip2 + Kpp
)
=
= − 1
4
(
Ip2 + Kpp
)2 (
X−1 ⊗ X−1
)
= − 1
2
(
Ip2 + Kpp
) (
X−1 ⊗ X−1
)
.
¤
(3)
dF
dX
= 1
2
(
Ip2 + Kpp
) (
Ip ⊗ X + X ⊗ Ip
)
for F = X2.
We get
dF = (dX)X + X dX,
d vec F =
(
Ip ⊗ X + X ⊗ Ip
)
d vec X,
d f = D+p
(
Ip ⊗ X + X ⊗ Ip
)
Dp d x.
158 On two matrix derivatives by Kollo and von Rosen
Hence
∂ f
∂x′
= D+p
(
Ip ⊗ X + X ⊗ Ip
)
Dp,
dF
dX
= Dp D
+
p
(
Ip ⊗ X + X ⊗ Ip
)
Dp D
+
p =
= 1
4
(
Ip2 + Kpp
) (
Ip ⊗ X + X ⊗ Ip
) (
Ip2 + Kpp
)
=
= 1
4
(
Ip2 + Kpp
)2 (
Ip ⊗ X + X ⊗ Ip
)
=
= 1
2
(
Ip2 + Kpp
) (
Ip ⊗ X + X ⊗ Ip
)
.
¤
In the following section we shall give some applications of
∂F
∂X
.
5 Some applications of
∂F
∂X
(1)
∂X
∂X
= 1
2
(
vec Ip
) (
vec Ip
)
′
+ 1
2
Kpp.
See von Rosen (1988, Lemma 2.1, d, i).
Derivation: Use
dX
dX
= 1
2
(
Ip2 + Kpp
)
.
See section 4 (1). It is easy to see that C
p
2
vec Ip2 = C
p
2
vec
(
Ip ⊗ Ip
)
= vec Ip⊗vec Ip.
Further C
p
2
vecKpp = vecKpp. For these properties we refer to (ii) and (iii) in the
Appendix.
Hence
vec
∂X
∂X
= 1
2
(
Kpp ⊗ Kpp
) (
vec Ip ⊗ vec Ip + vecKpp
)
= 1
2
(
vec Ip ⊗ vec Ip + vecKpp
)
= 1
2
vec
[(
vec Ip
) (
vec Ip
)
′
+ Kpp
]
,
from which (1) follows.
¤
(2)
∂X−1
∂X
= − 1
2
(
vec X−1
) (
vec X−1
)
′
−
1
2
Kpp
(
X−1 ⊗ X−1
)
.
See von Rosen (1988, Lemma 2.2i).
Heinz Neudecker 159
Derivation: Use
dX−1
dX
= − 1
2
(
Ip2 + Kpp
) (
X−1 ⊗ X−1
)
.
See section 4 (2). Then
vec
∂X−1
∂X
= − 1
2
(
Kpp ⊗ Kpp
)
C
p
2
vec
[(
Ip2 + Kpp
) (
X−1 ⊗ X−1
)]
= − 1
2
(
Kpp ⊗ Kpp
) (
vec X−1 ⊗ vec X−1
)
= − 1
2
(
Kpp ⊗ Kpp
)
vecKpp
(
X−1 ⊗ X−1
)
= − 1
2
vec X−1 ⊗ vec X−1 − 1
2
vecKpp
(
X−1 ⊗ X−1
)
= − 1
2
vec
[(
vec X−1
) (
vec X−1
)
′
]
−
1
2
vecKpp
(
X−1 ⊗ X−1
)
,
from which (2) follows.
We also used property 18 in Neudecker (2000). For a simple proof see (iv) in the
Appendix.
¤
(3)
∂X2
∂X
= 1
2
Kpp
(
Ip ⊗ X + X ⊗ Ip
)
+ 1
2
(vec X)
(
vec Ip
)
′
+ 1
2
(
vec Ip
)
(vec X)′.
Derivation: Use
dX2
dX
= 1
2
(
Ip2 + Kpp
) (
Ip ⊗ X + X ⊗ Ip
)
.
See section 4 (3). Then
vec
∂X2
∂X
= 1
2
(
Kpp ⊗ Kpp
)
C
p
2
vec
[(
Ip2 + Kpp
) (
Ip ⊗ X + X ⊗ Ip
)]
= 1
2
(
Kpp ⊗ Kpp
) (
vec Ip ⊗ vec X + vec X ⊗ vec Ip
)
+ 1
2
(
Kpp ⊗ Kpp
)
vec
[
Kpp
(
Ip ⊗ X + X ⊗ Ip
)]
= 1
2
(
vec Ip ⊗ vec X + vec X ⊗ vec Ip
)
+ 1
2
vec
[
Kpp
(
Ip ⊗ X + X ⊗ Ip
)]
= 1
2
vec
[
(vec X)
(
vec Ip
)
′
+
(
vec Ip
)
(vec X)′
]
+ 1
2
vec
[
Kpp
(
Ip ⊗ X + X ⊗ Ip
)]
.
Devectorization yields (3).
¤
(4)
∂F−1
∂X
= −
(
F−1 ⊗ Ip
) ∂F
∂X
(
F−1 ⊗ Ip
)
, where F = F(X).
See von Rosen (1988, Lemma 2.1, c, iii).
160 On two matrix derivatives by Kollo and von Rosen
Derivation: It is known that dF−1 = −F−1(dF)F−1, hence
dv
(
F−1
)
= D+p d vec F
−1 = −D+p
(
F−1 ⊗ F−1
)
d vec F
= −D+p
(
F−1 ⊗ F−1
)
Dp d v(F) = −D
+
p
(
F−1 ⊗ F−1
)
Dp
∂ f
∂x′
dx
and finally
∂v
(
F−1
)
∂x′
= −D+p
(
F−1 ⊗ F−1
)
Dp
∂ f
∂x′
.
By Corollary 4 and Lemma 1 we have
dF−1
dX
= −Dp D
+
p
(
F−1 ⊗ F−1
)
Dp
∂ f
∂x′
D+p
= − 1
2
(
Ip2 + Kpp
) (
F−1 ⊗ F−1
) dF
dX
= − 1
2
(
F−1 ⊗ F−1
) (
Ip2 + Kpp
) dF
dX
= −
(
F−1 ⊗ F−1
) dF
dX
.
Application of Section 2 (4) yields
vec
∂F−1
∂X
= −
(
Kpp ⊗ Kpp
)
C
p
2
vec
(
F−1 ⊗ F−1
) dF
dX
= −
(
Kpp ⊗ Kpp
)
C
p
2
(
Ip2 ⊗ F
−1
⊗ F−1
)
vec
dF
dX
= −
(
Kpp ⊗ Kpp
)
C
p
2
(
Ip ⊗ Ip ⊗ F
−1
⊗ F−1
)
vec
dF
dX
= −
(
Kpp ⊗ Kpp
) (
Ip ⊗ F
−1
⊗ Ip ⊗ F
−1
)
C
p
2
vec
dF
dX
= −
(
F−1 ⊗ Ip ⊗ F
−1
⊗ Ip
) (
Kpp ⊗ Kpp
)
C
p
2
vec
dF
dX
= −
(
F−1 ⊗ Ip ⊗ F
−1
⊗ Ip
)
vec
∂F
∂X
= −vec
[(
F−1 ⊗ Ip
) ∂F
∂X
(
F−1 ⊗ Ip
)]
.
Hence
∂F−1
∂X
= −
(
F−1 ⊗ Ip
) ∂F
∂X
(
F−1 ⊗ Ip
)
.
¤
Heinz Neudecker 161
6 Appendix
The following matrix properties have been used in this article. The first five involve
C
p
2
= Ip ⊗ Kpp ⊗ Ip.
(i) C
p
2
(A ⊗ B ⊗C ⊗ D)C
p
2
= A ⊗C ⊗ B ⊗ D, for (p × p) A, B,C,D.
(ii) C
p
2
vec (A ⊗ B) = vec A ⊗ vec B, for (p × p) A and B.
Proof.
C
p
2
vec (A ⊗ B) =
∑
i j
(
Ip ⊗ Ei j ⊗ E ji ⊗ Ip
)
vec (A ⊗ B)
=
∑
i j
vec
[(
E ji ⊗ Ip
)
(A ⊗ B)
(
Ip ⊗ E ji
)]
=
∑
i j
vec
(
E ji A ⊗ BE ji
)
=
∑
i j
vec
(
e j Ai. ⊗ B. j e
′
i
)
=
∑
i j
vec
[(
e j ⊗ B. j
) (
Ai. ⊗ e
′
i
)]
=
= vec


∑
j
vec B. j e
′
j


∑
i
vec ei Ai.

′

= vec
[
(vec B) (vec A)′
]
= vec A ⊗ vec B.
As usual Ai. is the i
th row of A, A. j is the j
th column of A.
¤
(iii) C
p
2
vecKpp = vecKpp.
Proof.
C
p
2
vecKpp =
∑
i j
C
p
2
vec
(
Ei j ⊗ E ji
)
=
∑
i j
(
vec Ei j ⊗ vec E ji
)
= vec
∑
i j
(
vec E ji
) (
vec Ei j
)
′
= vec
∑
i j
(
ei ⊗ e j
) (
e′j ⊗ e
′
i
)
= vec
∑
i j
(
ei e
′
j ⊗ e j e
′
i
)
= vec
∑
i j
(
Ei j ⊗ E ji
)
= vecKpp.
¤
(iv) C
p
2
vecKpp (A ⊗ B) = vecKpp (A ⊗ B
′) for (p × p) A and B.
162 On two matrix derivatives by Kollo and von Rosen
Proof.
C
p
2
vecKpp (A ⊗ B) = C
p
2
(
A′ ⊗ B′ ⊗ Ip ⊗ Ip
)
vecKpp
=
(
A′ ⊗ Ip ⊗ B
′
⊗ Ip
)
C
p
2
vecKpp =
(
A′ ⊗ Ip ⊗ B
′
⊗ Ip
)
vecKpp
= vec
(
B′ ⊗ Ip
)
Kpp
(
A ⊗ Ip
)
= vecKpp
(
Ip ⊗ B
′
) (
A ⊗ Ip
)
= vecKpp (A ⊗ B
′) .
¤
(v) Kp2, p2 = C
p
2
(
Kpp ⊗ Kpp
)
C
p
2
.
Proof. Consider the
(
p2, p2
)
matrix
Q =
∑
kl
(Ekl ⊗ Qkl)
or equivalently
Q = [Qkl] (k, l = 1, . . . , p)
with (p × p) matrix Qkl.
Then Kp2, p2 vecQ = vecQ
′ and
C
p
2
(
Kpp ⊗ Kpp
)
C
p
2
vecQ =
∑
i jst
C
p
2
(
Ei j ⊗ E ji ⊗ Est ⊗ Ets
)
C
p
2
vecQ
=
∑
i jstkl
(
Ei j ⊗ Est ⊗ E ji ⊗ Ets
)
vec (Ekl ⊗ Qkl)
= vec
∑
i jstkl
(
E ji ⊗ Ets
)
(Ekl ⊗ Qkl)
(
E ji ⊗ Ets
)
= vec
∑
i jstkl
(
E ji Ekl E ji ⊗ Ets Qkl Ets
)
= vec
∑
i jst
(
E ji ⊗ Ets Qi j Ets
)
= vec
∑
i jst
(
Qi j
)
st
(
Ei j ⊗ Est
)
′
= vec
∑
i j
(
Ei j ⊗ Qi j
)
′
= vecQ′,
where
(
Qi j
)
st
is the (s, t) element of Qi j.
¤
We further used the standard properties:
(vi) Kpp vec A = vec A
′ for (p × p) matrix A.
Heinz Neudecker 163
(vii) Dp D
+
p =
1
2
(
Ip2 + Kpp
)
, D+p Dp = Ip∗ with 2p
∗ = p(p + 1).
(viii) vec A BC = (C′ ⊗ A) vec B for compatible matrices A, B and C.
7 References
Ghazal, G. A. and Neudecker, H. (2000). On second-order and fourth-order moments of jointly distributed
random matrices: a survey. Linear Algebra and its Applications, 321, 61-93.
Kollo, T. and von Rosen, D. (2000). Distribution and density approximation of the covariance matrix in the
growth curve model. Statistics, 35, 1-22.
Magnus, J. R. and Neudecker, H. (1980). The elimination matrix: some lemmas and applications. SIAM
Journal on Algebraic and Discrete Methods, 1, 422-449.
Magnus, J. R. and Neudecker, H. (1999). Matrix Differential Calculus with Applications in Statistics and
Econometrics, revised edition. John Wiley, Chichester, England.
Neudecker, H. (2000). On expected values of fourth-degree matrix products of a multinormal matrix
variate. New Trends in Probability and Statistics. Proceedings of the 6th Tartu Conference on
Multivariate Statistics. TEV Vilnius/Utrecht.
Neudecker, H. and Wansbeek, T.
(1983). Some results on commutation matrices with statistical applications. Canadian Journal of
Statistics, 11, 221-31.
von Rosen, D. (1988). Moments for the inverted Wishart distribution. Scandinavian Journal of Statistics,
15, 97-109.
Resum
Aquest article estableix les relacions entre las derivades matricials de F respecte de X introduı¨des per
von Rosen (1988), Kollo i von Rosen (2000) i les derivades matricials de Magnus i Neudecker (1999).
Les operacions vectorials de duplicacio´ i transformacio´ en vectors so´n les usuals i les inverses de las
matrius duplicades so´n les de Moore-Penrose. Ambdues X i F = F(X) tenen la mateixa dimensio´.
MSC: primary 15A69, secondary 62H10
Paraules clau: Derivades i diferencials de matrius, vectoritzacio´, commutacio´ i duplicacio´ de matrius
Hola


On two matrix derivatives by Kollo and von Rosen

Abstract

Similar works

Full text

Available Versions

UPCommons. Portal del coneixement obert de la UPC

Diposit Digital de Documents de la UAB

UPCommons

Revistes Catalanes amb Accés Obert

Secretaría de Estado de Cultura