This paper gives a generalization of results presented by ten Berge, Krijnen, Wansbeek & Shapiro. They examined procedures and results as proposed by Anderson & Rubin, McDonald, Green and Krijnen, Wansbeek & ten Berge. We shall consider the same matter, under weaker rank assumptions. We allow some moments, namely the variance Ω of the observable scores vector and that of the unique factors Ψ to be singular. We require T'Ψ T > 0 where T Λ T' is a Schur decomposition of Ω. As usual the variance of the common factors Φ, and the loadings matrix A will have full column rank

Neudecker, Heinz

Revistes Catalanes amb Accés Obert

Statistics & Operations Research Transactions
SORT 28 (1) January-June 2004, 27-36
Statistics &
Operations Research
Transactions
On best affine unbiased covariance-preserving
prediction of factor scores
Heinz Neudecker∗
University of Amsterdam, The Netherlands
Abstract
This paper gives a generalization of results presented by ten Berge, Krijnen, Wansbeek & Shapiro.
They examined procedures and results as proposed by Anderson & Rubin, McDonald, Green and
Krijnen, Wansbeek & ten Berge. We shall consider the same matter, under weaker rank assumptions.
We allow some moments, namely the variance Ω of the observable scores vector and that of the
unique factors, Ψ, to be singular. We require T ′ ΨT > 0, where TΛT ′ is a Schur decomposition of Ω.
As usual the variance of the common factors, Φ, and the loadings matrix A will have full column rank.
MSC: 62H25, 15A24
Keywords: Factor analysis, factor scores, covariance-preserving, Kristof-type theorem
1 Introduction
We consider the factor model y = µy + A f + ε, where y is a p × 1 vector of observable
random variables called «scores», f is an m × 1 vector of non-observable random
variables called «common factors», A is a p × m matrix of full column rank whose
elements are called «factor loadings» and ε is a p × 1 vector of non-observable random
variables called «unique factors». The usual moment definitions and assumptions are
E(ε) = 0, E( f ) = 0, E(y) = µy, D(ε) = Ψ, D( f ) = Φ, C( f , ε) = 0.
∗ Address for correspondence: Oosterstraat, 13. 1741 GH Schagen. The Netherlands. Postal address: NL 1741 GH
13. E-mail: H.Neudecker@uva.nl
Received: December 2001
Accepted: September 2003
28 On best affine unbiased covariance-preserving prediction of factor scores
This yields the moment structure
Ω = AΦA′ + Ψ,
where Ω = D(y) and Ψ can be singular, Φ and A have full column rank.
Notice that
M(A) ⊂ M(Ω). (1.1)
The following additional assumption is made:
T ′ΨT > 0.
It is inspired by the Schur decompositionΩ = TΛT ′, with T ′T = Ir and diagonalΛ > 0.
Obviously p > r > m.
In two recent publications Krijnen, Wansbeek & ten Berge (1996) and ten Berge,
Krijnen, Wansbeek & Shapiro (1999) studied the problem of best linear prediction of
f given y, subject to the constraint E ˆf ˆf ′ = E f f ′, where ˆf = B′y is their predictor
function. Vectors f and y have a simultaneous distribution. The two expectations are
taken with respect to this distribution.
The constraint E ˆf ˆf ′ = E f f ′ is mistakenly referred to as «correlation-preserving».
We shall call it «covariance-preserving», although at face value only the RHS expression
is a variance matrix. We shall use an affine predictor function ˆf = a + B′y. It will be
shown that a+B′µy = 0. Hence the predictor function will become ˆf = B′(y−µy) which
is linear and unbiased. Consequently the LHS expression will become a variance matrix.
In their article ten Berge et al. (1999) examine three prediction procedures, due
to McDonald (1981) —who generalized a procedure proposed by Anderson & Rubin
(1956)—, Green (1969) and Krijnen et al. (1996), respectively.
We shall consider the same three procedures. The second and third are based on the
mean-squared-error matrix M = E
(
ˆf − f
) (
ˆf − f
)′
. Where Green minimizes its trace,
tr M, Krijnen et al. minimize its determinant, |M|. McDonald uses a different though
related criterion trΨ−1E
(
y − µy − A ˆf
) (
y − µy − A ˆf
)′
which he minimizes. Note that
these authors assume Ψ > 0, hence Ω > 0. ten Berge et al. conclude that McDonald’s
and Krijnen et al.’s solutions for B coincide.
In the present paper we shall again consider the above-mentioned procedures, under
weaker rank assumptions. We shall show that the MSE matrix M is positive definite.
Minimization of the trace and the determinant of M yields immediately a + B′µy = 0.
Minimization of McDonald’s criterion function yields the same result. As mathematical
methods we use a Kristof-type theorem and a matrix inequality developed by Zhang
(1999). Finally we show that 1) ˆfG, the Green predictor and ˆfK , the Krijnen et al.
predictor coincide when Φ and A′Ω+A commute, 2) ˆfM, the McDonald predictor and
ˆfK coincide when Ψ and AΦA′ commute.
Heinz Neudecker 29
2 A Kristof-type theorem
Two of the three criterion functions can be seen to belong to the class tr P′X, where P
and X have dimension p × m. The constant matrix P has rank q. The variable matrix X
satisfies the constraint X′X = Im. The aim is to maximize tr P′X subject to X′X = Im.
Define then the Lagrangean function
ϕ(X) = tr P′X − 12 tr L
(
X′X − Im
)
,
where L is a symmetric matrix of multipliers. Symmetry of L is vital. It is justified, of
course, by the symmetry of the constraint.
The differential of the function, namely
dϕ = tr P′dX − tr LX′dX = tr (P − XL)′ dX
has to be zero. This yields the equations
P = XL (2.1)
X′X = Im (2.2)
From these we obtain
P′P = L2 (2.3)
P = X
(
P′P
) 1
2 (2.4)
Which square root will be selected is still undecided. Consider equation (2.4). As
P
(
P′P
)+ 12 (P′P) 12 = P
it is consistent. The symbol «+» denotes the Moore-Penrose inverse. The symbols «+»
and « 12» are interchangeable in (P′P)+
1
2
. The general solution of (2.4) is
X◦ = P
(
P′P
)+ 12 + Q − Q (P′P) 12 (P′P)+ 12 , Q arbitrary (2.5)
When we use the singular-value decomposition P = F1Γ
1
2
1 G
′
1, with F
′
1F1 = G
′
1G1 = Iq
and (diagonal) Γ
1
2
1 > 0, we can write the solution as
X◦ = F1G′1 + Q
(
Im −G1G′1
) (2.6)
It follows from (2.5) that
tr P′X◦ = tr
(
P′P
) 1
2 . (2.7)
As we look for a maximum, we have to take the positive definite square root (P′P) 12 . The
solution X◦ is not unique, unless q = m. In that case it can be written as
X◦ = P
(
P′P
)− 12 = F1G′1 (2.8)
30 On best affine unbiased covariance-preserving prediction of factor scores
For the connaisseurs we shall examine the second differential
d2ϕ = −tr(dX)L(dX)′ (2.9)
When this expression is negative for all dX , 0 satisfying (dX)′X◦ + X′◦dX = 0, a
maximum has been found. The choice L = (P′P) 12 > 0 guarantees this.
3 The Green procedure
As stated we use the MSE matrix M = E
(
ˆf − f
) (
ˆf − f
)′
=
(
a + B′µy
) (
a + B′µy
)′
+
B′ΩB + Φ − B′AΦ − ΦA′B. Obviously a + B′µy = 0, as we have to minimize trM. As
a consequence E ˆf ˆf ′ = B′ΩB. Imposition of the constraint E ˆf ˆf ′ = E f f ′ yields then
M = 2Φ − B′AΦ − ΦA′B. Green (1969) defines the problem:
min
B
tr
(
2Φ − B′AΦ − ΦA′B
)
subject to B′ΩB = Φ.
We introduce C′ = Φ− 12 B′Ω 12 . Clearly C′C = Im. This yields the equivalent problem
max
C
trΦ
3
2 A′Ω+
1
2 C subject to C′C = Im
We used: A′Ω+ 12 CΦ 12 = R′Ω 12Ω+ 12Ω 12 B = R′Ω 12 B = A′B, with A = Ω 12 R due to (1.1).
Application of the Kristof-type theorem gives the solution
CG = Ω+
1
2 AΦ
3
2
(
Φ
3
2 A′Ω+AΦ
3
2
)− 12
,
from which follows the solution
BG = Ω+AΦ
3
2
(
Φ
3
2 A′Ω+AΦ
3
2
)− 12
Φ
1
2 +
(
Ip −Ω+
1
2Ω
1
2
)
Q, Q arbitrary.
The arbitrary component disappears in the predictor expression B′G(y − µy), because(
Ip −Ω
1
2Ω
+
1
2
) (
y − µy
)
= 0 with probability one (w. p. 1).
Hence we get as predictor
ˆfG = Φ 12
(
Φ
3
2 A′Ω+AΦ
3
2
)− 12
Φ
3
2 A′Ω+
(
y − µy
)
.
The reader can verify that A′Ω+A > 0.
An alternative expression is
CG = F2G′2,
where we have used the singular-value decomposition
Ω
+
1
2 AΦ
3
2 = F2Γ
1
2
2 G
′
2,
with F′2F2 = G
′
2G2 = G2G
′
2 = Im. Use was made of the fact that Ω+
1
2 A has full column
rank (m).
Heinz Neudecker 31
For nonsingular Ω the solution becomes that given by ten Berge et al. (1999) in their
presentation, namely between (6) and (7).
4 The McDonald procedure
This approach is based on the weighted-least-squares function
trΨ+E
(
y − µy − A ˆf
) (
y − µy − A ˆf
)′
.
Clearly
E
(
y − µy − A ˆf
) (
y − µy − A ˆf
)′
=
(
Ip − AB′
)
Ω
(
Ip − BA′
)
+
+A
(
a + B′µy
) (
a + B′µy
)′
A′.
Again we find that a + B′µy = 0, now having to minimize
trΨ+E
(
y − µy − A ˆf
) (
y − µy − A ˆf
)′
.
Notice that A′Ψ+A > 0.
Imposition of the constraint E ˆf ˆf ′ = E f f ′ leads to the problem of minimizing
trΨ+
(
Ip − AB′
)
Ω
(
Ip − BA′
)
subject to B′ΩB = Φ.
Using C′ = Φ− 12 B′Ω 12 we define the problem:
max
C
trΦ
1
2 A′Ψ+Ω
1
2 C subject to C′C = Im.
Application of the Kristof-type theorem yields the solution
CM = Ω
1
2Ψ
+AΦ
1
2
(
Φ
1
2 A′Ψ+ΩΨ+AΦ
1
2
)− 12
,
from which follows the solution
BM = Ω+
1
2Ω
1
2Ψ
+AΦ
1
2
(
Φ
1
2 A′Ψ+ΩΨ+AΦ
1
2
)− 12
Φ
1
2 +
(
Ip −Ω+
1
2Ω
1
2
)
Q, Q arbitrary.
Finally the predictor turns out to be
ˆfM = Φ 12
(
Φ
1
2 A′Ψ+ΩΨ+AΦ
1
2
)− 12
Φ
1
2 A′Ψ+
(
y − µy
)
.
Again we used (
Ip −Ω
1
2Ω
+
1
2
) (
y − µy
)
= 0 w.p.1.
The reader can verify that A′Ψ+ΩΨ+A > 0, using
A′Ψ+ΩΨ+A = A′Ψ+
(
AΦA′ + Ψ
)
Ψ
+A = A′Ψ+AΦA′Ψ+A + A′Ψ+A.
32 On best affine unbiased covariance-preserving prediction of factor scores
An alternative expression is
CM = F3G
′
3,
where
Ω
1
2Ψ
+AΦ
1
2 = F
1
2
3 Γ
1
2
3 G
′
3,
with F′3F3 = G
′
3G3 = G3G
′
3 = Im.
For nonsingular Ω the solution becomes that given by ten Berge et al. (1999) in their
presentation, namely between (4) and (5).
5 The Krijnen et al. procedure
Like Green’s this approach uses the MSE matrix M of ˆf . Instead of tr (2Φ − B′AΦ−
ΦA′B), Krijnen et al. use |2Φ − B′AΦ − ΦA′B| which has to be minimized. The first
thing to do is to prove that 2Φ − B′AΦ − ΦA′B > 0.
We have
2Φ − B′AΦ − ΦA′B = Φ 12
(
2Im − Φ−
1
2 B′AΦ 12 − Φ 12 A′BΦ− 12
)
Φ
= Φ
1
2
(
2Im − Φ−
1
2 B′Ω 12Ω+ 12 AΦ 12 − Φ 12 A′Ω+ 12Ω 12 BΦ− 12
)
Φ
1
2
= Φ
1
2 (2Im −C′V − V ′C)Φ 12
= Φ
1
2
[(C − V)′ (C − V) + (Im − V ′V)]Φ 12
where
V = Ω+
1
2 AΦ
1
2 and hence V ′V = Φ 12 A′Ω+AΦ 12 .
We shall show that all eigenvalues of V ′V are positive and less than unity. Pre-(post-)
multiply the moment structure Ω = AΦA′ + Ψ by Φ 12 A′Ω+
(
Ω
+AΦ 12
)
. This leads to
Φ
1
2 A′Ω+AΦ 12 =
(
Φ
1
2 A′Ω+AΦ 12
)2
+ Φ
1
2 A′Ω+ΨΩ+AΦ 12 , hence
Φ
1
2 A′Ω+AΦ
1
2 >
(
Φ
1
2 A′Ω+AΦ
1
2
)2
,
as Φ
1
2 A′Ω+ΨΩ+AΦ 12 > 0, and λi > λ2i where λi is any eigenvalue of Φ
1
2 A′Ω+AΦ 12 . This
proves the property. Hence Im − V ′V > 0. As (C − V)′(C − V) > 0 we have shown that
2Φ − B′AΦ − ΦA′B > 0.
Hence |2Φ − B′AΦ − ΦA′B| > 0. Consider then the positive definite matrix 2Im −
C′V − V ′C. We use (7.18) in Zhang (1999) which yields
C′V + V ′C 6 2U′
(
V ′CC′V
) 1
2 U
where U is an orthogonal matrix.
As C′C = Im we have CC′ 6 Ip. This in its turn leads to V ′CC′V 6 V ′V . The latter
inequality gives (V ′CC′V) 12 6 (V ′V) 12 . See Theorem 2.5.5 in Wang & Chow (1994).
Heinz Neudecker 33
Finally, we have
C′V + V ′C 6 2U′
(
V ′V
) 1
2 U
or equivalently
2Im −C′V − V ′C > 2
[
Im − U′
(
V ′V
) 1
2 U
]
.
From this we derive
∣∣∣2Im −C′V − V ′C
∣∣∣ >
∣∣∣∣∣2
[
Im − U′
(
V ′V
) 1
2 U
]∣∣∣∣∣ =
∣∣∣∣∣2
[
Im −
(
V ′V
) 1
2
]∣∣∣∣∣ .
It is easy to see that CK = V(V ′V)− 12 leads to the equality
∣∣∣2Im −C′KV − V ′CK
∣∣∣ =
∣∣∣∣∣2
[
Im −
(
V ′V
) 1
2
]∣∣∣∣∣ .
Hence CK solves the problem. It is not clear whether the solution is unique.
In fact, CK also solves the related problem
max
C
tr V ′C subject to C′C = Im.
The (unique) solution is CK by the Kristof-type theorem.
Application of Zhang’s (7.18) yields
2 tr V ′C = tr
(
C′V + V ′C
)
6 2 tr U′
(
V ′V
) 1
2 U = 2 tr
(
V ′V
) 1
2 ,
which again has solution CK . We then get the solution
BK = Ω+AΦ
1
2
(
Φ
1
2 A′Ω+AΦ
1
2
)− 12
Φ
1
2 +
(
Ip −Ω+
1
2Ω
1
2
)
Q, Q arbitrary.
From this follows the unique predictor
ˆfK = Φ 12
(
Φ
1
2 A′Ω+AΦ
1
2
)− 12
Φ
1
2 A′Ω+
(
y − µy
)
.
For nonsingular Ω the solution CK coincides with that given by ten Berge et al. (1999),
namely in (9).
6 Equality of ˆfG and ˆfK when Φ and A′Ω+A commute
ten Berge et al. (1999) showed that CG = CK under their assumptions when Φ and
A′Ω−1A commute. We shall prove that ˆfG = ˆfK under our milder conditions.
When Φ and A′Ω+A commute we have Φ = S MS ′ and A′Ω+A = S NS ′, where M
and N are positive definite diagonal matrices and S is orthogonal. Hence
Φ
3
2
(
Φ
3
2 A′Ω+AΦ 32
)− 12
= S M 32 S ′
(
S M 32 S ′S NS ′S M 32 S ′
)− 12
= S M 32 S ′
(
S M3NS ′
)− 12
= S M 32 S ′S
(
M3N
)− 12 S ′
= S N− 12 S ′ = (A′Ω+A)− 12 .
34 On best affine unbiased covariance-preserving prediction of factor scores
Further Φ 12
(
Φ
1
2 A′Ω+AΦ 12
)− 12
= (A′Ω+A)− 12 .
This yields ˆfG = ˆfK = Φ 12 (A′Ω+A)− 12 A′Ω+
(
y − µy
)
.
7 Equality of ˆfM and ˆfK when Ψ and AΦA′ commute
ten Berge et al. (1999) showed that CM = CK under their assumptions when Ψ is
nonsingular. Essential is the expression
Ω
−1
= Ψ
−1 − Ψ−1AΦ
1
2
(
Im + Φ
1
2 A′Ψ−1AΦ
1
2
)−1
Φ
1
2 A′Ψ−1.
Under our assumptions Im + Φ
1
2 A′Ψ+AΦ 12 is nonsingular because A′Ψ+A > 0 which
follows from T ′ΨT > 0 and (1.1). When we additionally assume that Ψ and AΦA′
commute we can establish the equality
Ω
+
= Ψ
+ − Ψ+AΦ
1
2
(
Im + Φ
1
2 A′Ψ+AΦ
1
2
)−1
Φ
1
2 A′Ψ+.
Proof. When Ψ and AΦA′ commute we have Ψ = S MS ′ and AΦA′ = S NS ′ where
M and N are positive definite diagonal matrices and S ′S = Im. Further AΦ
1
2 = S N 12 T ′,
with orthogonal T , a singular-value decomposition. Hence
Ψ
+ − Ψ+AΦ 12
(
Im + Φ
1
2 A′Ψ+AΦ 12
)−1
Φ
1
2 A′Ψ+
= S M−1S ′ − S M−1S ′S N 12 T ′
(
Im + T N
1
2 S ′S M−1S ′S N 12 T ′
)−1
T N 12 S ′S M−1S ′
= S M−1S ′ − S M−1N 12 T ′
(
Im + T M−1NT ′
)−1
T M−1N 12 S ′
= S M−1S ′ − S M−1N 12 T ′T
(
Im + M−1N
)−1
T ′T M−1N 12 S ′
= S M−1S ′ − S M−1N 12
(
Im + M−1N
)−1
M−1N 12 S ′.
Further Ω = AΦA′ + Ψ = S (M + N)S ′, and Ω+ = S (M + N)−1S ′. It is easy to see that
(M + N)−1 = M−1 − M−1N 12
(
Im + M−1N
)−1
M−1N
1
2 .
This yields the result. ¤
Recall that
ˆfM = Φ 12
(
Φ
1
2 A′Ψ+ΩΨ+AΦ
1
2
)− 12
Φ
1
2 A′Ψ+(y − µy)
and
ˆfK = Φ 12
(
Φ
1
2 A′Ω+AΦ
1
2
)− 12
Φ
1
2 A′Ω+(y − µy).
Heinz Neudecker 35
Consider
Φ
1
2 A′Ω+AΦ 12 = Φ 12 A′Ψ+AΦ 12 − Φ 12 A′Ψ+AΦ 12
(
Im + Φ
1
2 A′Ψ+AΦ 12
)−1
× Φ
1
2 A′Ψ+AΦ 12
=
(
Im + Φ
1
2 A′Ψ+AΦ 12
)−1
Φ
1
2 A′Ψ+AΦ 12
= (Im + E)−1 E,
Φ
1
2 A′Ψ+ΩΨ+AΦ 12 = Φ 12 A′Ψ+AΦA′Ψ+AΦ 12 + Φ 12 A′Ψ+AΦ 12
= Φ
1
2 A′Ψ+AΦ 12 +
(
Φ
1
2 A′Ψ+AΦ 12
)2
= E + E2,
Φ
1
2 A′Ω+ = Φ 12 A′Ψ+ − Φ 12 A′Ψ+AΦ 12
(
Im + Φ
1
2 A′Ψ+AΦ 12
)−1
Φ
1
2 A′Ψ+
=
(
Im + Φ
1
2 A′Ψ+AΦ 12
)−1
Φ
1
2 A′Ψ+
= (Im + E)−1Φ 12 A′Ψ+,
ˆfK = Φ 12
[
(Im + E)−1 E
]− 12 (Im + E)−1Φ 12 A′Ψ+
(
y − µy
)
,
ˆfM = Φ 12
(
E + E2
)− 12
Φ
1
2 A′Ψ+
(
y − µy
)
.
Clearly [
(Im + E)−1 E
]− 12 (Im + E)−1 =
(
E + E2
)− 12
as E > 0.
This establishes the equality of ˆfK and ˆfM.
8 Comments
1. ten Berge et al. (1999) claim that the McDonald method is undefined when Ψ is
singular. This is unjustified. What matters is the nonsingularity of T ′ΨT . We make
that assumption. It implies that A′Ψ+A > 0 which we use several times.
2. Application of Zhang’s result shows immediately that CG and CM yield the
maximum. The Kristof-type theorem shows the unicity of the solutions.
Acknowledgement
The author is grateful to Go¨tz Trenkler for drawing his attention to Zhang’s result (7.18)
which was fruitfully applied in Section 5. The reasoning why B′µy + a should be equal
36 On best affine unbiased covariance-preserving prediction of factor scores
to zero in all three procedures is due to Albert Satorra. This yields ˆf = B′
(
y − µy
)
, an
unbiased predictor.
9 References
Anderson, T. W. & Rubin, H. (1956). Statistical inference in factor analysis. Proceedings of the Third
Berkeley Symposium on Mathematical Statistics and Probability. Vol. V, 111-50. University of
California Press, Berkeley and Los Angeles.
ten Berge, J. M. F. (1993). Least Squares Optimization in Multivariate Analysis. DSWO Press, Leiden,
Netherlands.
ten Berge, J. M. F., Krijnen, W. P., Wansbeek, T. & Shapiro, A. (1999). Some new results on correlation-
preserving factor scores prediction methods. Linear Algebra Appl., 289, 311-8.
Green, B. F. (1969). Best linear composites with a specified structure. Psychometrika, 34, 301-18.
Krijnen, W. P., Wansbeek, T. J. & ten Berge, J. M. F. (1996). Best linear predictors for factor scores.
Communications in Statistics: Theory and Methods, 25, 3013-25.
Kristof, W. (1970). A theorem on the trace of certain matrix products and some applications. Journal of
Mathematical Psychology, 7, 515-30.
Lo¨wner, K. (1934). ¨Uber monotone Matrixfunktionen. Mathematische Zeitschrift, 38, 177-216.
Magnus, J. R. & Neudecker, H. (1999). Matrix Differential Calculus with Applications in Statistics and
Econometrics. Revised Edition. Wiley, Chichester, England.
McDonald, R. P. (1981). Constrained least squares estimators of oblique common factors. Psychometrika,
46, 337-41.
Wang, S. G. & Chow, S. C. (1994). Advanced Linear Models: Theory and Applications. Marcel Dekker,
New York.
Zhang, F. (1999). Matrix Theory. Basic Results and Techniques. Springer, New York.
Resum
Es do´na una generalitzacio´ dels resultats presentats per ten Berge, Krijnen, Wansbeek and Shapiro .
Aquests autors examinen me`todes i resultats basats en Anderson i Rubin. Mc Donald, Green i Krijnen,
Wansbeek i ten Berge. Considerarem el mateix plantejament pero` sota condicions de rang me´s de`bils.
Aixı´ suposarem que alguns moments, com les matrius de covaria`ncies Ω del vector de mesures
observades dels factors comuns i ψ dels factors u´nics, siguin singulars. Imposem la condicio´ T′ψT > 0,
essent TΛT′ la descomposicio´ de Schur de Ω. Com e´s usual, suposem que tenen rang ma`xim per
columnes les matrius de covaria`ncies Φ dels factors comuns i la matriu A del model factorial.
MSC: 62H25, 15A24
Paraules clau: ana`lisi factorial, mesures de factors, preservacio´ de la covaria`ncia, teorema tipus Kristof


On best affine unbiased covariance-preserving prediction of factor scores

Diposit Digital de Documents de la UAB

Statistics & Operations Research TransactionsSORT 28 (1) January-June 2004, 27-36Statistics &Operations ResearchTransactionsOn best affine unbiased covariance-preservingprediction of factor scoresHeinz Neudecker∗University of Amsterdam, The NetherlandsAbstractThis paper gives a generalization of results presented by ten Berge, Krijnen, Wansbeek & Shapiro.They examined procedures and results as proposed by Anderson & Rubin, McDonald, Green andKrijnen, Wansbeek & ten Berge. We shall consider the same matter, under weaker rank assumptions.We allow some moments, namely the variance Ω of the observable scores vector and that of theunique factors, Ψ, to be singular. We require T ′ ΨT > 0, where TΛT ′ is a Schur decomposition of Ω.As usual the variance of the common factors, Φ, and the loadings matrix A will have full column rank.MSC: 62H25, 15A24Keywords: Factor analysis, factor scores, covariance-preserving, Kristof-type theorem1 IntroductionWe consider the factor model y = µy + A f + ε, where y is a p × 1 vector of observablerandom variables called «scores», f is an m × 1 vector of non-observable randomvariables called «common factors», A is a p × m matrix of full column rank whoseelements are called «factor loadings» and ε is a p × 1 vector of non-observable randomvariables called «unique factors». The usual moment definitions and assumptions areE(ε) = 0, E( f ) = 0, E(y) = µy, D(ε) = Ψ, D( f ) = Φ, C( f , ε) = 0.∗ Address for correspondence: Oosterstraat, 13. 1741 GH Schagen. The Netherlands. Postal address: NL 1741 GH13. E-mail: H.Neudecker@uva.nlReceived: December 2001Accepted: September 200328 On best affine unbiased covariance-preserving prediction of factor scoresThis yields the moment structureΩ = AΦA′ + Ψ,where Ω = D(y) and Ψ can be singular, Φ and A have full column rank.Notice thatM(A) ⊂ M(Ω). (1.1)The following additional assumption is made:T ′ΨT > 0.It is inspired by the Schur decompositionΩ = TΛT ′, with T ′T = Ir and diagonalΛ > 0.Obviously p > r > m.In two recent publications Krijnen, Wansbeek & ten Berge (1996) and ten Berge,Krijnen, Wansbeek & Shapiro (1999) studied the problem of best linear prediction off given y, subject to the constraint E ˆf ˆf ′ = E f f ′, where ˆf = B′y is their predictorfunction. Vectors f and y have a simultaneous distribution. The two expectations aretaken with respect to this distribution.The constraint E ˆf ˆf ′ = E f f ′ is mistakenly referred to as «correlation-preserving».We shall call it «covariance-preserving», although at face value only the RHS expressionis a variance matrix. We shall use an affine predictor function ˆf = a + B′y. It will beshown that a+B′µy = 0. Hence the predictor function will become ˆf = B′(y−µy) whichis linear and unbiased. Consequently the LHS expression will become a variance matrix.In their article ten Berge et al. (1999) examine three prediction procedures, dueto McDonald (1981) —who generalized a procedure proposed by Anderson & Rubin(1956)—, Green (1969) and Krijnen et al. (1996), respectively.We shall consider the same three procedures. The second and third are based on themean-squared-error matrix M = E(ˆf − f) (ˆf − f)′. Where Green minimizes its trace,tr M, Krijnen et al. minimize its determinant, |M|. McDonald uses a different thoughrelated criterion trΨ−1E(y − µy − A ˆf) (y − µy − A ˆf)′which he minimizes. Note thatthese authors assume Ψ > 0, hence Ω > 0. ten Berge et al. conclude that McDonald’sand Krijnen et al.’s solutions for B coincide.In the present paper we shall again consider the above-mentioned procedures, underweaker rank assumptions. We shall show that the MSE matrix M is positive definite.Minimization of the trace and the determinant of M yields immediately a + B′µy = 0.Minimization of McDonald’s criterion function yields the same result. As mathematicalmethods we use a Kristof-type theorem and a matrix inequality developed by Zhang(1999). Finally we show that 1) ˆfG, the Green predictor and ˆfK , the Krijnen et al.predictor coincide when Φ and A′Ω+A commute, 2) ˆfM, the McDonald predictor andˆfK coincide when Ψ and AΦA′ commute.Heinz Neudecker 292 A Kristof-type theoremTwo of the three criterion functions can be seen to belong to the class tr P′X, where Pand X have dimension p × m. The constant matrix P has rank q. The variable matrix Xsatisfies the constraint X′X = Im. The aim is to maximize tr P′X subject to X′X = Im.Define then the Lagrangean functionϕ(X) = tr P′X − 12 tr L(X′X − Im),where L is a symmetric matrix of multipliers. Symmetry of L is vital. It is justified, ofcourse, by the symmetry of the constraint.The differential of the function, namelydϕ = tr P′dX − tr LX′dX = tr (P − XL)′ dXhas to be zero. This yields the equationsP = XL (2.1)X′X = Im (2.2)From these we obtainP′P = L2 (2.3)P = X(P′P) 12 (2.4)Which square root will be selected is still undecided. Consider equation (2.4). AsP(P′P)+ 12 (P′P) 12 = Pit is consistent. The symbol «+» denotes the Moore-Penrose inverse. The symbols «+»and « 12» are interchangeable in (P′P)+12. The general solution of (2.4) isX◦ = P(P′P)+ 12 + Q − Q (P′P) 12 (P′P)+ 12 , Q arbitrary (2.5)When we use the singular-value decomposition P = F1Γ121 G′1, with F′1F1 = G′1G1 = Iqand (diagonal) Γ121 > 0, we can write the solution asX◦ = F1G′1 + Q(Im −G1G′1) (2.6)It follows from (2.5) thattr P′X◦ = tr(P′P) 12 . (2.7)As we look for a maximum, we have to take the positive definite square root (P′P) 12 . Thesolution X◦ is not unique, unless q = m. In that case it can be written asX◦ = P(P′P)− 12 = F1G′1 (2.8)30 On best affine unbiased covariance-preserving prediction of factor scoresFor the connaisseurs we shall examine the second differentiald2ϕ = −tr(dX)L(dX)′ (2.9)When this expression is negative for all dX , 0 satisfying (dX)′X◦ + X′◦dX = 0, amaximum has been found. The choice L = (P′P) 12 > 0 guarantees this.3 The Green procedureAs stated we use the MSE matrix M = E(ˆf − f) (ˆf − f)′=(a + B′µy) (a + B′µy)′+B′ΩB + Φ − B′AΦ − ΦA′B. Obviously a + B′µy = 0, as we have to minimize trM. Asa consequence E ˆf ˆf ′ = B′ΩB. Imposition of the constraint E ˆf ˆf ′ = E f f ′ yields thenM = 2Φ − B′AΦ − ΦA′B. Green (1969) defines the problem:minBtr(2Φ − B′AΦ − ΦA′B)subject to B′ΩB = Φ.We introduce C′ = Φ− 12 B′Ω 12 . Clearly C′C = Im. This yields the equivalent problemmaxCtrΦ32 A′Ω+12 C subject to C′C = ImWe used: A′Ω+ 12 CΦ 12 = R′Ω 12Ω+ 12Ω 12 B = R′Ω 12 B = A′B, with A = Ω 12 R due to (1.1).Application of the Kristof-type theorem gives the solutionCG = Ω+12 AΦ32(Φ32 A′Ω+AΦ32)− 12,from which follows the solutionBG = Ω+AΦ32(Φ32 A′Ω+AΦ32)− 12Φ12 +(Ip −Ω+12Ω12)Q, Q arbitrary.The arbitrary component disappears in the predictor expression B′G(y − µy), because(Ip −Ω12Ω+12) (y − µy)= 0 with probability one (w. p. 1).Hence we get as predictorˆfG = Φ 12(Φ32 A′Ω+AΦ32)− 12Φ32 A′Ω+(y − µy).The reader can verify that A′Ω+A > 0.An alternative expression isCG = F2G′2,where we have used the singular-value decompositionΩ+12 AΦ32 = F2Γ122 G′2,with F′2F2 = G′2G2 = G2G′2 = Im. Use was made of the fact that Ω+12 A has full columnrank (m).Heinz Neudecker 31For nonsingular Ω the solution becomes that given by ten Berge et al. (1999) in theirpresentation, namely between (6) and (7).4 The McDonald procedureThis approach is based on the weighted-least-squares functiontrΨ+E(y − µy − A ˆf) (y − µy − A ˆf)′.ClearlyE(y − µy − A ˆf) (y − µy − A ˆf)′=(Ip − AB′)Ω(Ip − BA′)++A(a + B′µy) (a + B′µy)′A′.Again we find that a + B′µy = 0, now having to minimizetrΨ+E(y − µy − A ˆf) (y − µy − A ˆf)′.Notice that A′Ψ+A > 0.Imposition of the constraint E ˆf ˆf ′ = E f f ′ leads to the problem of minimizingtrΨ+(Ip − AB′)Ω(Ip − BA′)subject to B′ΩB = Φ.Using C′ = Φ− 12 B′Ω 12 we define the problem:maxCtrΦ12 A′Ψ+Ω12 C subject to C′C = Im.Application of the Kristof-type theorem yields the solutionCM = Ω12Ψ+AΦ12(Φ12 A′Ψ+ΩΨ+AΦ12)− 12,from which follows the solutionBM = Ω+12Ω12Ψ+AΦ12(Φ12 A′Ψ+ΩΨ+AΦ12)− 12Φ12 +(Ip −Ω+12Ω12)Q, Q arbitrary.Finally the predictor turns out to beˆfM = Φ 12(Φ12 A′Ψ+ΩΨ+AΦ12)− 12Φ12 A′Ψ+(y − µy).Again we used (Ip −Ω12Ω+12) (y − µy)= 0 w.p.1.The reader can verify that A′Ψ+ΩΨ+A > 0, usingA′Ψ+ΩΨ+A = A′Ψ+(AΦA′ + Ψ)Ψ+A = A′Ψ+AΦA′Ψ+A + A′Ψ+A.32 On best affine unbiased covariance-preserving prediction of factor scoresAn alternative expression isCM = F3G′3,whereΩ12Ψ+AΦ12 = F123 Γ123 G′3,with F′3F3 = G′3G3 = G3G′3 = Im.For nonsingular Ω the solution becomes that given by ten Berge et al. (1999) in theirpresentation, namely between (4) and (5).5 The Krijnen et al. procedureLike Green’s this approach uses the MSE matrix M of ˆf . Instead of tr (2Φ − B′AΦ−ΦA′B), Krijnen et al. use |2Φ − B′AΦ − ΦA′B| which has to be minimized. The firstthing to do is to prove that 2Φ − B′AΦ − ΦA′B > 0.We have2Φ − B′AΦ − ΦA′B = Φ 12(2Im − Φ−12 B′AΦ 12 − Φ 12 A′BΦ− 12)Φ= Φ12(2Im − Φ−12 B′Ω 12Ω+ 12 AΦ 12 − Φ 12 A′Ω+ 12Ω 12 BΦ− 12)Φ12= Φ12 (2Im −C′V − V ′C)Φ 12= Φ12[(C − V)′ (C − V) + (Im − V ′V)]Φ 12whereV = Ω+12 AΦ12 and hence V ′V = Φ 12 A′Ω+AΦ 12 .We shall show that all eigenvalues of V ′V are positive and less than unity. Pre-(post-)multiply the moment structure Ω = AΦA′ + Ψ by Φ 12 A′Ω+(Ω+AΦ 12). This leads toΦ12 A′Ω+AΦ 12 =(Φ12 A′Ω+AΦ 12)2+ Φ12 A′Ω+ΨΩ+AΦ 12 , henceΦ12 A′Ω+AΦ12 >(Φ12 A′Ω+AΦ12)2,as Φ12 A′Ω+ΨΩ+AΦ 12 > 0, and λi > λ2i where λi is any eigenvalue of Φ12 A′Ω+AΦ 12 . Thisproves the property. Hence Im − V ′V > 0. As (C − V)′(C − V) > 0 we have shown that2Φ − B′AΦ − ΦA′B > 0.Hence |2Φ − B′AΦ − ΦA′B| > 0. Consider then the positive definite matrix 2Im −C′V − V ′C. We use (7.18) in Zhang (1999) which yieldsC′V + V ′C 6 2U′(V ′CC′V) 12 Uwhere U is an orthogonal matrix.As C′C = Im we have CC′ 6 Ip. This in its turn leads to V ′CC′V 6 V ′V . The latterinequality gives (V ′CC′V) 12 6 (V ′V) 12 . See Theorem 2.5.5 in Wang & Chow (1994).Heinz Neudecker 33Finally, we haveC′V + V ′C 6 2U′(V ′V) 12 Uor equivalently2Im −C′V − V ′C > 2[Im − U′(V ′V) 12 U].From this we derive∣∣∣2Im −C′V − V ′C∣∣∣ >∣∣∣∣∣2[Im − U′(V ′V) 12 U]∣∣∣∣∣ =∣∣∣∣∣2[Im −(V ′V) 12]∣∣∣∣∣ .It is easy to see that CK = V(V ′V)− 12 leads to the equality∣∣∣2Im −C′KV − V ′CK∣∣∣ =∣∣∣∣∣2[Im −(V ′V) 12]∣∣∣∣∣ .Hence CK solves the problem. It is not clear whether the solution is unique.In fact, CK also solves the related problemmaxCtr V ′C subject to C′C = Im.The (unique) solution is CK by the Kristof-type theorem.Application of Zhang’s (7.18) yields2 tr V ′C = tr(C′V + V ′C)6 2 tr U′(V ′V) 12 U = 2 tr(V ′V) 12 ,which again has solution CK . We then get the solutionBK = Ω+AΦ12(Φ12 A′Ω+AΦ12)− 12Φ12 +(Ip −Ω+12Ω12)Q, Q arbitrary.From this follows the unique predictorˆfK = Φ 12(Φ12 A′Ω+AΦ12)− 12Φ12 A′Ω+(y − µy).For nonsingular Ω the solution CK coincides with that given by ten Berge et al. (1999),namely in (9).6 Equality of ˆfG and ˆfK when Φ and A′Ω+A commuteten Berge et al. (1999) showed that CG = CK under their assumptions when Φ andA′Ω−1A commute. We shall prove that ˆfG = ˆfK under our milder conditions.When Φ and A′Ω+A commute we have Φ = S MS ′ and A′Ω+A = S NS ′, where Mand N are positive definite diagonal matrices and S is orthogonal. HenceΦ32(Φ32 A′Ω+AΦ 32)− 12= S M 32 S ′(S M 32 S ′S NS ′S M 32 S ′)− 12= S M 32 S ′(S M3NS ′)− 12= S M 32 S ′S(M3N)− 12 S ′= S N− 12 S ′ = (A′Ω+A)− 12 .34 On best affine unbiased covariance-preserving prediction of factor scoresFurther Φ 12(Φ12 A′Ω+AΦ 12)− 12= (A′Ω+A)− 12 .This yields ˆfG = ˆfK = Φ 12 (A′Ω+A)− 12 A′Ω+(y − µy).7 Equality of ˆfM and ˆfK when Ψ and AΦA′ commuteten Berge et al. (1999) showed that CM = CK under their assumptions when Ψ isnonsingular. Essential is the expressionΩ−1= Ψ−1 − Ψ−1AΦ12(Im + Φ12 A′Ψ−1AΦ12)−1Φ12 A′Ψ−1.Under our assumptions Im + Φ12 A′Ψ+AΦ 12 is nonsingular because A′Ψ+A > 0 whichfollows from T ′ΨT > 0 and (1.1). When we additionally assume that Ψ and AΦA′commute we can establish the equalityΩ+= Ψ+ − Ψ+AΦ12(Im + Φ12 A′Ψ+AΦ12)−1Φ12 A′Ψ+.Proof. When Ψ and AΦA′ commute we have Ψ = S MS ′ and AΦA′ = S NS ′ whereM and N are positive definite diagonal matrices and S ′S = Im. Further AΦ12 = S N 12 T ′,with orthogonal T , a singular-value decomposition. HenceΨ+ − Ψ+AΦ 12(Im + Φ12 A′Ψ+AΦ 12)−1Φ12 A′Ψ+= S M−1S ′ − S M−1S ′S N 12 T ′(Im + T N12 S ′S M−1S ′S N 12 T ′)−1T N 12 S ′S M−1S ′= S M−1S ′ − S M−1N 12 T ′(Im + T M−1NT ′)−1T M−1N 12 S ′= S M−1S ′ − S M−1N 12 T ′T(Im + M−1N)−1T ′T M−1N 12 S ′= S M−1S ′ − S M−1N 12(Im + M−1N)−1M−1N 12 S ′.Further Ω = AΦA′ + Ψ = S (M + N)S ′, and Ω+ = S (M + N)−1S ′. It is easy to see that(M + N)−1 = M−1 − M−1N 12(Im + M−1N)−1M−1N12 .This yields the result. ¤Recall thatˆfM = Φ 12(Φ12 A′Ψ+ΩΨ+AΦ12)− 12Φ12 A′Ψ+(y − µy)andˆfK = Φ 12(Φ12 A′Ω+AΦ12)− 12Φ12 A′Ω+(y − µy).Heinz Neudecker 35ConsiderΦ12 A′Ω+AΦ 12 = Φ 12 A′Ψ+AΦ 12 − Φ 12 A′Ψ+AΦ 12(Im + Φ12 A′Ψ+AΦ 12)−1× Φ12 A′Ψ+AΦ 12=(Im + Φ12 A′Ψ+AΦ 12)−1Φ12 A′Ψ+AΦ 12= (Im + E)−1 E,Φ12 A′Ψ+ΩΨ+AΦ 12 = Φ 12 A′Ψ+AΦA′Ψ+AΦ 12 + Φ 12 A′Ψ+AΦ 12= Φ12 A′Ψ+AΦ 12 +(Φ12 A′Ψ+AΦ 12)2= E + E2,Φ12 A′Ω+ = Φ 12 A′Ψ+ − Φ 12 A′Ψ+AΦ 12(Im + Φ12 A′Ψ+AΦ 12)−1Φ12 A′Ψ+=(Im + Φ12 A′Ψ+AΦ 12)−1Φ12 A′Ψ+= (Im + E)−1Φ 12 A′Ψ+,ˆfK = Φ 12[(Im + E)−1 E]− 12 (Im + E)−1Φ 12 A′Ψ+(y − µy),ˆfM = Φ 12(E + E2)− 12Φ12 A′Ψ+(y − µy).Clearly [(Im + E)−1 E]− 12 (Im + E)−1 =(E + E2)− 12as E > 0.This establishes the equality of ˆfK and ˆfM.8 Comments1. ten Berge et al. (1999) claim that the McDonald method is undefined when Ψ issingular. This is unjustified. What matters is the nonsingularity of T ′ΨT . We makethat assumption. It implies that A′Ψ+A > 0 which we use several times.2. Application of Zhang’s result shows immediately that CG and CM yield themaximum. The Kristof-type theorem shows the unicity of the solutions.AcknowledgementThe author is grateful to Go¨tz Trenkler for drawing his attention to Zhang’s result (7.18)which was fruitfully applied in Section 5. The reasoning why B′µy + a should be equal36 On best affine unbiased covariance-preserving prediction of factor scoresto zero in all three procedures is due to Albert Satorra. This yields ˆf = B′(y − µy), anunbiased predictor.9 ReferencesAnderson, T. W. & Rubin, H. (1956). Statistical inference in factor analysis. Proceedings of the ThirdBerkeley Symposium on Mathematical Statistics and Probability. Vol. V, 111-50. University ofCalifornia Press, Berkeley and Los Angeles.ten Berge, J. M. F. (1993). Least Squares Optimization in Multivariate Analysis. DSWO Press, Leiden,Netherlands.ten Berge, J. M. F., Krijnen, W. P., Wansbeek, T. & Shapiro, A. (1999). Some new results on correlation-preserving factor scores prediction methods. Linear Algebra Appl., 289, 311-8.Green, B. F. (1969). Best linear composites with a specified structure. Psychometrika, 34, 301-18.Krijnen, W. P., Wansbeek, T. J. & ten Berge, J. M. F. (1996). Best linear predictors for factor scores.Communications in Statistics: Theory and Methods, 25, 3013-25.Kristof, W. (1970). A theorem on the trace of certain matrix products and some applications. Journal ofMathematical Psychology, 7, 515-30.Lo¨wner, K. (1934). ¨Uber monotone Matrixfunktionen. Mathematische Zeitschrift, 38, 177-216.Magnus, J. R. & Neudecker, H. (1999). Matrix Differential Calculus with Applications in Statistics andEconometrics. Revised Edition. Wiley, Chichester, England.McDonald, R. P. (1981). Constrained least squares estimators of oblique common factors. Psychometrika,46, 337-41.Wang, S. G. & Chow, S. C. (1994). Advanced Linear Models: Theory and Applications. Marcel Dekker,New York.Zhang, F. (1999). Matrix Theory. Basic Results and Techniques. Springer, New York.ResumEs do´na una generalitzacio´ dels resultats presentats per ten Berge, Krijnen, Wansbeek and Shapiro .Aquests autors examinen me`todes i resultats basats en Anderson i Rubin. Mc Donald, Green i Krijnen,Wansbeek i ten Berge. Considerarem el mateix plantejament pero` sota condicions de rang me´s de`bils.Aixı´ suposarem que alguns moments, com les matrius de covaria`ncies Ω del vector de mesuresobservades dels factors comuns i ψ dels factors u´nics, siguin singulars. Imposem la condicio´ T′ψT > 0,essent TΛT′ la descomposicio´ de Schur de Ω. Com e´s usual, suposem que tenen rang ma`xim percolumnes les matrius de covaria`ncies Φ dels factors comuns i la matriu A del model factorial.MSC: 62H25, 15A24Paraules clau: ana`lisi factorial, mesures de factors, preservacio´ de la covaria`ncia, teorema tipus Kristof

This paper gives a generalization of results presented by ten Berge, Krijnen, Wansbeek & Shapiro. They examined procedures and results as proposed by Anderson & Rubin, McDonald, Green and Krijnen, Wansbeek & ten Berge. We shall consider the same matter, under weaker rank assumptions. We allow some moments, namely the variance of the observable scores vector and that of the

unique factors,to be singular. We require T′ T > 0, where T T′ is a Schur decomposition of. As usual the variance of the common factors, , and the loadings matrix Awill have full column rank.Peer Reviewe

UPCommons

This paper gives a generalization of results presented by ten Berge, Krijnen, Wansbeek & Shapiro. They examined procedures and results as proposed by Anderson & Rubin, McDonald, Green and Krijnen, Wansbeek & ten Berge. We shall consider the same matter, under weaker rank assumptions. We allow some moments, namely the variance of the observable scores vector and that of the
unique factors,to be singular. We require T′ T > 0, where T T′ is a Schur decomposition of. As usual the variance of the common factors, , and the loadings matrix Awill have full column rank.Peer Reviewe

On best affine unbiased covariance-preserving prediction of factor scores

Abstract

Similar works

Full text

Available Versions

Revistes Catalanes amb Accés Obert

Diposit Digital de Documents de la UAB

UPCommons

UPCommons. Portal del coneixement obert de la UPC