Groß, Jürgen

Trenkler, Götz

Eldorado - Repository of the TU Dortmund

The Equality Between Linear Transforms of Ordinary LeastSquares and Best Linear Unbiased EstimatorJurgen Gro & Gotz TrenklerDepartment of StatisticsUniversity of DortmundVogelpothsweg 87D-44221 Dortmund, GermanyAbstract. The best linear unbiased estimator BLUE(CX) of a linear transform CX inthe general Gauss-Markov model fy;E(y) = X;Cov(y) = 2Vg is the linear transformCBLUE(X) of the best linear unbiased estimator BLUE(X) of X. Similarly, for theordinary least squares estimator, OLSE(CX) = COLSE(X). The problem of equalityof OLSE(X) and BLUE(X) has been widely discussed in the literature. In this note,characterizations of the equality COLSE(X) = CBLUE(X) are given in terms ofprojectors and subspaces.Keywords: Ordinary least squares estimator, best linear unbiased estimator, prediction,linear transform, orthogonal projectorAMS 1991 Mathematics Subject Classication: 62J051. Introduction. Consider the general Gauss-Markov model denoted byM = fy;X; 2Vg; (1.1)where y is an observable n 1 random vector with E(y) = X and Cov(y) = 2V,X is a known n  p matrix of rank r, 0 < r < n ,V is a known n  n symmetricnonnegative denite matrix (possibly singular),  is a p  1 vector of unknownparameters, and 2> 0 is either known or unknown.Our interest focuses on estimation of a linear transform CX of X, where Cis a given k  n matrix. Recall that estimation of X0 with known X0= CX canalso be seen as (classical) prediction of an unobservable random vector y0satisfyingE(y0) = X0.e-mail: gross@amadeus.statistik.uni-dortmund.de, trenkler@amadeus.statistik.uni-dortmund.de1It is well known [10, 11] that a representation of the best linear unbiased estimator(BLUE) of CX is given byBLUE(CX) = CX(X0T+X)+X0T+y; (1.2)where T = V+XX0. Here, A0and A+denote the transpose and the Moore-Penroseinverse of an arbitrary matrix A, respectively. Since we haveBLUE(X) = X(X0T+X)+X0T+y; (1.3)it is clear that BLUE(CX) = CBLUE(X). Although there exist further repre-sentations of BLUE(X), all of them coincide almost surely, so that without loss ofgenerality we may conne ourselves to (1.3). Consider now the competing estimatorCOLSE(X), whereOLSE(X) = XX+y (1.4)is known as the ordinary least squares estimator (OLSE) of X. The problem ofequality of BLUE(X) and OLSE(X) has been widely discussed in the literature.See e.g. [2] where two dierent versions of this problem are investigated, and [9] foran excellent overview. As one (among many other) necessary and sucient conditionfor BLUE(X) = OLSE(X), Puntanen and Styan [9, cond. AS2] statePXV = PXVPX; (1.5)where the symbol PA= AA+is used to represent the orthogonal projector ontothe range (column space) R(A) of an arbitrary n p matrix A. The symbol QA=In  PAwill denote the orthogonal projector onto the orthogonal complement ofR(A), compare also [6, Chap. 12].In the following we derive a condition similar to (1.5) for the less restrictiveequality CBLUE(X) = COLSE(X).2. Equality of estimators. By conning ourselves to the representation (1.3)of BLUE(X) we observe that CBLUE(X) = COLSE(X) almost surely if andonly ifCXX+y = CX(X0T+X)+X0T+y for all y 2 R (X) +R(V), (2.1)2the latter being equivalent to the identitiesCXX+X = CX(X0T+X)+X0T+X (2.2)andCXX+V = CX(X0T+X)+X0T+V: (2.3)But since we have X = XX+X andX = X(X0T+X)+X0T+X; (2.4)the latter being true in view of R(X0) = R(X0T+X), see e.g. [3, Theorem 2],condition (2.2) is always met. Thus, the equality under study holds if and only if(2.3) is satised.Proposition 1. Under model M = fy;X; 2Vg, equality of CBLUE(X) andCOLSE(X) holds almost surely if and only ifCPXV = CPXVPX: (2.5)Proof. It remains to show that (2.5) is equivalent to (2.3). Since by (2.4) andX0T+T = X0we haveX(X0T+X)+X0T+V = X(X0T+X)+X0T+(T XX0)= X(X0T+X)+X0T+T XX0= X(X0T+X)+X0 XX0; (2.6)it is clear that (2.5) follows from (2.3) by right-multiplication of (2.3) with XX+.Conversely assume that (2.5) is satised. Then,CPXV = CPXV(X+)0X0= CPXV(X+)0X0T+X(X0T+X)+X0= CPXVPXT+X(X0T+X)+X0= CPXVT+X(X0T+X)+X0:But since in view of (2.6) we haveVT+X(X0T+X)+X0= X(X0T+X)+X0T+V;3it follows thatCPXV = CPXX(X0T+X)+X0T+V = CX(X0T+X)+X0T+V;showing (2.3).As an immediate corollary we obtain the following.Corollary. Under model M = fy;X; 2Vg, the following statements are equiv-alent :(i) BLUE(X) = OLSE(X),(ii) X0BLUE(X) = X0OLSE(X),(iii) X+BLUE(X) = X+OLSE(X),Proof. Observe that for C = X0and C = X+, equations (2.5) and (1.5) areequivalent.Obviously each of the numerous equivalent conditions for BLUE(X) =OLSE(X) is sucient for CBLUE(X) = COLSE(X), includingPXV = VPX; (2.7)which is called Zyskind's condition in [9].It is clear that (2.5) may be reformulated asCPXVQX= 0; (2.8)where QX= In PX. If we are interested in characterizing all matrices C satisfyingCBLUE(X) = COLSE(X) almost surely, then we simply have to inspect thegeneral solution to (2.8) with respect to C, beingC = Z(In PXVQX(PXVQX)+); (2.9)where Z is an arbitrary k  n matrix. In view ofrank(PXVQX)  rank(QX) = n  rank(X) < nit is obvious that there always exists more than one (trivial) solution to (2.8).4If we are interested in characterizing all nonnegative denite matrices V sat-isfying CBLUE(X) = COLSE(X) almost surely, then we have to inspect thegeneral nonnegative denite solution to (2.8) with respect to V. It is clear that (2.8)is equivalent to MVQX= 0, where M = PXC0CPX, see [6, Lemma 11.6.2], whichin turn is equivalent toPMVQX= 0: (2.10)The general nonnegative denite solution to (2.10) can be derived from Theorem2.5 in [7]. By letting Rmndenote the set of m n real matrices and Rnndenotethe set of n  n real (symmetric) nonnegative denite matrices, we may state thefollowing.Proposition 2. Under model M = fy;X; 2Vg, the following two statementshold:(i) For given X 2 Rnpand V 2 Rnnthe set of all matrices C 2 RknsatisfyingCBLUE(X) = COLSE(X) almost surely is given byfC = ZQLjZ 2 Rkng ;where L = PXVQX.(ii) For given X 2 Rnpand C 2 Rknthe set of all matrices V 2 RnnsatisfyingCBLUE(X) = COLSE(X) almost surely is given byfV=PMZ1PM+QXZ2QX+(PX PM)Z3(PX PM)jZ1;Z2;Z32 Rnng;where M = PXC0CPX.Proof. The proof of (i) is clear from the above considerations. For the proof of(ii) observe beforehand thatPXPM= PM= PMPX; (2.11)(PM+QX)+= PM+QX; (2.12)see also [4, Theorem 3.1.1] for (2.12). From Theorem 2.5 in [7], the general nonneg-ative denite solution to (2.10) with respect to V is given byV = ( PM+QX)+(A+B)(PM+QX)++[In  (PM+QX)+(PM+QX)]Z3[In  (PM+QX)+(PM+QX)];5where Z32 Rnnis arbitrary, and A and B are arbitrary nonnegative denitesolutions ofA(PM+QX)+QX= 0; (2.13)PM(PM+QX)+B = 0; (2.14)such that A+B is nonnegative denite. By using (2.11) and (2.12) it follows that(2.13) is equivalent to AQX= 0 with general nonnegative denite solution, see [7,Theorem 2.2],A = ( In QX)Z1(In QX) = PXZ1PX; (2.15)where Z12 Rnnis arbitrary. Moreover, it is seen that (2.14) is equivalent toPMB = 0 with general nonnegative denite solution, see [7, Theorem 2.2],B = ( In PM)Z2(In PM) = QMZ2QM; (2.16)where Z22 Rnnis arbitrary. Clearly A+B is nonnegative denite for all choicesof Z1;Z22 Rnn. By using again (2.11) and (2.12), we observe thatIn  (PM+QX)+(PM+QX) = PX PM:Therefore, the general nonnegative denite solution to (2.10) is given byV = ( PM+QX)(A+B)(PM+QX) + (PX PM)Z3(PX PM);where A and B are as in (2.15) and (2.16), respectively. By writing(PM+QX)(A+B)(PM+QX)= ( PM+QX)(PXZ1PX+QMZ2QM)(PM+QX)= PMPXZ1PXPM+QXQMZ2QMQX= PMZ1PM+QXZ2QX;we arrive atV = PMZ1PM+QXZ2QX+ (PX PM)Z3(PX PM);where Z1;Z2;Z32 Rnnare arbitrary.6A related but somewhat dierent problem is to determine the subspace ofpossible observation vectors y for xed C, X and V satisfying CBLUE(X) =COLSE(X). Under the assumptions r = rank(X) = p, rank(V) = n andC = BX+, where B is an arbitrary (but xed) k  p matrix, this subspace hasbeen identied in [5] to beE = R(X)R(X?) \R(VX?) [R(X) \ N (C)]; (2.17)where X?denotes any matrix of maximal rank such that X0X?= 0, and N (C)denotes the null space of C. In case B = Ip, k = p, the subspace E reduces toE = R(X)R(X?) \ R (VX?); (2.18)which has been observed earlier in [8].We will now demonstrate that the subspace (2.17) remains the appropriate choiceunder the more general assumptions of model (1.1), when in addition y is restrictedto be inR(X)+R(V). The latter guarantees that dierent choices of representationsof BLUE(X) cannot lead to dierent estimates of X.Proposition 3. Under model M = fy;X; 2Vg, for given X 2 Rnp, V 2 Rnnand C 2 Rkn, the set E of all vectors y 2 R(X)+R(V) satisfying CBLUE(X) =COLSE(X) is given byE = R(X) [R(X?) \ F];where F = R(VX?) [R(X) \ N (C)].Proof. The set of all vectors y 2 E isE = N [C(PX R)] \ [R(X) +R(V)]; (2.19)where R = X(X0T+X)+X0T+. Let F = [ R(X) +R(V)] \N (CR). ThenF = R(VX?) [R(X) \N (C)] (2.20)follows similarly as in the proof of Lemma 1 in [5], and it remains to showN [C(PX R)] \ [R(X) +R(V)] = R(X)R(X?) \ F: (2.21)Let y be a vector belonging to the left-hand subspace of (2.21). Such a vector y canbe written as y = a + b for some a 2 R(X) and some b 2 R(X?). Obviously, b =7y a 2 R (X)+R(V). Moreover we have CPXy = Ca and CRy = CRa+CRb =Ca + CRb, yielding 0 = C(PX  R)y =  CRb. Therefore, b 2 F = N (CR) \[R(X) + R(V)], showing that the left-hand subspace of (2.21) is contained in theright-hand subspace of (2.21). To demonstrate the reverse inclusion let y = Xa+b,where a 2 Rpand b 2 R(X?) \ F, where clearly R(X?) \ F  R (X) +R(V),and therefore y 2 R (X) +R(V). Moreover, X0b = 0 and CRb = 0, and in viewof (PX  R)X = 0 we obtain C(PX  R)y = C(PX  R)Xa + C(PX  R)b =C(PX  R)b = C(X+)0X0b   CRb = 0   0 = 0, showing y 2 N [C(PX  R)].Hence, the right-hand subspace of (2.21) is contained in the left-hand subspace of(2.21).Note that for the special choice C = In, k = n, we have F = R(VX?), showingthat the set E of all vectors y 2 R (X)+R(V) satisfying BLUE(X) = OLSE(X)is given by (2.18).3. Example. Consider the one-way classication modelyij= + i+ eij; i = 1 ; : : : ; a; j= 1 ; : : : ni;where the eij's are uncorrelated random variables with means 0 and variances dij2.Assume for a numerical example a = 3, n1= 3, n2= 2, n1= 1, and dij= 1if (i; j) 6= (1 ;3). Assume in addition d136= 1 but otherwise unknown. Then theerror variances are not homogenous within groups, and from Corollary 4 in [1] itfollows that we do not have equality of OLSE and BLUE of any parametric function.However, if we consider the contrast 2 3= c0X, where c0= (0 ;0; 0; 12; 12; 1),it follows easily from our Proposition that OLSE(2  3) = BLUE(2  3) =y31 12y21 12y22.Acknowledgements. Support by Deutsche Forschungsgemeinschaft undergrants Tr 253/2-3 and SFB 475 is gratefully acknowledged.REFERENCES1. J.K. Baksalary, Criteria for the equality between ordinary least squares and best linear un-biased estimators under certain linear models, The Canadian Journal of Statistics, 16, pp.97{102 (1988).82. J.K. Baksalary and A.C. van Eijnsbergen, A comparison of two criteria for ordinary-least-squares estimators to be best linear unbiased estimators, The American Statistician, 42, pp.205{208 (1988).3. J.K. Baksalary, S. Puntanen and G.P.H. Styan, A property of the dispersion matrix of thebest linear unbiased estimator in the general Gauss-Markov model, Sankhya, Ser. A, 52, pp.279{296 (1990).4. S.L. Campbell and C.D. Meyer, Jr., Generalized Inverses of Linear Transformations, Pitman,London (1979).5. J. Gro and G. Trenkler, When do linear transforms of ordinary least squares and Gauss-Markov estimator coincide?, Sankhya, Ser. A, 59, pp. 175{178 (1997).6. D.A. Harville, Matrix Algebra From a Statistician's Perspective, Springer, New York (1997).7. C.G. Khatri and S.K. Mitra, Hermitian and nonnegative denite solutions of linear matrixequations, SIAM Journal of Applied Mathematics, 31, pp. 579{585.8. W. Kramer, A note on the equality of ordinary least squares and Gauss-Markov estimates inthe general linear model, Sankhya, Ser. A, 42, pp. 130{131 (1971).9. S. Puntanen and G.P.H. Styan, The equality of the ordinary least squares estimator and thebest linear unbiased estimator (with discussion), The American Statistician, 43, pp. 153{164(1989).10. C.R. Rao, Unied theory of linear estimation, Sankhya, Ser. A, 33, pp. 370{396 (1971).11. C.R. Rao, Representations of best linear unbiased estimators in the Gauss{Marko modelwith a singular dispersion matrix, Journal of Multivariate Analysis, 3, pp. 276{292 (1973).9

The Equality Between Linear Transforms of Ordinary Least Squares and Best Linear Unbiased Estimator

https://eldorado.tu-dortmund.de/bitstream/2003/4856/1/98_14.pdf

The Equality Between Linear Transforms of Ordinary Least Squares and Best Linear Unbiased Estimator

Abstract

Similar works

Full text

Available Versions

Eldorado - Repository of the TU Dortmund