Spjøtvoll, Emil

NORA - Norwegian Open Research Archives

Statistical Research Report Institute of Mathematics University of Oslo A NOTE ON ROBUST ESTIMATION IN ANALYSIS OF VARIANCE By Emil Spj¢tvoll No. 6 September 1967 - 1 -1. I N T R 0 D U C T I 0_1!. Consider the e~sample model, in which the observations are {1.1) = 1,2, ••• ,'}-t.• 1 i = 1,2, ••• ,c, where the variables U. are independently distributed with cumulative 1C¥:.-distribution function F. Let (1.2) Y •• = med (X. - X. A) 1J 1 Qf_ • J r be the median of the ni nj differences Xi~- Xj (3 (0( =1,2, ••• ,ni, ~= 1,2, ••• 'lJ ) • It has been shown by the Hodges and Lehmann [2 J that the estimate Yij of S i - S j has more robust efficiency than the standard estimate T. . = X. - X. where X. = L X. ~ / n .• 1J 1. J• 1· l, 1 The estimates Y .. do not satisfY the linear relations satisfied lJ by the differences they estimate. To remedy this, the~ estimates ~ by Lehmann [3J replaced by adjusted estimates Zij of the form ~i This was done by minimizing the sum of squares (1.3) r:: ( y.. - { c 1. - ¢J. ))2 1TJ 1J S !:J. giving (see [ 2 J) (1.4) z. . = y. - y. 1J 1. J. Y .. were 1J 6 - 9j . where Y. = (1/c) S Y . . and where Y .. is defined to be zero for all i. 1. 1J 11 The purpose of this note is to argue that in the sum of squares (1.3) there should be used weights according to the number of observations on which the different Y. . are based. 1J - 2 -For purpose of reference we state a theorem of Lehmann. Let the sample sizes n. tend to infinity in such a way that n. = q. N(N =~ n.). ~ ~ ~ t--~ Then we have the following theorem (Theorem 2 of [3] ) . T H E 0 R E M 1. (i) The joint distribution of (V1 ~ Vz, ••• ,Vc-l) where 1 Vi = N2 (Yic- ( § i -5o ) is asymptotically normal with zero mean and covariance matrix Var (Vi)= (1/12)(1/5 i + 1/~c) I <jt2{x)dx)2 Cov(Vi ,Vj) = (l/12~c)/({f2 (x)dx) 2 • Here the density f of F ~s assumed to satisfy the regularity conditions of Lemma 3~a) of [ 1] . (ii) Fer a.ny i and j , , )1!2 y. . "'-.) ]IJ2 ( y. - y. ) ~J ~c JC where rv indicates that the difference of the two sides tends to zero in probability. 2. WE I G. TED ESTIHATES. Define the ~c (c-1)- component vector yc-l,c] • Denote the covariance matrix of Y by A. Y"' = ~12 'yl3, ••• ' Suppose that E Y .. ~J =~.-~. 5~ .?J (conditions under which this holds or approximately holds are given in [2 J·) for all J. and J • To estimate the difference.a g i - ,$j can then be treated as an ordinary regression proble..'ll. The minimum variance unbiased .linear estimates of the §i -j'j are obtained by minimizing - 3 -i" kl -1 where a. J' denote the elements of A • Since from Theorem 1 asymptotically E Yij = § i - ~ j it seems reasonable to minimize (2.1 } even i:f the Y.. are not exactly l.J unbiased estimates o:f ~ i - ~ j :for :finite N. Unfortunately the elements of A are unknown. But suppose we use an arbitrary :matrix ~..r with elements w .. kl such that we shall l.J' minimize (2.2) ~w •• kl(Y .. - (C. -~.))(Y - (' -~ )). l.J ' l.J , 1 .7J kl k 1 Let t'. . denote the minimizing l.J value of <j. - ~. in (2.2). vie shall study the asymptotic distribution , ]. ./ J of the t' ... We shall allow the matrix W to vary with the number o:f l.J observations, and use the notation W(n1 , n2 , ••• , nc) = WN. tend to infin~ty as in Theorem 1. THE 0 R EM: 2. Let the n. l For any sequence luN J of matrices of rank ~ c-1 converging to a matrix U o:f rank ~ c-1 , asymptotically for any i and j 0 'fNl (Y.~- Y .• ) --JO lJ lJ P R 0 0 F. To get a :full rank regression problem we introduce the parameters Then (2.4) 5 i - ~. J i = 1,2, ••• ,c-l. =e. _e.;. l j Let B den( te the design matrix sw.h that ( 2. 2} can be 'Wl'i tten - 4 -The value ot Q minimizing (2.5) is ~N = (B' UN B)-l B"' w1~ Y. Define Y1 by Y' = (Y1 Y 1 ). 1 c, ••• , c- ,c B,y Theorem l(i) , (2.6) N2 (Y- B Y1 )~0. We have ~ ""' (2.7) N (QN- Y1 ) B'W0 N~ (Y - BY1 ). B,y (2.6)and the continuity of the second function &nd the uniform convergence in any closed interval of the first fUnction on the right hand side of (2.7) it follows that N~ ('CiN - BY1 ) '""' 0. Hence N~(~ - Y. ) ,..., 0 for any i • By Theorem l(ii) and the fact that 1C 1C 1 t!. = t:' - ~ it follows that N2 (~·. - Y .. )""'0 for any i and j. The 1J 1C JC 1J 1J theorem is proved. It is seen from the above theorem that the asymptotic distribution of the estimates does not depend on the matrice~ WN. Hence the o.sympto~ic distribution will be the same as for the best linear estimates. {The solution of (2.1)). given by Lehmann. In particular this is true for the estimates z .. 1J But , of course, the best unbiased linear estimates will give better estiinates than the Z .. for finite N. Since A is unknown we cannot find the 1J - 5 -former. By Theorem l(i) the asymptotic vo.lue of A is kno'Wil, but it is singular and cannot be used in (2.1). We now propose to use the asymptotic variances of theY .. ~J as weights i.e. we want to minimize (2.8) Q ~ (1/n. + 1/n. )-1 (Y .. - { S: • = L-- ~ j ~J ~~ - ?- • ))2 with respect to 5- • - S:.. • 5 J 5 ~ SJ vle introduce ( 2. 3) and ( 2. 4) in ( 2. 8) • .();f'b~:u derivation of ( 2. 8) with respect to the Q. it is found that the minimizing values are given by the ~ solutions of the equations A J;l • ... -- ~ go\.<+-- n. + n "1 L :F<:l... ~ -.., (2.9) ) n. = :L ___,;;;;~-n. + n. ~ J. y .. (;/..~ Ol = 1,2, ••• ,c-l. It does not seem easy to find an explicit algebraic solution of (2.9), though for each specific set of the n. we can solve {2.9), if ~ necessary with the aid of an electronic computer. i'\ the Q. l It follows from Theorem 2 that the asymptotic distribution of A g. ~s equal to the asymptotic Cl.istribution of the estimates J Z.. and hence the same is true regarding asymptotic efficiencies. l.J We now proceed to prove that in some respects the estimates /' /\. Q. - g is better than the Z ..• Let D be a subset of the integers ~ j ~J 1,2, ••• ,c. Suppose that n. -) 0. N as ~ _)~ N .....,. 00 when if D while n./N ~0 ]. when i ' D. We shall study the asymptotic distribution of the estimates in this case. Hi thout loss of generality we may assume D = { 1,2, ••• , b} for some b <..c. THE 0 R EM 3. Suppose that N -:.oo such that n. ~g. N when i=l,2, ••• ,b b l ]. A """' <Z~· = 1). Then the asymptotic distribution ofthe Q. - Q. of (2.9) for 1 ]. ~ J i,j f: b ~s equal to the asymptotic distribution of the y .. in Theorem 1 ~J when c is replaced by b. - 6 -P R 0 0 F. Q can be written Q = ~ (1/n. - ~ max(i,j)~b (2.lo) + L (n~/N)(n./N)(R1/N +n./N}-1 (Y .. -(;..: - ~J. })2. J ' J ~J ~ ... ~ max(i,j))b By assumption the last expression on the right hand side of (2.1o) tends to zero when N 7 oo. Hence Q rv) maxTr,j )'b (lln.+l/n. }-1 (Y •• -(f.-f.) )2 ~ J ~J l. J which is of the form (2.8} with c replaced by b. The theorem now easily follows since the same results holds for the Y .. with i,j 'b as ror the l.J Y •• with i,j ~c. ~J In [3] is given an example which shows that the estimate z12 of f 1 - f 2 is not consistent when unless also n3 tends to infinity (c=3). ... nl and n2 tends to infinity Theorem 3 proves that the new A A estimates Q. - Q. ~ J do not have this definciency. If for the same i -"' /'. tends to infinity then Q.- Q. is a consistent estimate and J n. ~ and n. J ~ J of ~i -ij. 3. A N A L T E R N A T I V E E S T I M A T E • Since the estimates Q.- Q. ]. J of '(2.9} is not easily computed unless one haveaecess to an electronic computer, we shall give alternative simpler estimates which also are weighted estimates. We shall minimize (3.1) n. n. (Y .. -(~.- ~.))2 • l. J ~J l. s J - 7 -By differentiation it is easily found that the values of .;. - ~ . ~ J minimizing (3.1) is (3.2) W .. = ~ n (Y. - Y.,..,.) = Y: - Y. ~J ...___ Q( ~':X. J ""' ~ J where we have introdused the weighted differences Yi = () no( )-l L no< Yio(" Compare (1.4). It follows from Theorem 2 that the estimates W.. have the .....-:::-... ~J . • same asymptotic properties as the Z.. and 9. -1t.. Furthermore ~t 1.s ~J ~ J easily seen that Theorem 3 holds for theW ..• ~J 4. THE C A S E When n1 = n2 Z. . • Further we have ~J -. • • = n = n . c. - ....--.. both 9. - 9. c ~ J If n = n = ••• = n = n 1 2 c and W . . reduce to ~J T H E 0 R E M 4. then the estimates Z.. are the minimum variance unbiased linear estimates. ~J P R 0 0 F Define " 2 and a by (4.1) Note that both b2 and a depend upon n and F. By symmetiY we have Var Y .. = 62 i=t= j. l.J (4.2) Cov (Y .. 'ykl) = 0 i =i= k, l. + 1, j:f= k, j =1-=1. ~J a6 2 Cov (Y .. ,Y. 1 ) = j =t= 1, l. =+= j' i=f= 1. ~J ~ a6 2 Cov (Y .. 'yk.) = i 'f= k, i + j' k+ J• ~J J 2 Cov (Y .. ,Y.1 ) = -ab i 1= 1, i t J' j + 1, l.J J 2 Cov (Y .. 'yk. ) = -ao k=i= j' i ~= j' k =I= i l.J .~ - 8 -Let the covariance matrix of Y given by (4.2) be G(a)62• It can be verified that the inverse of G(a) is (1+2{c-2)da)-l G(d) where d= - a(l+(c-4)a)-l • Hence (2.1) is proportional to j-1 Ql = r: L (Y .. - ( ~. - t.) )2 . . 1 ~J 7 ~ :.:iJ J ~= + 2 d r. J :!:: ~ (Y .. - (~. -~.))( Yh.- (c.\h- c\J.)) i=l h=i+l ~J ~ /J J 7 5 j-1 j-1 + 2 d :r ,L !:::: J i=l h=l The part of Q1 involving ~o{ can be written ) (Y. -( ~. _2 ) )2 ~ .~ ~ ;;"( + 2 <2' j L (Y. - <5. oo:S ))( Y •• - <dJ. -E; )) i=f ~ JO( . J. ~ J~ ../ .;;; ... + 2 d~ ~J j-1 A \ L (Y. - (7~ . -t. ) )( y. - <$ l.. - £..,)) • i = 1 J IX, J .,Ale ~ t:J( :;;; .... We find ..... o'Q. ·~ / = 2 {l+d(c-1))(~ -.L3i · :.?~ ~~i). """" Hence the minimizing values Ji satisfy /\ /' §01\ = C· -12: 5i + c-1 ~ Y«: J. l. and hence by (1.4) 1'. - ~ . - c -1 ) y . ..) l. .;:; J -r- 0( l. - c -1 -::::;:--£.....-YO(. = J J z .. ~J - 9 -5. An example. In this section the estimates are compared on an example taken from Scheffe:" The Analysis of Variance". p. 140. It is a two-way l~out with factors genotype of the foster mother and that of the litter. The obser~ations are weights(average) of the litter. Let A1 , A2, ~' A4 and B1 , B2 , B3, B4 denote the different genotypes of the foster mother and the litter respectively. The observatoons are: 61.5 68.2 64.0 65.0 59.7 r;: 1 37.0 36.3 68.0 55.0 52.5 42.0 6o.3 50.8 56.5 51.3 42.0 61.8 54.0 51.7 64.7 59.0 40.5 60.2 49.5 61.0 49.3 61.7 47.2 52.7 48.2 48.0 64.0 53 0 39.6 62.0 B3 ./ A2 A3 A4 Al A2 A3 A4 56.3 39.7 50.0 59.0 59.5 45.2 44.8 69.8 46.0 43.8 57.4 52.8 57.0 51.5 67.0 61.3 54.5 54.0 56.0 61.4 53.0 55.3 47.0 42.0 55.7 54.0 Let f .. denote the expectation of the variables from l.J (Ai' Bj). In Table 2 are given the estimates of the differences ~ij-~44 obtained by the different methods. Cla.3sical Classical - lo -Table 2 • ~ll-~44~~1:544~31-~44 541-~44 Jl2:_$44 322:§.lfif~32-.§ 44 ~42-~44 y 14.2 3.5 6.1 o.o 3.9 lo.7 5.0 - 2.2 z 14.35 3.86 4.65 0.14 3.14 12.0 4.82 - 2.87 w 14.29 3.88 4.57 0.06 3.ol 11.95 4.83 - 2.8a Q 14.31 3.87 4.60 0.09 3.07 11.98 4.82 - 2.89 ' 14.62 3.34 5.07 - o.lo 3.27 11.58 4.87 - 3.16 513-J44J23- ~44 _$33- s44 ~43- s44 ~14-j44 s24:§\4 _$34-§44 y - 7.8 15.5 2.7 0.5 5.0 6.5 5.5 z - 7.510 15.48 3.11 0.31 5.19 6.68 5.74 w - 7.84 15.51 3.12 0.33 5.21 6.68 5.73 Q - 7.66 15.49 3.12 0.31 5.21 6.67 5.74 - 1.96 15.31 2.54 0.37 5.29 7.04 5.47 From Table 2. the . f' ~ ~kl can be estl.ID.S.tes o any .:;:_; . . -1J found. It is seen that in this example the estimates z t w and_ Q do not differ much. The estimates Q tend to lie bet·.ieen Z and w. In the example the sample sizes vary from 2 to 6 while c=l6. The results seem to indicate that for such a small variation of sample sizes relative to the value of c, the weight.ed estimates will not much change the estimates Z • . ... ·;::. ·. To see the effect for smaller c when the variation from 2 to 6 of' sample sizes seems more important, we select the factor combinations {A1 ,B1 ), (A4,B1 ) and (A4,B2 ). Then we have c=3 and sample sizes 6,6 and 2. The estim~tes are given in Table 3. Plassical - 11 -Table 3. Jll - ~41 j'll _ _g42 _g 41 - ~ 42 y 15.8 18,05 3.1 z 15.52 18.3 2.78 w 15.66 18.40 2.74 Q 15.61 18.38 2.77 14.72 17.78 3.06 It is seen that the estimate of ,.J ~ 9- 1r 3 41 based on W and are closer to the original estimate y than the estimate based on z. This is as should be expected since there are 6 + 6 observations behind the estimate of the difference 511- S 41 , while there are 6 + 2 observations behind the other estimates. Q - 12-REFERENCES ~1) HODGES, J.L. JR: and LE~\NN, E.L. (1961). Comparison of the normal scores and Wilcoxon tests. Proc.Fourth.Berkeley Symp. Math. Statist. Frob. 1 307-318. Univ. of California Press. L:2] HODGES, J.L.,JR. and LEHMANN, E.L. (1963). Estimation of location based on ranks. Ann.Math. Statist. 34 598-691. ~3-~ LEIDviANN, E.L. (1963). Robust estimation in analysis of -- variance. Ann. Math. Statist. 34 957-966. 

A NOTE ON ROBUST ESTIMATION IN ANALYSIS OF VARIANCE

Abstract

Similar works

Full text

Available Versions

NORA - Norwegian Open Research Archives