Includes bibliographical references (p. 6)

Bock, M.E.

Judge, George G.

Illinois Digital Environment for Access to Learning and Scholarship Repository

bHP™EemmSmSm
JHI
UNIVERSITY OF
ILLINOIS LIBRARY
AT URBANA-CHAMPAIGN
BOOKSTACKS
Digitized by the Internet Archive
in 2012 with funding from
University of Illinois Urbana-Champaign
http://www.archive.org/details/comparisonoftrad204judg

3 3 0/K fs
Faculty Working Papers
A COMPARISON OF TRADITIONAL AND STEIN-RULE
ESTIMATORS UNDER SQUARED ERROR LOSS
George G. Judge and Mary E. Bock
#204
College of Commerce and Business Administration
University of Illinois at Urbana-Champaign

FACULTY WORKING PAPERS
College of Commerce and Business Administration
University of Illinois at Urbana-Champaign
September 6, 1974
A COMPARISON OF TRADITIONAL AND STEIN-RULE
ESTIMATORS UNDER SQUARED ERROR LOSS
George G. Judge and Mary E. Bock
#204
I-.:'- .--.i •' •' ! ... ;
28 August, *.*><i
A COMPARISON OF TRADITIONAL AND STEIN-RULE ESTIMATORS
UNDER WEIGHTED SQUARED ERROR LOSS
George G. Judge and Mary Ellen Bock
University of Illinois at Urbana-Champaign
Using a weighted squared error loss measure of goodness maximum likelihood,
pre test and Stein rule estimators are analytically compared and the conditions
necessary for one estimator to be superior to another are developed.
1. Introduction
For the problem of estimating the mean vector of a multivariate normal
distribution or more explicitly for our purposes, the problem of estimating
the K dimensional parameter vector 8_ for the orthonormal regression model
(1.1) v_ * X6 e = XS~V*S + e * Z6 e
,
where y_ and e_ are (T x 1) normal vectors with means Z8_ and £, respectively,
and covariance o 2 IT , the following results have been obtained under a total
squared error loss measure of goodness, with risk E [ (8 - 6) ! (£ - 8.)] J i)
James and Stein [1961] developed a Stein-rule estimator, 0* = (1 - c*/u)£,
which, for K > 3 and < c* < 2(K-2) (T-K)K*" 1 [T-K+2]" 1
, where u = f/f/Ka2 = F-K T_ K)
is the likelihood ratio test statistic, dominated the conventional maximum
likelihood estimator, £ - Z'yJ; ii) Baranchik [1964] and Stein [1966] developed a
positive part variant of the Stein-rule estimator, 9 = I r # . 0-0 (1 - c*/u)8,
~ Lc , °°) —
where I r # ^ (u) is a zero-one indicator function, that dominated the James
and Stein estimator, 6_*; iii) Strawderman [1971] developed minimax rules
which were admissible but did not dominate the Stein rule positive part
estimator, iv) Sclove, Morris, and Radhakrishnan [1972] showed the pre
test estimator, 6 = I r n(u)6, an estimator often used in econometric work,
— [c, °°) — *
is uniformly inferior or dominated by a modified version of the positive
part Stein-rule estimator, 6* ' = I
r
,(u)I
r rt «a (u) (1 - c*/u)£, and v) Efron
and Morris [1973], using empirical Bayes ideas, showed that the Stein positive
rule estimator, 9_ , is a member of a class of good rules that have Bayesian

~2-
properties.
These results apply to the |3 space where the coefficient variances
and weights are equal. In practice nk-ny situations may arise where the
coefficient variances are not all equal or we wish to weight certain elements
of B_ differently than others. Therefore, the purpose of this paper is to
summarize the conditions necessary for the James and Stein estimator [1961]
and its variants to dominate the conventional and pre test estimators for
the regression statistical model under a weighted squared error loss
measure of goodness. It is noted that under situations usually found in
econometric paractice, where nonexperimental data and model building pro-
cedures are employed, these conditions may often not be fulfilled.
2. Estimator Comparisons
To compare the estimators we use the weighted risk function
(2.1) E[t| - evwcS - 6)] - e[(|> - (ys^ws-^ce - 03],
where W is a (K x K) symmetric positive definite matrix and from (1.1),
u
= S 3. If as a special case the weight matrix is W « I., then the weight
-W -k -1 -1
matrix in the space is S S = 5 " (X'X) and for expository purposes
this is the case we will consider, although it should be understood that the
results we present are valid for any positive definite W matrix.
In the comparing the estimators,
_9, will be said to dominate, iB,
if
(2.2) E[(6 - 6)'S"^-/S"^(e - 6)3 - E[(£ - 6) 8 S"\s"^(6 - 6)3 < 0,
for all 9_, with strict inequality holding for some 6L
From the work of Bock, Yancey and Judge [1973] and Bock [1974], we
may write, for the case W - I™., the risk function for the James and Stein
estimator 0*, as

-3-
(2.3) E[(£*> .ys'^S'^Ce*- 0J] = a
2 tr$~ l + a 2 trS -1 c*(T - K)
F{(K -f 2H)" l (K - 2 + 2H)" 1 [c*(T - K + 2) - 2(K - 2)
+ Ci'S'^/ePeCtrS" 1 }) 2H(c*(T - K 2)
- 2{3 f e(trS" 1 }/i'S"
1
^ - 2})]i
wjjere H is a Poisson random variable with parameter £*8/o" 2 .
In order for the risk of the James and Stein estimator, 9*, to be
equal to or less than the risk of the maximum likelihood estimator, 8_,
over the entire range of the parameter space, 0, the expression between
the {} brackets must be zero or negative, For this to occur and thus
E [(£ "" £)' s" C£ - £)3 - E[(£* - £)'S (£* - £)] l 0, it must be true that
K
-1
(2.4) I. d./d, = trS /d {. > 2
(2.5) o < c* < 2(T - KHtrS" 1 d" 1 - 2)/K(T - K + 2),
where the d. are the roots of S" , with d. being the largest. For the
1 L
general weighted case these roots wou~d be associated With the matrix
s~\ s' 1*.
These conditions imply that under the weighted squared error loss, the
superiority of the James and Stein estimator, 6*, relative to the conventional
estimator, §_, depends not only on the number of explanatory variables or
hypotheses, as was one of the requirements for the traditional unweighted loss
in the
_6 space, but also on whether or not trS or the sum of characteristic
-i
roots divided by the largest characteristic root of S is equal to or larger than !
-1
If trS /d
L
< 2, then for no value of c > does the James-Stein dominate
the least squai-es estimator.

-4-
Since the degree of col linearity of the columns of the design matrix
X is related to the magnitude of the roots of S , and ill conditioned X'X
matrix may affect whether or not the risk functions cross at some point in the
parameter space and thus has a direct impact on the choice of estimator. An
analysis of the design matrix for several auto regressive and conventional estimated
economic relationships suggests that the condition trS^/d, > 2 may often
not be fulfilled in econometric work and therefore the James and Stein
estimator does not fulfill the minimax criterion,
These same conditions or requirements (trS Vd > 2) and (2.S) must
Li
also hold in order for the James and Stein positive part estimator, 8_,
and the extension of the Sclove modified positive part Stein-rule or pre-
liminary test estimator, 6_ , to dominate the least squares, §_, and con-
ventional preliminary test, £, estimators, respectively. This means the
appearance of three or Biore regressors and a suitably small c* do not insure,
as they did in the unweighted loss (prediction in the
J3
space) case, that the
risk of these various extensions of the Stein-rule estimators, will be less over
the entire parameter space than conventional and pre test estimators , As a
consequence the condition (2.4) suggests a new rule for determining the range of
d. or in another sense the coefficient weights or degree of collinearity per-
missible among the columns in the design matrix X to permit these extensions
of the Stein-rule estimators to dominate the conventional sampling theory estimators.
3. Concluding Remarks
As we change from the traditional statistical model usually analyzed in
the literature when considering Stein-rule estimators to that of the more
general regression case with a weighted squared error loss function, the
following analytical results hold:
i) In order for the extended Stein-rule estimators specified in this
paper to dominate conventional estimators, the ratio of the sunt of
the characteristic roots of the (X'X) matrix, to the largest root of

:this matrix must be equal to or greater than 2.
ii) If this ratio is not greater than two then some members of the
family of potential risk functions for extensions of the
Stein-rule mators problem, cross the risk function
of the least squares est
iii) If conditions (2A and 2.S) are fulfilled, the Sclove modified
Stein-rule (pre aator, for the general model, is
uniformly superior to the conventional pre test estimator over
the parameter space. Like the conventional pre test estimator
however, its risk function crosses that of the least squares
estimator for large values of the critical value c or small values
of a; the Stein-ruie only yields , ftinima estimator for smaller
values of c or larger values of a than are ordinarily used in practice,
iv) The incidence of col linearity among the columns of the X matrix to the
extent that the X s X matrix is "borderline" full rank, and one root is
small relative to the other roots , means that, the conventional pre
test and the family of .Stein-rule estimators are superior (smaller
risk) to the least squares estimator only over a small interval of
the parameter space and are inferior (larger risk) over a large, and
in some esses, infinite range of the parameter space.
The ans^ :1 results summarized herein suggest that unless the re-
searcher has great confidence that his linear hypotheses are true, under the risk
measure we have employed when col linearity among the columns of the design matrix
is present, there is much to lose and very little to gain by broadening the class
of estimators and using the two stage pre test or Stein-rule procedures.
Finally we note the rules discussed in this paper, with the exception of
those by Strawderman [1971] do not satisfy the conditions necessary for a generalized
Bayes estimator and thus are not admissible. One important area of work for the

-6-
future is t0 evaluate the performance of these under alternative
specifications and loss In regard to alternative measures of per-
formance it shou ited that BednareK-Kozek [1973], using his measure of
.1- - JU
goodness E[S* 2 (9 - 6 has, my making use of the work of
Stein [1SS5] demonstrated ths ?ast s< s estimator is admissible.
References
Biiaranchikj A. }: ''Multiple Re ion and the Estimation of the
Mean of ariate Normal Distribution/' Stanford University
Technics >ort, No. Si.
Bednarek-Ki :, B. (1973): "On Estimation in the Multidimensional Gaussian
Model," . -iHatyki, 13:511-20,
Bock, M. E. (1 ax Estimators of the Mean of a Multi-
variate Nor. distribution ," Ph.D. Thesis, University of Illinois.
Bock, M. E., T. A
.. Eicey, and G. G. Judge (1973): "The Statistical Con-
sequences of Pre tary Test Estimators in Regression/' Journal of
.
jrican statistical Association, 68:109-116.
Sfron, B. and '" >rris {1973) "Stein*s Estimation Rule and Its Compet-
itors roach," Journal of American Statist ical
Association 30.
James, W. and C. S>. "Estimation with Quadratic Loss," Proceedings
of -.:-.;ium on Mathematical Statistic Problems,
Unive of & ess, Berkeley, pp. 361-79.
Sclove, S. S. is, and R. Radhakrishnan (1972): "Non Optiaiality of
Preli . stiraators for the Multinomial Mean," Annals of
Mathematics Si 30.
Stein, C. (1966' : proach to the Recovery of Interblock Information in
lanced igns," Research Papers in Statistics*:
Festschrift for J. Key-man, F. N. David (ed.), John Wiley and Sons,
Inc., New York, pp. 351-66,
ftein C. (1955): "Inadmissibility of the Usual Estimator for the Mean of
a Multivariate Normal Distribution," Proceedings of the Third Berkeley
Symposium , University of California Press, Berkeley, 1:197-206..
Strawderman, W. E. (1371) : "Proper Bayes Minimax Estimates of the Multivariate
Normal Mean," Annals of Mathematical Statistics, 42: 385-88.


UNIVERSITY OF ILLINOIS-URBAN A
3 0112 060296743
mmsLmtwamm
Hf
"HIV
nRHBHSMBRfl
wftpwunmaMaBMMiHimmm
BSbBMUBBBB
KBaamMHm
ffiWSfl
ML.mmSm
illilB
W4MW
WmH
If
bush
11111811
HBtaiMWflUwyi
Bra
m


A comparison of traditional and Stein-rule estimators under squared error loss / 204

IDEALS @ Illinois

https://www.ideals.illinois.edu/bitstream/handle/2142/32877/comparisonoftrad204judg.pdf?sequence=2

A comparison of traditional and Stein-rule estimators under squared error loss / 204

Abstract

Similar works

Full text

Available Versions

Illinois Digital Environment for Access to Learning and Scholarship Repository

IDEALS @ Illinois