There are no author-identified significant results in this report

Hallum, C. R.

Pore, M. D.

English

NASA Technical Reports Server

  
 
 
N O T I C E 
 
THIS DOCUMENT HAS BEEN REPRODUCED FROM 
MICROFICHE. ALTHOUGH IT IS RECOGNIZED THAT 
CERTAIN PORTIONS ARE ILLEGIBLE, IT IS BEING RELEASED 
IN THE INTEREST OF MAKING AVAILABLE AS MUCH 
INFORMATION AS POSSIBLE 
https://ntrs.nasa.gov/search.jsp?R=19800020288 2020-03-21T16:53:11+00:00Z
JU^^ggO JSC-13036
LEC•-10532
t Y^
"IT, "30FR91wom
`Me
f
"" 0808111e Under NASA sponsorship
in the interest of earli an
	 '
semination r
t
^esources Survey
Program infnd without liability
for any useeof; R
g,0-10.
NASA t;rc-
16
A DISCRIMINANT APPROACH TO PARAMETER ESTIMATION
IN THE LINEAR MODEL WITH UNKNOWN
VARIANCE—COVARIANCE MATRIX
C. R. Hallum
National Aeronautics and Space Administration
Lyndon B. Johnson Space Center
Houston, Texas
and M. D. Pore
Lockheed Electronics Company, Inc.
Sy:;:ems and Services Division
Houston, Texan
(E80-10192) A DISCRIMINANT ,APPROACH TO
PARAMETER ESTIMATION IN THE LINEAR MODEL
WITH rj micm O wm VARIANCE - COVARIANCE MATRIX
(1,ockheed Electronics Co.) 10 P	 Unclas
HC A02/MF A01	 CSCL 05B G3/43 00192
Annual Meeting of the American Statistical Association
Chicago, Illinois
August 15-18, 1977
NBO-28789
a
A DISCRIMINANT APPROACH TO PARAMETER ESTIMATION
IN THE LINEAR MODEL WITH UNKNOWN
VARIANCE-COVARIANCE MATRIX
C. R. Hallum
National Aeronautics and Space Administration
Lyndon B. Johnson Space Center
Houston, Texas
M. D. Pore
Lockheed Electronics Company, Inc.
Systems and Services Division
Houston, Texas
ABSTRACT
An estimate of the nonrandom vector, 0, of parameters is obtained
in the linear model Y = XO + c, where c is an unobservable random
vector of disturLances and is assumed to satisfy E(e) = 0 (the
zero vector) and E(ec T ) ='V, with V assumed unknown. The esti-
mate obtained is the one which yields maximal similarity to the
sample Y l ? Y21 •••, Y N via the Sebestyen similarity function.
Under the normality assumption, the resulting estimate is seen
to be an unbiased astimate and justification given for selecting
the maximum likelihood estimate for V in the Gauss-Markov osti-
mato for B.
1. INTRODUCTION
Consider the linear model Y = X0 + e, where Y is an n x 1 observ-
able random vector, X is an n x m matrix of fixed elements and
rank W = m _< n, 6 is an m x 1 nonrandom vector of parameters
to be estimated. and E is an unobservable random vector of dis-
turbances with e assumed to satisfy B(c) =, 0 with unknown
variance-covariance matrix E(ECT V. It is well known that in
case V is known (up to at least a scalar multiple), the Gauss-
Markov theorem [1] applies and the best linear unbiased estimate
of 0 is given by
i(XTV-1X)- XTV-1Y
Other authors [2, 3 0 4] have considered the problem of obtaining
optimal estimates for 0 when V is unknown. Rao [4] showed that
the estimate of S obtained by merely substituting an estimate
A
V for V in equation (1) is not necessarily best; in particular,
it may be possible to use known or inferred knowledge of the
covariance V to obtain an estimator with better characteristics.
Born [2] has written a recursive estimator when V is not known
but is assumed to be blc.;:k diagonal with equal diagonal blocks.
McElroy [3] obtained necessary and sufficient conditions on V
for equation (1) to be equivalent to the least-squares solution
jj	 1
s = I XTx
1
t XTY
In this paper, we assume that the only available information is
that contained in a sample Yl, Y20' 	 YN , and an estimate,,
of 0 is obtained which results in maximal similarity to the given
sample via the Sebestyen [5] similarity function. The resulting
estimate appears in the form of equation (1), with V replaced by
the standard (and in the normal theory case, the maximum likeli-
hood) estimate of the variai.ce-covariance matrix.
2. THE SEBESTYEN INTERSET SIMILARITY FUNCTION
If R  denotes Euclidean n-space and P is the class of finite
sequences of sample observations in R  (i.e., W,Z C P, provided.
W = {Wl, W2' ..., W N I and Z	 jz l , Z 21 ..., ZMJ where
Wi ,Z j E R  for i = 1, 2, ..., N; ]	
1, 2 ?
	., M), the
Sebestyen [5] similarity function is defined as follows.
Definition: If W,Z E P with N and M elements, respectively, and
if A is any m x n matrix, define the function S A : P x P-)-RO , where
R0 is the set of nonnegative real numbers, by
A
(2)
SA(W'Z) = NM La 'E (Wi - Z j ) TATA(Wi - Z j }	 (3)
i=1 j=1
(The superscript T denotes the transpose.) Given a tranpforma
tion A, SA ('W,Z) is a measure of the similarity of the two samples
W and Z in the transformed space (i.e., the resulting space after
transforming R  by A), and if W and Z are random samples from
populations w 1 and 7t 2 , respectively, then SA (W,Z) may be con-
sidered as a measure of the similarity of 
n1 to ff2,
If W,Z E P have sample variance-covariance matrices V 1 and V2,
respectively, that is,
N
V1 N
	
^Wi - W 1 Wi - W1T
i=1	 I
(4)
_	 M	 T_V2 	 M	 ^Zi 	 Z ) (Zi - Z `
i=1	 )
and Tr denotes the trace operator (and S(W,Z) 11 S1(W,Z)],
then the properties below are easily verified.
Properties:
1. SA (W,Z) = Tr [A(V1 + V2 )AT] + (W - Z) TATA(W -Z) .
2. S (W, W) = 2Tr (Vl ) and SA (W, W) = An-Tr ^AVIAT)
3. SA WIZ) = SA (Z,w) .
4. SA (W,Z) > 0 for every W,Z E P and for each m x n matrix.
5. If V E Rn then SA (W, V) = SA (W, M ) = Tr (AVAT)
+ (W - V) T A T A (W - V) .
6. If W = { W1 , W2, ...' WNf and Z	 Z1, Z2, ..." ZM 1, then
1 M
M	 SA(W'{ZJ) 
__ SA (W I Z).
j=l
ri
The Sebestyen decision rule is to classify an unknown u as belong-
ing to category W provided
SA (we {u}) < SB (Z,{u})	 (5)
A transformations for W and Z,
the function
is the discriminant function for
with classification of u into W or Z
the sign of f(u); that is, u is
or Z depending on whether f(u) > 0
where A and B are preselectei
respectively. Consequently,
f(u) = SB ('Z,{ u}) - SA(Wf{ u })
the Sebestyen decision rule,
being accomplished by noting
classified as belonging to W
or f(u) < 0, respectively.
3. A TRANSFORMATION TO MINIMIZE THE INTRASET DISTANCE
Thus far, no specifications have been placed on the transforma-
tion A; however, if A is an orthogonal matrix, the transformation
results in a rotation of the original space whereby distances and,
hence, angles are preserved. If the determinant of A [Det(A)) is
1, A is a volume-preserving transformation. The transformation
of interest in this paper is specified in Theorem 1 1 the proof
of which is dependent on the following well-known relationship
between the arithmetic and geometric mean.
Lemma 1: If di > 0 for i = 1,1 2, • - - , n, then
n	 n	
1/n
n E di	 fj di
	 (6)
with equality holding if and only if d 1 = d2 =	 = dn.
Phc^orom 1: Under the condition Det (A) = 1 and S A is positive
definite, an n x n matrix A minimizes SA (W,W) [that is, the
similarity of a set with itself] if and only if AV 1AT XI,
where a = [Det(V1 )] 1/n and V1 is the sample variance-covariance
matrix of W specified in equation (4).
Proof: If B is any n x n matrix with Det(B) == 1 and A is the
matrix specified in the hypothesis, from Property/ 3 and the fact
that AV1AT 	XI, we have S B (W,W) - SA (W,W) = 2TrIBV1BT )
` 
 - 2nX.
Letting U be the orthogonal matrix such that UBVIBTUT = D. where
D is diagonal, then
2TrIBV1BTl - 2nX = 2Tr(UBVIBTUT ) - 2nX
2Tr(D) - 2nX
n	 n	 1/n
= 2n n	 di -	 di	 (7)
i-1	 i=1
But equation (7) is nonnegative, by Lemma 1, with equality hold-
ing if and only if d 1 = d 2	••• = dn , in which case BVZBT 	 = XI,
which was to be demonstrated. Note that Aw exists if V 1 is
positive definite. Indeed Aw EV, where E is the matrix whose
columns are the eigenvectors of V  and V is a diagonal matrix
whose ith diagonal element is a /R i , i = 1, 2, •••, n, where
^ i	th eigenvalue of V1.
Theorem 1 associates with each sample W E P, a transformation,
AW, with the property that AW causes W to cluster in a spherical
fashion after transformation with uncorrelated variates having
equal variances. If W and Z are samples from populations f1 and
72 and if A and B are selected such that
A = X1-1/2A0	 B	
a2-1/2B0
	 (8)
51'
y _,,. __	 y
where h l 	[Det(V1))l/n^ a2 = (Det(VHI/n , and AO and HO are
determined independently for W and Z, respectively, by Theorem 1,
the effect is that of a normalization of the intraset similarity
in that not only are the intraset distances minimal but
SA (W,W) = SB (Z,Z) = 2n as well (i.e., the normalization gives
each intraset similarity the same value). Moreover, if instead
of a threshold of zero in the Sebestyen decision rule [see
eq. (5)], we choose the threshold
T = 1nDet(Vl)p2
	 (9)
Det(V2)pl
the resulting decision rule is the Bayes maximum likelihood
decision rule [61(wthen the population distributions are Normal)
with equal costs of misclassification and a priori probabilities
pl and p2 for Tr l and Tr 2 , respectively.
4. THE ESTIMATE FOR S
In the weighted least-squares procedure, an estimator of R was
selected which minimized (Y - XS) TV- 1 (Y - XR). The optimal
estimate is specified in equation (l) and the significance of
such an estimator is that of being able to predict, or adjust in
some applications, Y to a given matrix X. Since V is ordinarily
unknown, we proceed as described below.
Collect N sample values; denote this sample by
W _ ; yl' y2' " " YNI and the sample variance-covariance
matrix by
N
V _ N L^ (Yi - Y) (Y i - Y) T	(10)
i=1
If A is selected such that
TAVA = ^I	 (11)
where
CDnt (V) ] 1/n	 (1,2)
then Theorem 1 guarantees that the similarity of W with itself
is a minimum after transformation by A. For prediction or adjust-
ment purposes, what we now want to do is to select the vector
Z = Xp which, after transformation, is more similar to the repre-
sentative sample W than any other such vector. Equi. 3alently, we
want to select ^ such that SA (W,{Z}) is a minimum where Z = X(3
and A satisfies equation (11).
Theorem 2: The value of a which minimizes SA (W,X5) is given by
(XTV-1X)-l -lY 	 (13)
where V is the sample variance-covariance matrix of W, A is the
transformation specified in Theorem 1, and Y is the sample mean
of W.
Proof: From Property 6 of SA,
SA(W,XS)	 Tr(AVAT) + (Y - Xa)TATMY - XR)
	 (14)
Differentiating this expression with respect to $, equating to 0,
and solving for R yields
y 1
R	 XTATAX l - XTATAY 	 (15)
However, from the condition that AVA T = a2
ATA = (1/X) V-1
	
(16)
which results in equation (8) after substitution into equa-
tion (9), which was to be demonstrated.
7
R =
Under the normality assumption on Y [i.e., y u MVN(XR,V)] where
V is unknown, V and are independent; therefore
E(0)  = EN T V -1 X) -1 XTV-lY
= E [ ( XTV 1X)
l 
XTV_ "] E (Y )
(17}
1	 _
E [(XTV-lX XTV 1^ Xa
E [( XTV-1X 1 XTV 1X
_
Consequently,
	
is unbiased under these conditions.
5. SUMMARY
An estimate,
	
of 0 in the linear model Y = XR + e was obtained
such that X^ yielded maximal, similarity to the sample Y l , Y21
•••, Y  via the Sebestyen similarity function. The unobservable
random error term, e, was assumed to satisfy E(e) = 0 (the zero
vector) and E(EeT) = V, with V assumed to be unknown. The
resulting estimate is seen to be in the same form as the standard
Gauss-Markov estimate
„ 
	(XTV-1X)
1 
XTV-lY
	 (18)
except V is replaced by the standard (and under the normality
assumption, the maximum likelihood) estimate of the variance
covariance matrix.
6. REFERENCES
1. Graybill, Franklin A.; Theory and Application of the Linear
Model. Duxbury Press (Belmont, Calif.), 1976.
2. Born, H. G.; and Tapley, B. D.: Sequential Estimation of
State and the Observation Error Covariance Matrix. Instit.
Aero. Astro., 1969.
3. McElroy, F. M.: A Necessary and Sufficient Condition That
Ordinary Least-Squares Estimations Be Best Linear Unbiased.
J. American Statis. Assoc., Vol. 62, pp. 1302-1304 0 1967.
4. Rao, C. R.: Least Squares Theory Using an Estimated Disper-
sion Matrix and Its Application to Measurement of Signals.
Proceedings of the Fifth Berkeley Symposium on Mathematical
Statistics and Probability. Univ. of Calif. Press (Berkeley,
Calif.) Vol. 1, pp. 355-371, 1967.
5. Sebestyen, G. S.: Decision-Making Processes in Pattern
Recognition. MacMillan Company (Yew York), 1962.
6. Anderson, T. W.: An Introduction to Multivariate Statistical
Analysis. John Wiley and Sons, Inc., 1958.
9
_J
file:///C|/Microfiche%20Scans/T 1/ 1980020288/index.txt
C:\Microfiche Scans\T1\1980020288\0001A01.tif,11/17/2009 2:29:22 PM,1,1,1,1,334,276,189,236
C:\Microfiche Scans\T1\1980020288\0001A02.tif,11/17/2009 2:29:25 PM,1,1,2,2,529,277,191,235
C:\Microfiche Scans\T1\1980020288\0001A03.tif,11/17/2009 2:29:29 PM,1,1,3,3,728,277,189,235
C:\Microfiche Scans\T1\1980020288\0001A04.tif,11/17/2009 2:29:32 PM,1,1,4,4,925,277,190,238
C:\Microfiche Scans\T1\1980020288\0001A05.tif,11/17/2009 2:29:33 PM,1,1,5,5,1121,277,190,236
C:\Microfiche Scans\T1\1980020288\0001A06.tif,11/17/2009 2:29:34 PM,1,1,6,6,1318,278,189,235
C:\Microfiche Scans\T1\1980020288\0001A07.tif,11/17/2009 2:29:34 PM,1,1,7,7,1514,278,190,236
C:\Microfiche Scans\T1\1980020288\0001A08.tif,11/17/2009 2:29:35 PM,1,1,8,8,1711,278,190,236
C:\Microfiche Scans\T1\1980020288\0001A09.tif,11/17/2009 2:29:36 PM,1,1,9,9,1908,278,190,236
C:\Microfiche Scans\T1\1980020288\0001A10.tif,11/17/2009 2:29:48 PM,1,1,10,10,2105,279,190,236
file:///C|/Microfiche%20Scans/T1/1980020288/index.txt1 1/17/2009 2:30:01 PM


A discriminant approach to parameter estimation in the linear model with unknown variance-covariance matrix

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server