We study the problem of classi¯cation for multivariate repeated measures data with struc- tured correlations on both time and spatial repeated measurements. This is a very important problem in many biomedical as well as in engineering ¯eld. Classi¯cation rules as well as the algorithm to compute the maximum likelihood estimates of the required parameters are given.Kronecker product covariance structure; Repeated observations; Maximum Likeli- hood Estimates.

Anuradha Roy

Ricardo Leiva

English

Research Papers in Economics

WP # 0090MSS-253-2009  Date June 23, 2009 
THE UNIVERSITY OF TEXAS AT SAN ANTONIO, COLLEGE OF BUSINESS 
Working Paper SERIES 
     
 
 
 
 
 
 
 
 
 
October 2006 
ONE UTSA CIRCLE    
SAN ANTONIO, TEXAS  78249-0631         
210 458-4317  |  BUSINESS.UTSA.EDU 
Copyright © 2009, by the author(s). Please do not quote, cite, or reproduce without permission from the 
author(s). 
 
 
Anuradha Roy 
Department of Management Science and Statistics 
University of Texas at San Antonio 
 
Ricardo Leiva 
Universidad Nacional de Cuyo 
Classification Rules for Multivariate Repeated Measures Data with Equicorrelated 
Correlation Structure on both Time and Spatial Repeated Measurements Classi¯cation Rules for Multivariate Repeated Measures Data with
Equicorrelated Correlation Structure on both Time and Spatial
Repeated Measurements
Anuradha Roy
Department of Management Science and Statistics
The University of Texas at San Antonio
San Antonio, TX 78249, USA
E-mail: aroy@utsa.edu
Ricardo Leiva
Departamento de Matem¶ atica
F.C.E., Universidad Nacional de Cuyo
5500 Mendoza, Argentina
Abstract
We study the problem of classi¯cation for multivariate repeated measures data with struc-
tured correlations on both time and spatial repeated measurements. This is a very important
problem in many biomedical as well as in engineering ¯eld. Classi¯cation rules as well as the
algorithm to compute the maximum likelihood estimates of the required parameters are given.
Key Words: Kronecker product covariance structure; Repeated observations; Maximum Likeli-
hood Estimates.
JEL Classi¯cation: C10, C13
1 Introduction
We develop classi¯cation rules for multivariate repeated measures data with structured corre-
lations on repeated measures on both spatial as well as over time. The available classi¯cation
rules for multivariate repeated measures data consider structured correlation only on repeated
measures over time. Nevertheless, in many biomedical applications just one variable is measured
on di®erent parts of the body and repeatedly over time, where use of structured correlations on
repeated measures on spatial as well as over time would be natural/ bene¯cial. These problems
are computationally very challenging, as it is not possible to tract them analytically or ¯nd any
closed form solution.
Classi¯cation problem on multivariate repeated measures data, where measurements on a
number of variables are measured repeatedly over time, was ¯rst studied by Gupta (1980, 1986).
1Roy and Khattree (2005 a, b; 2007) considered the problem in small sample situation by assum-
ing Kronecker product structure on variance covariance matrix. They assumed equicorrelated or
compound symmetry correlation structure on the repeated measures in their 2005 b paper, and
an autoregressive of order one
¡
AR(1)
¢
structure on repeated measures in their other two papers.
In many clinical trial problems it is found that the measurements on a single variable is measured
on di®erent body positions and repeatedly over time. For example positron emission tomography
(PET) imaging aids in diagnosing di®erent types of dementia. A healthy brain shows normal
metabolism levels (measurements) throughout the scan. Low metabolism in the temporal and
parietal lobes (sides and back) on both sides (sites) of the brain is seen in Alzheimer's disease.
Repeated measurements of PET scan may diagnosis a patient with Alzheimer's disease. In an-
other example, for the classi¯cation of patients between two di®erent osteoporosis drug treated
populations in two clinical trials. Osteoporosis can be detected by a test of Bone Mineral Den-
sity (BMD), the assessments of which are obtained at di®erent anatomic locations of the body,
such as the spine, radius, femoral neck and the total hip and all the measurements were observed
at repeatedly over time. In this article we develop classi¯cation rules for these kinds of data.
Di®erent time points as well as di®erent sites may have di®erent measurement variations for the
variables, and we should take these variations into account while analyzing these kinds of data. It
is well known (Hand, 1997) that the correlation structure on the repeated measurements follows
a simple pattern such as compound symmetry or a ¯rst-order autoregressive
¡
AR(1)
¢
structure
as opposed to the unstructured variance-covariance matrix, where the mean vectors and the vari-
ances and covariances among the pu variables are arbitrary. Therefore, for both the data sets it
is expected that measurement variation over sites as well as over time both will have patterned
covariance structures. In other words, marginal variance-covariance matrices over di®erent sites
as well as over di®erent time points will have patterned covariance structures. In this paper we
will develop classi¯cation rules for multivariate repeated measures data where both the marginal
variance-covariance matrices over di®erent sites as well as over di®erent time points have patterned
covariance structures.
Let yjr;ts be the measurement on the rth individual at the sth site (location) and at the tth
time point in the jth population; r = 1;:::;n; s = 1;:::;u; t = 1;:::;p;j = 1;:::;k: Let yjr;t be
the u-variate vector of all measurements corresponding to the rth individual at the tth time point,
that is, for each r; and t; yjr;t is obtained by stacking the response of the rth individual at the tth
time point at the ¯rst site (location), then stacking the response at the second site and so on. Let
yjr = (y0
jr;1;y0
jr;2;:::;y0
jr;p)0 be the pu-variate vector of all measurements corresponding to the
rth individual. For two populations, Y j =
£
yj1;yj2;:::;yjn
¤
be nj independent random samples
from populations Npu
¡
¹j;­j
¢
; where ¹j 2 Rpu and the matrix ­j is assumed to be pu £ pu
positive de¯nite matrix. We assume the form of the covariance matrix ­j as
­j
pu£pu
= V j
p£p
­ ¢j
u£u
;
2where both V j and ¢j have equicorrelated structures. However, this may result in identi¯ability
problem. We circumvent this problem by taking V j as a equicorrelated correlation structure and
¢j as a equicorrelated covariance structure. The matrix ¢j has the form
¢j =
¡
¾2
j;0 ¡ ¾2
j;1
¢
Iu + ¾2
j;1Ju;
where Iu is the u £ u identity matrix, 1u is a u £ 1 vector containing all elements as unity,
Ju = 1u10
u. It is well known that
¢¡1
j =
¡
¾2
j;0 ¡ ¾2
j;1
¢¡1 Iu +
1
u
h¡
¾2
j;0 + (u ¡ 1)¾2
j;1
¢¡1 ¡
¡
¾2
j;0 ¡ ¾2
j;1
¢¡1i
Ju:
That is, ¢¡1
j also has the form
¢¡1
j = hjIu + kjJu; (1)
where
hj =
¡
¾2
j;0 ¡ ¾2
j;1
¢¡1 ;
and
kj =
1
u
h¡
¾2
j;0 + (u ¡ 1)¾2
j;1
¢¡1 ¡
¡
¾2
j;0 ¡ ¾2
j;1
¢¡1i
The determinant of ¢j is given by
j¢jj =
¯ ¯¾2
j;1 ¡ ¾2
j;0
¯ ¯u¡1 ¯ ¯¾2
j;0 + (u ¡ 1)¾2
j;1
¯ ¯: (2)
The correlation matrix V j; j = 1;2 is given by
V j = (1 ¡ ½j)Ip + ½j1p10
p:
The elements vlm
i of V ¡1
j is given by
vlm
j =
8
> > <
> > :
1 + (p ¡ 2)½j
(1 ¡ ½j)f1 + (p ¡ 1)½jg
; if l = m;
¡
½j
(1 ¡ ½j)f1 + (p ¡ 1)½jg
; if l 6= m:
(3)
The determinant of V j is given by
jV jj =
¡
1 + (p ¡ 1)½j
¢
(1 ¡ ½j)p¡1;j = 1;2: (4)
Since V j has to be positive de¯nite, we should have ¡ 1
p ¡ 1 < ½j < 1. However, we further
assume that 0 < ½j < 1:
32 Classi¯cation Rules
Case 1: ­1 = ­2.
Sample classi¯cation rule is given by:
Classify an individual with response y to Population 1 if
(b ¹1 ¡ b ¹2)
0
³
b V
¡1
­ b ¢
¡1´
y ¸
1
2
(b ¹1 ¡ b ¹2)
0
³
b V
¡1
­ b ¢
¡1´
(b ¹1 + b ¹2)
0 ;
and to Population 2 otherwise.
Maximum likelihood estimation of ¹1; ¹2; V ; and ¢: Let n = n1 + n2 be the total number of
random samples Y j =
h
yj1;yj2;:::;yjnj
i
from Population j;j = 1;2: Here we assume ¹j =
(¹j;ts)
0
t=1;:::;p;s=1;:::;u. Using (1) and(2) the log likelihood function lnL(¹1;¹2;V ;¢;Y 1;Y 2) is
given by
lnL = ¡
npu
2
ln(2¼) ¡
nu
2
lnjVj ¡
np(u ¡ 1)
2
ln
¯
¯¾2
0 ¡ ¾2
1
¯
¯ ¡
np
2
ln
¯
¯¾2
0 + (u ¡ 1)¾2
1
¯
¯
¡
1
2
2 X
j=1
nj X
r=1
p X
m=1
m+1 X
l=m¡1
u X
s=1
hvlm (yjr;ls ¡ ¹j;ls)(yjr;ms ¡ ¹j;ms)
¡
1
2
2 X
j=1
nj X
r=1
p X
m=1
m+1 X
l=m¡1
u X
s=1
u X
s¤=1
kvlm (yjr;ls ¡ ¹j;ls)(yjr;ms¤ ¡ ¹j;ls¤): (5)
An alternative expression for lnL is
lnL = ¡
npu
2
ln(2¼) ¡
n
2
lnjV ­ ¢j ¡
1
2
tr(V ­ ¢)
¡1 (S1 + S2)
¡
1
2
tr(V ­ ¢)
¡1
2 X
j=1
nj
¡
yj ¡ ¹j
¢¡
yj ¡ ¹j
¢0 :
where
Sj =
nj X
r=1
¡
yjr ¡ yj
¢¡
yjr ¡ yj
¢0 ; for j = 1;2;
and yj is the sample mean vector for the jth group. The vector yj =
³
y0
j;1;y0
j;2;:::;y0
j;p
´0
; with
yj;t = 1
nj
nj P
r=1
yjr;t =
¡
yj¢;t1;yj¢;t2;:::;yj¢;tu
¢0 ; for t = 1;:::;p. It is obvious that the MLEs of ¹j
are b ¹j = yj for j = 1;2: Now, replacing ¹j by b ¹j the log likelihood function reduces to
lnL = ¡
npu
2
ln(2¼) ¡
n
2
ln(jV j
u j¢j
p) ¡
1
2
tr
¡
V ¡1 ­ ¢¡1¢
S;
where S = S1 + S2. By substituting the values of jV j and V ¡1 in the above equation we get
lnL = ¡
npu
2
ln2¼ ¡
n(p ¡ 1)u
2
ln(1 ¡ ½) ¡
nu
2
lnf1 + (p ¡ 1)½g
¡
np
2
lnj¢j ¡
1
2(1 ¡ ½)
c¤
1 +
½
2(1 ¡ ½)f1 + (p ¡ 1)½g
d¤
1; (6)
4where c¤
1 = tr[(Ip ­ ¢¡1)S] and and d¤
1 = tr[(Jp ­ ¢¡1)S]:
Di®erentiating (6) with respect to ½, equating it to zero and simplifying we get,
(p ¡ 1)k0½3 + fk0 ¡ (p ¡ 1)k0 + (p ¡ 1)2c¤
1 ¡ (p ¡ 1)d¤
1g½2
+ f2(p ¡ 1)c¤
1 ¡ k0g½ + (c¤
1 ¡ d¤
1) = 0; (7)
where k0 = nu(p ¡ 1)p. Alternatively, from (5) we get
lnL = ¡
npu
2
ln(2¼) ¡
nu
2
jV j ¡
np(u ¡ 1)
2
ln
¯
¯h¡1¯
¯ ¡
np
2
ln
¯
¯m¡1¯
¯
¡
1
2
h
µ
b¤
1;1 ¡
1
u
b¤
1;2
¶
¡
1
2u
mb¤
1;2;
where
h =
1
¾2
0 ¡ ¾2
1
;
m =
1
¾2
0 + (u ¡ 1)¾2
1
;
b¤
1;1 =
2 X
j=1
nj X
r=1
p X
m=1
p X
l=1
u X
s=1
vlm ¡
yjr;ls ¡ yj¢;ls
¢¡
yjr;ms ¡ yj¢;ms
¢0 ;
and b¤
1;2 =
2 X
j=1
nj X
r=1
p X
m=1
p X
l=1
u X
s=1
u X
s¤=1
vlm ¡
yjr;ls ¡ yj¢;ls
¢¡
yjr;ms¤ ¡ yj¢;ms¤
¢
:
Di®erentiating (Harville, 1997) the above equation with respect to h¡1 and m¡1 separately and
then equating them to zero we get
d h¡1 =
1
np(u ¡ 1)
µ
b¤
1;1 ¡
1
u
b¤
1;2
¶
;
and d m¡1 =
1
np
b¤
1;2:
After some simpli¯cations we get
b ¾2
0 =
b¤
1;1
npu
; (8)
and b ¾2
1 =
b¤
1;2 ¡ b¤
1;1
npu(u ¡ 1)
: (9)
The MLEs b ½;b ¾2
0 and b ¾2
1 are obtained by simultaneously and iteratively solving (7), (8) and (9)
by substituting the values of vlm; l;m = 1;2;:::;p; from equation (3). The computations can be
carried out by the following algorithm. The MLE of V is obtained from
b V = (1 ¡ b ½)Ip + b ½1p10
p; (10)
5and the MLE of ¢ is obtained from
b ¢ = Iu
¡
b ¾2
0 ¡ b ¾2
1
¢
+ Jub ¾2
1: (11)
Algorithm Outline:
Step 1: Get the pooled sample variance covariance matrix G for the repeated measures. Then
obtain an initial estimate of ½ as b ½o = (10
pG1p ¡ trG)=p(p ¡ 1):
Step 2: Compute b ¾2
0 and b ¾2
1 from (8) and (9), and then compute b ¢ from (11).
Step 3: Compute c¤
1 and d¤
1 using b ¢ obtained in Step 2.
Step 4: Compute b ½ by solving the cubic equation (7). Ensure that 0 < b ½ < 1. Truncate b ½ to 0 or
1, if it is outside this range.
Step 5: Compute the revised estimate b V from b ½ by using (10).
Step 6: Repeat Steps 2 to 5 until convergence is attained. This is ensured by verifying that the
maximum of the absolute di®erence between two successive values of b ½, b ¾2
0 and b ¾2
1 is less than ².
Even though ½ is always between ¡ 1
p ¡ 1 and 1, we have assumed 0 < ½ < 1. Still, b ½ may fall at
the boundary ½ = 1, in which case the standard asymptotic theory may not be directly applicable.
See, Self and Liang, (1987) for more details.
Case 2: ­1 6= ­2 (V 1 6= V 2;¢1 6= ¢2).
Sample classi¯cation rule is given by:
Classify an individual with response y to Population 1 if
2 X
j=1
(¡1)
j¡1
·
y0
j
³
b V
¡1
j ­ b ¢
¡1
j
´
y ¡
1
2
y0
³
b V
¡1
j ­ b ¢
¡1
j
´
y
¸
¸
1
2
2 X
j=1
(¡1)
j¡1
h
ln
¯ ¯ ¯b V j
¯ ¯ ¯
u ¯ ¯ ¯ b ¢j
¯ ¯ ¯
p
+ y0
j
³
b V
¡1
j ­ b ¢
¡1
j
´
yj
i
;
and to Population 2 otherwise.
Using (1) and (2) the the log likelihood function lnL(¹1;¹2;V 1;V 2;¢1;¢2;Y 1;Y 2) is given by
lnL = ¡
npu
2
ln2¼ ¡
n1(p ¡ 1)u
2
ln(1 ¡ ½1)
¡
n2(p ¡ 1)u
2
ln(1 ¡ ½2) ¡
n1u
2
lnf1 + (p ¡ 1)½1g
¡
n2u
2
lnf1 + (p ¡ 1)½2g ¡
n1p
2
lnj¢1j
¡
n2p
2
lnj¢2j ¡
1
2(1 ¡ ½1)
c1 ¡
1
2(1 ¡ ½2)
c2
+
½1
2(1 ¡ ½1)f1 + (p ¡ 1)½1g
d1 +
½2
2(1 ¡ ½2)f1 + (p ¡ 1)½2g
d2; (12)
6where cj = tr[(Ip ­ ¢¡1
j )Sj] and and dj = tr[(Jp ­ ¢¡1
j )Sj]: Di®erentiating (12) with respect to
½j;j = 1;2, equating it to zero and simplifying, results in the following equation
(p ¡ 1)kj0½3
j + fkj0 ¡ (p ¡ 1)kj0 + (p ¡ 1)2cj ¡ (p ¡ 1)djg½2
j
+ f2(p ¡ 1)cj ¡ kj0g½j + (cj ¡ dj) = 0; (13)
where kj0 = nju(p ¡ 1)p. Alternatively from (12) we get
lnL = ¡
npu
2
ln(2¼) ¡
n1u
2
lnjV 1j ¡
n2u
2
lnjV 2j ¡
n1p(u ¡ 1)
2
ln
¯ ¯h¡1
1
¯ ¯
¡
n2p(u ¡ 1)
2
ln
¯
¯h¡1
2
¯
¯ ¡
n1p
2
ln
¯
¯m¡1
1
¯
¯ ¡
n2p
2
ln
¯
¯m¡1
2
¯
¯
¡
1
2
h1b1;1 ¡
1
2
k1b1;2 ¡
1
2
h2b2;1 ¡
1
2
k2b2;2; (14)
where
hj =
1
¾2
j;0 ¡ ¾2
j;1
;
mj =
1
¾2
j;0 + (u ¡ 1)¾2
j;1
bj;1 =
nj X
r=1
p X
m=1
p X
l=1
u X
s=1
vlm
j
¡
yjr;ls ¡ yj¢;ls
¢¡
yjr;ms ¡ yj¢;ms
¢
and bj;2 =
nj X
r=1
p X
m=1
p X
l=1
u X
s=1
u X
s¤=1
vlm
j
¡
yjr;ls ¡ yj¢;ls
¢¡
yjr;ms¤ ¡ yj¢;ms¤
¢
:
After some algebraic simpli¯cation from (14) we get
lnL = ¡
npu
2
ln(2¼) ¡
n1u
2
lnjV 1j ¡
n2u
2
lnjV 2j ¡
n1p(u ¡ 1)
2
ln
¯ ¯h¡1
1
¯ ¯
¡
n2p(u ¡ 1)
2
ln
¯
¯h¡1
2
¯
¯ ¡
n1p
2
ln
¯
¯m¡1
1
¯
¯ ¡
n2p
2
ln
¯
¯m¡1
2
¯
¯
¡
1
2
h1
µ
b1;1 ¡
1
u
b1;2
¶
¡
1
2
h2
µ
b2;1 ¡
1
u
b2;2
¶
¡
1
2u
m1b1;2 ¡
1
2u
m2b2;2:
Di®erentiating (Harville, 1997) the above equation with respect to h¡1
j and m¡1
j separately and
then equating them to zero we get
d h¡1
j =
1
njp(u ¡ 1)
µ
bj;1 ¡
1
u
bj;2
¶
;
and
d m¡1
j =
1
njpu
bj;2:
After simpli¯cation we get
b ¾2
j;0 =
bj;1
njpu
; j = 1;2; (15)
7and b ¾2
j;1 =
bj;2 ¡ bj;1
njpu(u ¡ 1)
; j = 1;2: (16)
The maximum likelihood estimates b ½1; b ½2;b ¾2
10;b ¾2
11, b ¾2
20 and b ¾2
21 are obtained by simultaneously
and iteratively solving (13), (15) and (16). The computations can be carried out by a similar
algorithm presented in Case 1. The MLEs of V j and ¢j are obtained as
b V j = (1 ¡ b ½j)Ip + b ½j1p10
p:
and b ¢j = Iu
¡
b ¾2
j;0 ¡ b ¾2
j;1
¢
+ Jub ¾2
j;1:
Acknowledgements: The ¯rst author was partially supported by the College of Business Sum-
mer Research Grant at the University of Texas at San Antonio.
References
[1] Gupta, A.K., 1986. On a Classi¯cation Rule for Multiple Measurements. Comput. Math.
Applic. 12A, 301-308.
[2] Gupta, A.K., 1980. On a Multivariate Statistical Classi¯cation Model. Multivariate Statistical
Analysis, R. P. Gupta (Ed.). Amsterdam: North-Holland, 83-93.
[3] Hand, D.J., 1997. Construction and Assessment of Classi¯cation Rules. Wiley, England.
[4] Harville, D.A., 1997. Matrix Algebra from Statistician's Perspective, Springer-Verlag, New
York.
[5] Self, S.G., Liang, K., 1987. Asymptotic Properties of Maximum Likelihood Estimators and
Likelihood Ratio Tests Under Nonstandard Conditions. J. Amer. Statis. Assoc. 82(398), 605-
610.
[6] Roy, A., Khattree, R., 2007. Classi¯cation of Multivariate Repeated Measures Data with
Temporal Autocorrelation, Journal of Applied Statistical Science 15(3), 283-294.
[7] Roy, A., Khattree, R., 2005 a. Classi¯cation Based on Multivariate Repeated Measures with
Time E®ect on Mean Vector and an AR(1) Correlation Structure on the Repeated Measures,
Calcutta Statistical Assoc. Bulletin 57, 49-65.
[8] Roy, A., Khattree, R., 2005 b. On Discrimination and Classi¯cation with Multivariate re-
peated Measures Data, Journal of Statistical Planning and Inference 134(2), 462-485.
8

a. Classi¯cation Based on Multivariate Repeated Measures with Time E®ect on Mean Vector and an AR(1) Correlation Structure on the Repeated Measures, Calcutta Statistical Assoc.

Asymptotic Properties of Maximum Likelihood Estimators and Likelihood Ratio Tests Under Nonstandard Conditions.

b. On Discrimination and Classi¯cation with Multivariate repeated Measures Data,

Classi¯cation of Multivariate Repeated Measures Data with Temporal Autocorrelation,

Construction and Assessment of Classi¯cation Rules.

Matrix Algebra from Statistician's Perspective,

On a Classi¯cation Rule for Multiple Measurements.

On a Multivariate Statistical Classi¯cation Model. Multivariate Statistical Analysis,

Classification Rules for Multivariate Repeated Measures Data with Equicorrelated Correlation Structure on both Time and Spatial Repeated Measurements

http://business.utsa.edu/wps/MSS/0090MSS-253-2209.pdf

Classification Rules for Multivariate Repeated Measures Data with Equicorrelated Correlation Structure on both Time and Spatial Repeated Measurements

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics