The class of all linear predictors of minimal order for a stationary vector-valued process is specified in terms of linear transformations on the associated Hankel covariance matrix. Two particular transformations, yielding computationally efficient construction schemes, are proposed

Baram, Yoram

English

NASA Technical Reports Server

NASA Technical Memorandum lo0069 
Linear Prediction of Stationary 
Vector Sequences 
Yoram Baram 
{ B A S A - T f l - l O O C 6 9 )  L I I L A R  EEEPXCTIOBI OF I88-18366 
S' lATICPBIiY VECTOR SEQUENCES ( E A S A )  14 p 
C S C L  32B 
Uaclas 
G3/66 0125852 
March 1988 
National Aeronautics and 
Space Administration 
https://ntrs.nasa.gov/search.jsp?R=19880008982 2020-03-20T08:25:46+00:00Z
~~~ 
NASA Technical Memorandum 100069 
Linear Prediction of Stationary 
Vector Sequences 
Yoram Baram, Ames Research Center, Moffett Field, California 
March 1988 
National Aeronautics and 
Space Administration 
Ames Research Center 
Moffett Field, California 94035 
LINEAR PREDICTION OF STATIONARY VECTOR SEQUENCES 
Yoram Baram* 
SUMMARY 
The class of all linear predictors of minimal order for a stationary vector- 
valued process is specified in terms of linear transformations on the associated 
:lankel covariance matrix. Two particular transformations, yielding computationally 
efficient construction schemes, are proposed. 
1. INTRODUCTION 
The prediction of a vector-valued stationary sequence possessing a linear 
representation of finite order can be approached, in principle, by first construct- 
ing such a representation from the process second order statistics and then matching 
a Kalman filter to it. The construction of a linear representation can be performed 
by spectral factorization, as suggested by Anderson [ l ]  o r  by stochastic realiza- 
tion, as suggested by Faurre [2]. Given the covariance function of the process in 
the time domain, both approaches would be indirect, as the first requires the trans- 
formation to the frequency domain, while the second requires the intermediate so lu -  
tion of a different problem, namely, the deterministic input-output realization 
problem. 
Riccati equation. 
suggested by Faurre [3] and by Son and Anderson [4], also require the solution of 
Riccati equations. 
The construction of the Kalman filter further requires the solution of a 
More direct predictor constructions from the covariance function, 
In this paper we propose a direct approach to the construction of linear pre- 
dictors for stationary vector sequences from the covariance function. 
inspired by Akaike's coordinate-free realization concepts [ 5 ] ,  is based on simple 
geometric principles. 
of the class of all minimal order linear predictors, in terms of linear transforma- 
tions on the Hankel covariance matrix associated with the sequence. It does not 
give rise to Riccati or  Lyapunov equations. 
tion matrices defines specific predictor construction techniques. 
The approach, 
It suggests an explicit coordinate-dependent characterization 
The selection of particular transforma- 
*Y. Baram is a Senior Research Associate of the National Research Council at I 
the NASA Ames Research Center, Moffett Field, CA 94035, on sabbatical leave from the 
Department of Electrical Engineering, Technion, Israel Institute of Technology, 
Haifa 32000, Israel. 
1 I 
2. GENERALIZED FORMULATION 
Let the covariance function of a zero-mean, full rank stationary sequence 
yn E Rm be given by 
where E denotes the expectation operation, and suppose that there exists a posi-  
tive integer P and a set of scalars al, ..., ap,l, a, = 1 ,  such that for any n z P 
P- 1 
k = O  
It is well known that when 
driven by white noise, a relationship of the type (2.2) is satisfied. 
yn is generated by a finite dimensional linear system 
The prediction problem is one of estimating the values of yn+k, k = O,l, ..., 
T T  T T  given the values of ynel, yn-2 ,... . 
and Yi = (Y,,Y,+~,. ..) 
Let us denote by Y- = (y,-,,~,-~ ,..., yo) n 
the past and the future vectors at time n. Let yk,i T T  
denote the ith element of yk and let YkIn-l denote the linear mean-square 
projection of yk on Yi. Let us further denote by (YAlYi) the space generated by 
)T the k-step past and future vectors at time n and T T  Yi(k) = (Yn,Yn+19".9Yn+k,l 
by 'nIn-1 (k) the linear mean-square projection of Yi(k) on Yi. We have 
where 
R(k,n) = E{YA(k)Yi) 
R2 
R3 
Rk+ 1 
... 
... 
... 
R 
2 
I t  f o l l o w s  
of R(k ,n )  
spanned  by 
from (2 .2 )  t h a t  for a n y  k 1 P t h e  rows beyond t h e  f i r s t  P rn rows 
are l i n e a r l y  d e p e n d e n t  on  t h e  p r e v i o u s  o n e s .  The s p a c e  ( Y A I Y ; )  is t h e n  
T .  
I t  c a n  a lso be s e e n  from (2 .2 )  t h a t  t h e  co lumns  of R ( k , n )  beyond t h e  f i r s t  P - rn 
columns are l i n e a r l y  d e p e n d e n t  on t h e  p r e v i o u s  o n e s .  I t  fol lows t h a t  t h e  f i r s t  
maximal se t  of l i n e a r l y  i n d e p e n d e n t  rows of R(P ,n )  is indexed  as t h e  se t  of  such  
rows of t h e  m a t r i x  
R = E ( Y i ( P ) [ Y i ( P ) ]  T 1 = 
. - -  R p  
1 
... R 1  R2 
R2 3 R 
RP+ 1 
... 
R2P- 
The s u b s e q u e n t  a n a l y s i s  w i l l  r e q u i r e  some f u r t h e r  n o t a t i o n .  Let A and  B be 
any  two matrices s u c h  t h a t  A h a s  no  fewer rows and no  fewer columns  t h a n  B .  Let 
u s  d e n o t e  by 
maximal set of l i n e a r l y  i n d e p e n d e n t  rows of B. S i m i l a r l y ,  l e t  u s  d e n o t e  by cg (A)  
t h e  m a t r i x  whose columns  are t h o s e  columns of A i ndexed  as t h e  f i r s t  maximal se t  
of l i n e a r l y  i n d e p e n d e n t  columns of B .  We w i l l  f i n d  i t  c o n v e n i e n t  t o  write these 
row and column s e l e c t i o n  o p e r a t i o n s  i n  terms of m a t r i x  p r o d u c t s  
rB(A) t h e  m a t r i x  whose rows are those rows of A ,  i ndexed  as  t h e  f irst  
A r ( A )  E r A B B 
and 
A'  cB(A) 5 AcB 
where 
For i n s t a n c e ,  s u p p o s e  t h a t  A has d imens ion  5 x 7 and  t h a t  rows 1 ,  3 ,  and  4 and 
columns 1 ,  2 ,  and  5 of B form maximal i n d e p e n d e n t  sets. Then 
rA and  cA B B are p r o p e r l y  d i m e n s i o n e d ,  f u l l  r a n k  matrices of z e r o s  a n d  o n e s .  
B -  
e 
and 
1 0 0 0 0 0 0  
A 0 1 0 0 0 0 0  CB = 
0 0 0 0 1 0 0  [ 
Since it may be assumed that the matrices involved have compatible dimensions, we 
will use the somewhat abbreviated notation 
~ 
A A rB z rB , cB cB 
Let M and N be arbitrary nonsingular square matrices of dimension rn-P and 
let 
and 
‘n = MYnln-l + ( e )  
Zn = NY-(P) n 
Let us further define 
x = r  X 
M R N ~  
and 
T T  
ZT n = ‘nCMRNT 
I It can be seen that xn 
sense that each element of the latter space can be obtained by a linear combination 
of the elements of xn. 
(YilYi+l). The latter may be decomposed as (YiIYi+l) = (YnlY;) o Vn where Vn is 
the space generated by 
is a vector of minimal dimension which spans (YiIY,) in the 
+ Clearly, x ~ + ~  belongs to ( Yn+, I Yi+ ) , which is included in 
+ 
4 
and @ denotes the Cartesian product. Since Vn is orthogonal to (Yi IY,), it 
follows that there exist matrices An and B, such that 
+ B v  ( 2 . 4 )  = AnXn n n n+ 1 X 
Multiplying (2.4) on the right by 
is orthogonal to Yi, hence, to zn, we 
T 
T z n ,  taking expectation and noting that 
obtain 
v, 
where 
T T T  (P)[Yi(P)l I N  c T E{xnzn) = r 
TME{'n(n-l M R N ~  MRN 
= r  M R N ~ C ~  
M R N ~  M R N ~  
- -T = MRN 
where 
- - 
M = r  M ,  N = c  N 
M R N ~  M R N ~  
T It can be seen that E{xnzn) is a non-singular matrix. We also have 
m m r n m  
- S-T = MR N 
where 
3 R 
R4 
pP+2 
... 
... 
... 
( 2 . 6 )  
(2 .7)  
(2.8) 
5 
Substituting (2.6) and (2.7) into (2.51, we obtain 
= A = M R ~ ~ ~ ( ~ ; R N  -T -1 
*n 
It follows from the definition of x, that there exists a matrix C, such 
that 
Yn = c x  n n  + v n  ( 2 . 9 )  
Multiplying (2.9) on the right by z;f and taking expectation, we obtain 
(2.10) 
where 
where (MRNT)l is the first block row of MRNT. Hence, 
cn = c = ( M R N ~ ) ~ ~  T(i?~iT)-’ 
MRN 
Multiplying (2.4) on the right by vx and taking expectation, noting that v, is 
orthogonal to xn, we obtain 
Bn = B = E{X~+,V~)[E{V~V~)I T 
T -1 
where 
T = - MR - AIIC - 
and 
(2.11) 
(2.12) 
= Ro - CnCT (2.13) 
6 
where 
T II = E(xnxn 1 
and 
R = (RT R2 T RP) T T  - (2.14) 
We next derive the term nCT, which appears in both (2.12) and (2.13).  Noting that 
T k T  y 1 = CA nC Rk = E{Yn+k n 
it can be seen that 
Where E is defined by (2.14) and 
(2.15) 
Since Ynln-l(P) = Ox, and since the covariance rank of x, is the same as that 
of Ynln-l(P), it follows that 0 is full rank. Hence, 
rICT = (0 T 0) -1 0 T E 
which, substituted in (2.12) and (2.13), completes the derivation of the matrix B, 
defined by (2.11). In summary, the generalized predictor is given by 
x = o  
0 
X = AX + Bvn , n n+ 1 
(2.16) 
k = CA xn 'n+k In- 1 
7 
where 
- S-T -T - 1  A = MR N ( ~ R N  
T T  C = (MRN 
M R N ~  
T B = [i - A ( O  0 ) -  
MRiT ) - 
T T -1 T 
0 ]RIRo - C ( e  0) 0 E] 
(2.17) 
(2 .18)  
(2.19) 
l and 
v = yn - cxn n 
The class of all minimal order linear predictors for the sequence 
defined by (2.16).  
the nonsingular matrices M and N. Two particular selections are suggested in the 
following section. We note that 
yn is now 
A particular member of the class is specified by selection of 
X = Axn + Bvn n+ 1 
(2.20) 
yn = cxn + vn 
With 
Yn * 
A ,  B, C, and vn as defined above, is a minimal realization of the sequence 
3 .  SPECIFIC CONSTRUCTIONS 
A .  Original Coordinates 
When the matrices M and N are taken t o  be identity matrices, the vectors 
Y-(P) and Yi(P) are left in their original coordinates. The minimal redictor for 
tgis choice is specified in the preceding section, with MRNT and MRSN' replaced 
simply by R and RS. 
early independent rows and linearly independent columns of R. The following proce- 
dure selects a maximal set of such rows. Extension to the selection of linearly 
independent columns is immediate. 
The predictor is essentially defined by maximal sets of lin- 
The procedure consists of checking the rank deficiency of matrices Ri, con- 
sisting of the first i-1 linearly independent rows of R and the row following 
them. If Ri is rank-deficient, its last row is replaced by the next row of R 
and the rank check is repeated. If Ri has full-column rank, the next row of R 
a 
is attached to Ri to form Ri+l and a rank check is performed. This procedure is 
continued until rank Ri = M o r  until all the rows of R have been checked. The 
final Ri will contain a maximal set of iinearly independent rows of R. This 
procedure can be extended to the construction of a linear predictor o r  a miqimal 
realization from a sample covariance sequence, employing a recently proposed statis- 
tical rank test method [6]. 
B. Canonical Coordinates 
Construction in the original coordinates requires a sequence of rank checks on 
submatrices of the Hankel covariance matrix R. A computationally more attractive 
technique may be obtained by selecting the matrices M and N so as to diagonalize 
the matrix MRNT. Let M and N be the matrices of normalized eigenvectors of RRT 
and R R, respectively, corresponding to the nonzero eigenvalues. These may be 
obtained directly from the singular value decomposition of R (see, e.g., [ 7 ] ) .  It 
can be readily seen that 
T 
Denoting 
0 C 
where s , ,  ..., sp are the squared nonzero singular values of R, the predictor is 
now specified by the matrices 
A = [ I p  O]MRSNT [ig S-’ (3.2) 
c = [S’ 0 e.. 01s-I (3.3) 
and B is defined by (2.191, with r defined by (3.l), 0 defined by (2.15), 
and A and C defined by (3.2) and (3.3). We note that while the above procedure is 
computationally more effective than the one performed in the original coordinates, 
when the exact covariance function is known, it is not presently extendable by means 
of statistical inference t o  the case where only a sample covariance sequence is 
given. We also note that the above construction is different from the canonical 
variates realization method suggested by Larimore [8], which produces an approximate 
M R N ~  
9 
representation for the process, even when the latter possesses a finite order reali- 
zation. 
with the parameters specified above. 
The present construction produces an exact realization of the form (2.20) 
4. CONCLUSION 
A direct approach t o  the construction of a minimal order linear predictor for a 
stationary vector sequence from its covariance function has been proposed. 
class of all such predictors has been characterized in terms of linear transforma- 
tions on the Hankel covariance matrix. 
specified. 
The 
Two specific construction schemes have been 
10 
REFERENCES 
1.  B. D. 0. Anderson, "The Inverse Problem of Stationary Covariance Generation," *!. 
Stat. Phys., Vol. 1, No. 1,  pp. 133-1117', 1969. 
2. P. L. Faurre, "Identification par minimisation d'une representation markovienne 
de processus aleatoire," Symp. on Optimization, Nice, June 1969, Springer- 
Verlag Lecture Notes on Mathematics, 132. 
3. P. L. Faurre, "Stochastic Realization Algorithms,'' in System Identification, 
R. K. Mehra and D. G .  Lainiotis eds., Academic Press, New York, 1976. 
4 .  L. H. Son and B. D. 0. Anderson, "Design of Kalman Filters Using Signal Model 
Output Statistics," Proc. IEE, Vol. 120, No. 2, pp. 312-318, February 1973. 
5. H. Akaike, "Stochastic Theory of Minimal Realization," IEEE Trans. on Automatic 
Control, Vol. AC-19, No. 6, December 6, 1974. 
6. Y. Baram and B. Porat, "Identification of Minimal Order State Space Models from 
Stochastic Input-Output Data," to appear in SIAM J. on Contr. Optimization, 
Vol. 26, No. 1, January 1988. 
7. C. Golub and W. Kahan, "Calculating the Singular Values and Pseudo-Inverse of  a 
Matrix," SIAM J. Numerical Analysis, Ser. B, Vol. 2, pp. 205-224, 1965. 
8. W. E. Larimore, "System Identification, Reduced Order Filtering and Modelling 
via Canonical Variates Analysis," Proc. Amer. Contr. Conf., H. S. Rao and T. 
Dorats, eds., IEEE, New York, pp. 445-451, 1983. 
11 
Report Documentation Page 
1. Report No. 2. Government Accession No. 
NASA TM-100069 
4. Title and Subtitle 
Linear Prediction of Stationary Vector Sequences 
7. Authork) 
Yoram Baram 
3. Recipient’s Catalog No. 
5. Report Date 
March 1988 
6. Performing Organization Code 
8. Performing Organization Report No. 
A-8008 7 
10. Work Unit No. 
9. Performing Organization Name and Address 
Ames Research Center 
Moffett Field, CA 94035 
17. Key Words (Suggested by Authorls)) 
System compensation 
Inversion identification 
12. Sponsoring Agency Name and Address 
National Aeronautics and Space Administration 
Washington, DC 20546 
18. Distribution Statement 
Unclassified - Unlimited 
Subject category - 66 
505-66-1 1 
19. Security Classif. lof this report) 20. Security Classif. (of this page) 
Unclassified Unclassified 
11. Contract or Grant No. 
21. No. of pages 22. Price 
12 A0 2 
13. Type of Report and Period Covered 
Technical Memorandum 
14. Sponsoring Agency Code 
I 
15. Supplementary Notes 
Point of Contact: Heinz Erzberger, MS 210-9, Ames Research Center, 
Moffett Field, CA 94035, (415) 694-5425 or FTS 464-5425 
16. Abstract 
The class of all linear predictors of minimal order for a stationary vector- 
valued process is specified in terms of linear transformations on the associated 
Hankel covariance matrix. Two particular transformations, yielding computation- 
ally efficient construction schemes, are proposed. 


Linear prediction of stationary vector sequences

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server