A procedure for calculating a kxn rank k matrix B for data compression using the Bhattacharyya bound on the probability of error and an iterative construction using Householder transformation was developed. Two sets of remotely sensed agricultural data are used to demonstrate the application of the procedure. The results of the applications gave some indication of the extent to which the Bhattacharyya bound on the probability of error is affected by such transformations for multivariate normal populations

Decell, H. P., Jr.

Marani, S. K.

English

NASA Technical Reports Server

General Disclaimer 
One or more of the Following Statements may affect this Document 
 
 This document has been reproduced from the best copy furnished by the 
organizational source. It is being released in the interest of making available as 
much information as possible. 
 
 This document may contain data, which exceeds the sheet parameters. It was 
furnished in this condition by the organizational source and is the best copy 
available. 
 
 This document may contain tone-on-tone or color graphs, charts and/or pictures, 
which have been reproduced in black and white. 
 
 This document is paginated as submitted by the original source. 
 
 Portions of this document are not fully legible due to the historical nature of some 
of the material. However, it is the best reproduction available from the original 
submission. 
 
 
 
 
 
 
 
Produced by the NASA Center for Aerospace Information (CASI) 
https://ntrs.nasa.gov/search.jsp?R=19770006816 2020-03-22T12:22:23+00:00Z
7f +
E
rf
f
<v^^^^Y OFhoG
DEPARTMENT OF MATHEMATICS	 30
UNIVERSITY OF HOUSTON	 HOUSTON, TEXAS
1.	 1
%ded X92
i
(NASA-CR- 151 130) FzATU kl^ COMB117ATIONS AND	 N77-13759
THE BHATTAC€iAliM CRITERION (Houston Univ.)
11 p HC A02/MF A01
	 CSCL 12A
U.nclas
G3/65 56937
j
i

FEATURE COKIINATIONS AND THE
BHATTACHt1RYYA CRITERION
Henry P. Decell, Jr. and Salma K. Marani
Departmen- of Mathematics
University of Houston
A31STRACT
We develop a procedure for calculating a kxn rank k matrix B
for data compression using the Bhattacharyya bound on the proba-
bility of error and an iterat.ve construction using Householder
transformations. Two sets of remotely sensed agricultural data
are used to demonstrate the application of the procedure. The
results of the applications gave some indication of.the extent to
which the Bhattacharyya bound on the probability of error is af-
fected by such transformation;. for multivariate normal popula-
tions.
1. 114TRODUCTION
For n-dimensional normal classes N(E i) i 1,...,m, the
Bhattacharyya coefficient (Andrews, 1972) for class i and j is
k
r1	 I	 i	 v-
f
given by
p(i,J) = (gigY4 )4 J n {Pi (x)pj (x)}2dx
R
and the Bayes probability of error (Anderson, 1958) (Andrews, 1972)
by
.r
'
Pe ,= 1 -	 3 n max {gip(x) }dxR7 lLiL-m
where	 p i (x)	 denotes the conditional density ofthe random vari-
able	 X	 given that
	
X 'U N(µi :, E )	 and	 gl,...,gm,respectively,
denote the (known) arp iori probabilities of the classes N(,JiEi)
i = 1,...,m.
It has been shown (Andrews, 1972) (Kaileth, 1967) that
Pe < 	 {c^ iq^ }^ J n{pi (X) Pj (x) }2dx.i--i+4 1	 R
3
If one considers a kxn rank k linear transformation B of the ran-
dom variable X (i.e., YEW), then the Bhattacharyya,coefficient
for class i and J `for the classes N(BN.1,BEiBT), i = 1,...,m is:,
PB (i .3) 
= {qig i}fR1 {p i (y , B ) p (y,B)} dY
1
and the Bayes probability of error for the classes 	 N(*Bµi,BEiBT),
i	 1,...,m is:	 I
i
-	 _P (B)	 1	 k max { pi (y . B ) }dyf
- 1LiLm
where	 pi (y,B), i	 1,._..,m denotes the conditional density of the
Trandom variable
	
Y = BX	 given that
	
Y ti N(Bµi,$EiB).	 it follows,
E .
Y
f
M-1
since Pe< P EE	 : , P(i,j), that.
i=1 ' j=i+1
m
	
Pe (B) L P(B) = _ _
	
Pa (i,j)
j	 1
and moreover, (Decell and Quirein, 1973) (Kaileth, 1967), tt
(1) Pe ,L Pe (B) 4- P (B)
(2) Pe' = Pe (B) if and only if p = P(B)
2. THEORETICAL PR.LIMINARIES
Let	 k	 be an integer	 (0 < k < n), and	 N(P.., Ei) i
be n-variate normal populations with arp iori probabilities	 3
gl,•.,,gm'	 We lould like to construct_ a kxn rank k matrix 'B that
will minimize	 p(B).	 The theoretical extent to which this is pos-
sible and the basis for the construction (Decell and Smiley, ,to':
appear) is summarized in the following theorem. 	 Let
C = { u e0:jjujj =1} and	 T(H) ={H=I-2uuT : u e C} denote the
set of Householder transformations on
	 Rn (Householder, 1958).
Theorem. For each positive i, let Hi E: T(H)	 be chosen such that
p•((Ik i Z ) H1) = g.l.b P ((I	 I Z)H)kHeT (H)
and
P ((Tk Z) Hi+1Hi 9 H1) = g.1.b.p((IkIZ)HHi...Hl)HeT (H)
	 i
then,
(3^
	
P'((IkIZ)Hi+1Hi*••Hl) ^5P((IkIZ)Hi...Hl)•
i	 I
(2)	 p((Ik JZ)H i+1 •.. Hl) < p((Ik IZ)Hi ...HlH	 H e T(H)).
(3)	 P((I,k j Z )H.	 H..	 .H )^p((I 	IZ)mi ...H	 H e T(H)).1i+1 i
	 k	 1
r
s
a
(4)	 P(( I I Z)H..g.	 Hg	 H) < o((I	 I Z)H.	 H..• • •g ), H E T(H)k	 ^— (p =1)	 i— (p+1) 1 — '	 k+1 i	 1
and	 p =- 0, ... , i- 2 .
(5)	 The monotone sequence of real numbers
	 ) } 00 	 where
sy
C.
Bi (IkIZ)Hi •• Hl
 is bounded below by Pe
 and hence
lim P(Bi) = g.1.b..1p(Bi)}i-K0	 i
We knowDecell and	 ,(	 Quirein 1973) that there is some kxn rank.
k matrix, say B, that minimizes P(B). If p(B) < g'i'b'{P(Bi)l
Co
we will call the sequence {B.},
=1 sub optimal (optimal in the	
«.
^ 
	
	 i
a
case of equality). There are several results (Decell and Smiley,
1
to appear) that lend credibility to the conjecture that the seq-
uence is optimal and cofinally
 constant beyond the index
i = min{k,n-k}. We will proceed with the develipment of an itera-
tive procedure'for constructing the subject sequence and, finally,
tabulate results of applications to remotely sensed agricultural
data with equal priori class probabilities. The approach (and
its merit) will depend upon the bound provided by the inequality
Pe < P(Bi) i 1,2,..., the non-increasing nature of the sequence
Co{p(Bi)}i_1, and the ability to manipulate the expressions far
p(Bi), i 1,2,..	 in the care of normal populations.
-`'	 3. THE-GRAMENT OF p((IkIZ)H)
i	 We will develop an expression (for the case of normal n-vari,,
r	 ate populations N(N1,Ei), i = 1,...,m) for thegradient of
PMk IZ)H) where H £ T(H) has the form H = I-2 xxT 
x  8.j	
-	 Tx x
This expression will be used :n a steepest descent procedure to'
calculate each Householder transformation Hl , H2 , H3,... des-
cribed in the preceding theorem.. For -m populations N(iE ),
E `
	
	 i 1,...,m it is easy to establish that in order to calculate
Hi+1' one need r Aly apply the steepest descent procedure to the
Bhattacharyya coefficient determined by the populations
	 +
p(Hi .00Hlµ3 ,Hi .06H1Z Hl..:H)
f
r
a
ORIG	 ^1NAL PAGE
OF POOR  ALa yjQU
The expression for Pa	 (i,j) is given by (Andrews, 1972)kIZ)H
(Kaileth, 1967) (for the case of equal a rP iori probabilities
l
P(I 
Z)H (i ^j) =mexp_ 4^1j Q +E^) -l6	 21n	 IEi+EjI`ijk1	 k	
, ^j rZ ^i 17-21 
where d ij	 (IkIZ)H(/.L.-µj)
 and Ei = (Ik lZ)HE iH(Ik IZ) T, in which	
..
case,
M-1
P(( Ikl Z)H) _ E	 P	 (i1J)•	 1
i= L	 ,.+1 (Ik^Z)Hj=	 j
•	 1
°i
If we define
1
^ ^
_..
Fi' - - 4S . (Ei+E . Si . aad Gi ,	 Zl	 in k ^ +E,J	 J	 J	 J	 J	 2	 lil,l
.	 J
we have that the differential oL=P
	
(i -j) is(Ik IZ)H '`
d(P(IkIZ)H(i.j))	 m ex,)(Fij+Gij (d (Fi ) + d(Glj))•
from whence it follows that
{
_	 m
d(P((Ik^Z)H)) = me xp(Fij+Gij ){d(Fij ), + d(Gij)),1=1 J i 1
In order to simplify the notation, define Eij = Ei
 + E3 and
T
ij	 (Ai-µj ) (µ -µj )
Let tr( • ) denote the trace of (•) and 1 • 1 _ det(•). With
a bit of matrix algebra it follows" that
Fij	 ^r{ tC Ik^Z)HEijH (Ik^Z)T^-1(Ik^Z)HoijH^kIZ)T}
-a
r
and
Gi3 =- 
2 lnI(Ik IZ)HEijH(Ik IZ) T I +_.4 1nj.(IkIZ)HEiH(II
+ 4 lnI(Ik I,Z)HE3H(Ik IZ) T I + 2 ln2.
We will now develop expressions for d(F ij ) and d(Gij),
According to Decell and,Quirein (1973)
d(Fj)	
-.12 tr{d((IkIZ)H)Qijl
Substituting the latter in the expression
d'(Fi^) _- 2 tr {d(( 'ik IZ)H)Qij }
and using the fact that tr(AB) = tr(?A), we have
2(1 I Z)
d(Fi3) = 2tr	 Tk L f(d(x)xT-xd(x)^r)xxT-xxT (d(x)xT-xd(x) T)I Qij
T2 tr{Qi^ (IkI Z) [ (d(x)xT-xd(x)T)xxT-xxT (d(x)xT-xd (x)T)]}(x x)
l 2 tr{xxTQi] (IkI Z ) (d(x)x"-xd(x) T) -Qij (Ik I Z)xxT(d(x)xTT (x x)
-xd (x) T)}
With a little matrix algebra (and some patience) it follows that
d(Fi^	
x
) _ T1 2 tr{ [ (xxTQi3 (Ik I Z) - Qi^ (IkI Z) T)T
( x)
- {,XTQi j (IkI Z) 	 Qi; (IkI Z)xxTT) ]xd(x)T}
`	 We now find an expression -for d(Gi ). First, recall
`	 (Kullback, 1968) that
d (lnl BE BTI) _ 2tr{.d (B)E BT (M BT)-1}
so that
f ,	 a
d(Gi1)	 -tr(d((IkI Z) H)E 3H(IkI Z ) T ((IkI Z)H FiJH(IkI Z) T) 1}
G
-2tr{d((IkIZ ) H) EiH(Ik iZ)T((IkIZ)HEiH(Iklz)T)
_l
+.Itr{d((IkIZ)H)E^H(IkIZ)T((IkIZ)HE3H(IkIZ)T) }.
r	 _
I
9.l
obviously, the summands in the expression for d(G ij ) differ
from the expression
d(Fij ) _ -2 tr{d((Ik
 IZ )H) Qij}
only by multiplicative constants and the matrix Qij . Hence, we
may use the final expression for d(Fij) to obtain the expression
for d(Gij ) by simply adjusting the multiplicative constants and
replacing Qij (in each summand in d(G ij )) with the expressions	 ^ ..
Ji j	 EijH(Ik IZ) T [ (Ik IZ)HE,jH(Ik IZ)T]-1
E H(I Z) T [(I Z)HE H(I Z) T ^ -1K
	
ij	 i	 kI	 k(	 k^
	
Lij	 EiH(Ik IZ) T [ (Ik IZ)HEjH(Ik IZ)T]-1
J
l	
At this point we will simplify the notar,ion. Let
Qij = (XX Qlj(Ik IZ)-Qij(IkI Z)sxT ) T-(xxTQi^(IkIZ)-Qi^(Ik)Z)xxT)
and let Jij , Kij , and L.. be similarly defined by substituting,
respectively, Jij ,Klj , and Lij for, Qij in the expression for Qij,
=.. 1 1	 M. It' follows that
3
d(Fi3j	 T 	 tr(Ql xd(x)T)	
^.Tx—x7 
'	
d (Gij)	 T 2 tr (Ji^ xd (x)T) -	
T _ 2 
tr (Ki j xd (x) T)
r (X X)	 (x x)
r	 1'	
T
_
	
	 T 2 tr (L ,xd (x) ) .
	 3(x x)
In order that x be extremal, it is sufficient that x satisfy
1G(x) _ 1 - P(Fij ij) (Q + 2J - K - h )x =
m	 1	 xT 2	 ij	 ^.i	 iJ	 i3i= 3	 ( x)
Of course, the function G(x) is the gradient of
T
P((Ik IZ)(I - 2^ )) with respect to x.
x x
With G(x), we use a steepest descent technique to const
Hl . The process is repeated for the construction of H2 since
given Hl , the problem of constructing H2 is identical to that
constructing H1 provided the populations are taken to be
N (Hipi ,111E ,Hl) i=- 1,
Test results are presented in the following tables fors
twelve channel, .0-1 flight line agricultural classes: soybez
corn, oats, red-clover, alfalfa, rye, bare soil, and two type
wheat. The Hill County data is sixteen channel data for five
Agricultural classes: winter wheat, fallow crop, barley, grz
and stubble.
C-1 FLIGHT LINE DATA
n=12,m 9,k=6, P=.024
Iteration H
^1
H
B2
11
B3
0 .327 .109 .134
1 .223 .060_ .034
2 .171 .062 .033
3 .135 .068 .032
4 .116 .058 .031
5	 - .1157 .055 .0309
I6
.1150 .054 .0303
i
RILL 'COUNTY DATA
n •16, m = 5, k = 6. p = .107
363	 264	 223
BIBLIOGRAPHY
Anderson, T.W. (1958). An Introduction to Multivariate Statistical
Analysis. New York: John Wiley and Sons, Inc.
•
Andress, H.C.(1972). Introduction to Mathematical Techniques in
Pattern Recognition. New York: Wiley-Interscience.
Dell, H.P. Jr. and Quirein, J.A. (March, 1973). "An Iterative
Approach to the Feature Selection Problem". IEEE Cat.
#CK0834- 2 1, pp. 3B-1--3B-12.
Decell, H.P. and Smiley, W. "Householder Transformations and
Optimal Linear Combinations". Report # 38 NAS-9-12777, Dept.
of Mathematics, Univ. of Houston, Texas.	 j
Householder, Alston S. (1958).- "Unitary Triangularization of a..
Non-Symmetric Matrix". J. Assoc. Comput. Mech., 339-342.
Kaileth, T. (Feb. 1967). "The Divergence and Bhattacharyya Dis
tance Measures in Signal Selection". IEEE Transaction on
Communications Theory, Vol. 15, N0. 1, pp-52-60.
Kullback Solomon (1968). Information Theory and Statistics.
New York: Dover Publications.
i
K
C
t
Iteration HB
1
R
2
HB
3
0 .872 .336 .299
1 .785 .310 .287
2 .525 .286 .232
3 .439 .273 .227
4 .576 .267 .226
5 .386 .265 . V4


Feature combinations and the Bhattacharyya criterion

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server