The qualitative behavior of the probability of misclassifying observations from two normally distributed populations as the classification regions are varied in a prescribed way is described in order to provide a preliminary generalization of the results obtained by Walton for the case of normally distributed observations with varying a priori probabilities

Walker, H. F.

English

NASA Technical Reports Server

NASA CR-
DEPARTMENT OF MATHEMATICS
UNIVERSITY OF HOUSTON HOUSTON, TEXAS
(NASA-CR-140385) SOME QUALITATIVE REMARKS N75-13576ON THE VARIATION OF THE PROBABILITY OF
ERROR (Houston Univ.) 18 p HC $3.25
CSCL 12A Unclas
G3/65 04901
SOME QUALITATIVE REMARKS ON
THE VARIATION OF THE PROBABILITY
OF ERROR.
BY HOMER F. WALKER.
AUGUST 1974. REPORT #34
PREPARED FOR
EARTH OBSERVATION DIVISION, JSC
UNDER
CONTRACT NAS-9-12777
3801 CULLEN BLVD.
HOUSTON, TEXAS 77004
https://ntrs.nasa.gov/search.jsp?R=19750005504 2020-03-23T00:22:14+00:00Z
Some Quat&tative Remwdz on the Vojiation o6
the Pitobab.i~ty o4 EAvt
Augw,6t 1974
H-omeA F. Wa,-heA
Vepavubnent o 6 Mathematic.6
UniveA~t/t o6 Hou6.ton
f-fou~ton, Texa6
Repolvut #34
Some Qualitative Remarks on the Variation of
the Probability of Error
by
Homer F. Walker
Department of Mathematics
University of Houston
1. Introduction.
In this note, we discuss the qualitative behavior of the probability of
misclassifying observations in 0 n from two normally distributed populations
as the classification regions are varied in a prescribed way. This discussion
is intended to provide a preliminary generalization of the results obtained
by Walton [4] for the case of normally distributed observations in R1 with
varying a priori probabilities. We hope to provide quantitative analogues of
these results in subsequent reports.
We assume that observations from two populations T 1 and T 2 are known
to have a priori probabilities a01 and t2 and normal density functions
(x)n/2(x i = 1,2,P01X) (2r)n/2 !i 2  -1/
01-
for x = (x1 ,...,xnT E n. We further assume that these observations are
classified, not by using the true Bayes optimal (maximum likelihood) classi-
fication scheme for TI and 2'. but by using the Bayes optimal classification
scheme defined by a priori probabilities al(t) and a 2 (t) and density functions
1 -1/2(x-p (t) Ei (t)- (x-1i (t)
i((2)n/21 i(t) 2 , i = 1,2,
where the functions Ca (t), i (t), and Zi(t) are continuously differentiable
functions of the parameter t in a neighborhood of t = 0. This is to say that
and observation x E a n is classified as coming from 7i if and only if
ai(t) p.(x,t) = m Cj(t) p (x,t). (We assume that Pl(x,t) 1 P2 (x,t) as a
function of x in a neighborhood of t = 0).
Under these assumptions, the probability of error in classifying an obser-
vation is a function of t in a neighborhood of t = 0, given by
Pe (t)= f 0 2P02 (x)dx + a 0lP 01 (x)dx,
R1(t) R2 (t)
where the regions R1 (t) and R2 (t) are defined as follows. Let
Fl (t)Pl(Xt)
F(x,t) = log
Q2(t)p2 (x,t)
3((t) W1/2
a ( t )  E2(t) 1/2 1 -1 1 T -1
= log 1/2 -2(x-pl(t))T I ( t )  (x-pl(t)) + 2(x-y2(t))T 2(t)-(x-p2
Then R1 (t) = {x E.n:F(x,t) 0} and R2 (t) = {x ERn:F(x,t) < 0}. (For a
more thorough disFussion of the probability of error, see Anderson [1].)
Our goal is to examine qualitatively the rate at which P (t) varies
as c varies in a neighborhood of zero. In our main result, the exact rate of
variance of p (t) is seen to depend on a number of factors. However, an
inequality of the form
IPe (t) - P e (0)1 K ltl
is obtained in every case. In other words, Pe(t) is always Hblder continuous
at t = 0. In the following, the exponent a is determined precisely in each
case. The constant K is merely asserted to exist; no estimate of its size is
given. Unfortunately, to implement such an inequality in practice, one must know
both the size of K and the range of t for which the inequality holds.
In the sequel, large constants are denoted generically by K,K', etc.
Distinguished constants are subscripted. The common boundary of R1 (t) and
R2 (t) is denoted by S(t).
2. The variation of P (t).
Our objective is to prove the following theorem.
Theorem If VF(x,0) 0 0 on S(0), then there exists a constant K such that
IPe (t) - Pe (0)l Kjtj for small t. If VF(x,0) vanishes somewhere on
S(O), then there exists a constant K such that IPe (t) - Pe (0)I : KjtIj-'+
for small t, where m is the number of non-zero eigenvalues (counting multi-
plicity) of 2(0)-1 - E1(0) - .
Remarks: If VF(x,O) vanishes anywhere, then the assumption pl(x,0) I P2 (x,O)
implies that m> 0. Thus Pe (t) is H3lder continuous at t = 0 with exponent
e
at least E. In the special case in which ai (0) = COi' 1(0) = 01i E(O) = EO
i = 1,2, exponents of H5lder continuity larger than those specified above can
be obtained. The determination of these exponents is not carried out here.
Before beginning the proof of the theorem, we establish several lemmas.
For a subset X c n, define
inf Ix-yl if X 0
yeX
d(x,X) =
Sif X = .
Let T = {x e I n:VF(x,0) = 01.
For non-negative p and q and positive r and s, define
L pr(t) = x E : xIx rX
Mpq,r,s(t) = x E ln:Ix : rltl - p  and d(x,T) sit q
Npqr t= x E n:xI 5 rIlt p  and d(x,T) slt
When there is no danger of ambiguity, we will omit the subcripts p,q,r, and
i.e., L ,r(t) = L(t), etc.P,
5Lemma i: Suppose that 0 < q < i and 0 < p < l-q.
Then there exists a constant K, independent of p,q,r, and s, such that,
if t is sufficiently small, then IVF(x,t)I ; Kslt q for all x E M(t).
Proof: Writing F(x,t) = x TA(t)x + B(t)x + C(t), one obtains VF(x,t) = 2A(t)x +
T 3 d d T
B(t) and VF(x,t) = 2 A(t)x + - B(t) . From this, it is seen that there
exist constants K' and K", independent of p,q,r, and s, such that
iVF(x,0)l 2 K'sltjq  and 2-VF(x,0) I K"(1 + r)t-P  for x E M(t) and t
small. It follows that there exists a constant K, independent of p,q,r,
and s, such that, for x E M(t),
IVF(x,t)I > K'slt q4 - K"(1 + r)ItI -' p 2 Kslt j4
whenever t is small.
Lemma 2: Suppose that 0 < q 5 2 and 0 p 1
If t andr) are sufficiently small, or if t is sufficiently small
s
and 0 - q, then, for It0 1 < Itl and x E M(t)fl S(t0 ), the solution
y(x,T) of the initial-value problem
dF (xT)
SdrI VF(y 12
y(x,t0) = x
exists and is continuously differentiable in x and T for ITI 5 Itl.
6Remarks: Note that, wherever y(x,T) exists, y(x,T) E S(T) and d-y(x,T)
is normal to S(T). It is seen in the proof that ly(x,T) - x1 : K (I+r)2  tl-2p-q
where the constant K is independent of p,q,r,s, and t for small t.
Proof: From Lemma I and the fact that TF(x,t) is quadratic in x, one sees
that, if t is sufficiently small, then there exists a constant K, independent
of p,q,r,s, and t, such that
at< K(1+r) t j-2p-q
IVF(x,T) I s
for i'TI < Itl and x , Mp,q,2r,s/2(t). Consequently,
ly(x,T) - xj 5 2K (+r) Itl l - 2 p - q  whenever T lies in the domain of existence
1+r (1+r1 (1+r) 2
of y(x,T). If is so small that 2K (l+r) < r and 2K (l) 2 then,
a s s 2
since 1-2p-q > q and 1-2p - q > -p, we have
(1) ly(x,T) - xj < r Itj-P and ly(x,T) - xl < _ ItI q
whenever x E Mp,q,2r,s/2(t) and T lies in the domain of existence of y(x,T).
I
If 0 - p < 5- q, then 1-2p-q > q and 1-2p-q > -p, and one easily verifies
that (1) again holds for small to Thus (1) holds under the hypotheses of
the lemma.
Suppose that the hypotheses of the lemma are satisfied (so that (1) holds)
and that there exists a t0, It 01 < !t , and x e Mp,q,r(t)n  S(t 0) such
that y(x,T) does not exist for all T, TI < Itl. Then one can find a tl,
!t11 ItI, for which y(x,t1) E pq2rs/2(t) [2]. But this contradictsp q,2r,s/2
(1), and the lemma is proved.
1 1
Lemma 3: Suppose that 0 < q 5 - and 0 1 p q-
2+r
If t and s are sufficiently small, or if t is sufficiently small and
s
0< p < 2- q, then
(i) R1 (t)AR 1 (0) c ITl<Itl S(T),
(ii) R1 (t)AR 1(0)lM qrs(t) c {y(x,T),EOn:ITl < It and x E S() M ,,2rs2(t)},p q,r<s psq,2r,/2
(iii) {y(x,T) E :JITI 5 Itl and x E S(O)( M s(t)} c t S(T)M ,3r, (t).
Pf6of:
(i) Suppose that t > 0 and x E R (t) - R1 (0). Set t0 = inf{T:x*R1 (T) - R1(0)}.
Clearly, x E S(t0). The other cases follow similarly.
(ii) If we [R (t)AR (0)]M pqs(t), then w E S(t 0) for some tO,
It0 - It!. If t and 1+r are small, or if t is small and 0 - p < - q,
then, by Lemma 2, y(W,T) exists for ITI ! Iti. In particular, x = y(w,0)
satisfies y(x,t0 ) = w. Now, by the remarks after Lemma 2,
Ix-wl < K (l+r) 2 Itil -2p -q  for a constant K independent of p,q,r,s,
l+r
and t. If s and t are sufficiently small, or if t is small and
0 < - q, then one sees that fx-wj < r It-P  and Ix-vI < It q
Consequently, x E S(0)Mpq s2r (t).
(iii) If x E S(0)IM pq s2rs (t), then, as in the proof of (ii), one uses an
inequality ly(x,T) - xl < K( + r ) 2  tl l - 2 p - q  to obtain y(x,r) E Mp s (t)
Spq,3r
for T It, if t and (l+r) are sufficiently small, or if t is
small and 0 p < - q.
8Proof of the theorem:
One has
Pe (t) - Pe (0) = a c02P2 (x)dx + o 01 Pl(x)dx - fS 0 2P2 (x)dx - a 0 1p1(x)dx
R1 (t) R2 (t) R1 (0) R2 (0)
f G(x)dx - S G(x)dx,
R1 (t)-R 1 (0) R1 (0)-R 1(t)
where G(x) = [CO 1P1 (X) -a02P2()]. Thus
IP (t) -P () < S IG(x) dx,
R(t) R1 (0)
and, for given p,q,r, and s, we obtain
(2) iPe (t) - Pe (0) < JIG + G5G + IGI .
[R1 (t)AR1(0)] L(t) [R1(t)AR (0)] M(t) [RI(t)AR1(0)]( N(t)
We consider the following cases:
(1) VF(x,O) never vanishes on S(O),
(2) VF(x,O) vanishes somewhere on S(O) and m > 1i,
(3) VF(x,O) vanishes somewhere on S(0) and m - 1.
Case 1: First, the following lemma is needed.
Lemma 4: Suppose VF(x,O) 0 0 on S(0) and 0 sp < .
If t and s are sufficiently small, then S(T)( N(t) = 0 for ITI < Iti
Proof: Suppose that the lemma is false. Then there exist sequences {s },
{t 1, {Tj , and {xj} with sj -0, t1  0, ' Ijl I tjl, and
xj E S(T ) N(t ). Note that VF(xj,T 1 )--4-0, since IVF(xj,T )I 5 Ksltj q .
Now F(x,t) = x A(t)x + B(t)x + C(t), where A(t) 2 (t) - (t))
is symmetric and A(t) and B(t) are continuously differentiable near t = 0.
Denote by al(A(O)) the null-space of A(O). Writing A(t) = A(O) + 01(t)
,
B(t) = B(0) + 02 (t), and x = Y + zj, where yj E (A(0)) and zj e (A(0)),
one obtains
j0 = "VF(xj j) = Jom{2A(O)y + O1 Tj)x j +B(Tj T}.
Since xj - r ItjI- p  and 0 <p < , O(Tj) x -- O. It follows that
Yj ---Oy E (A(0O)) , and
0 = 2A(0)y + B(0) = VF(y ,O).
Note that this equation implies that B(0)T E ~ (A(O)) . Furthermore, we have
0 = F(xj,T' ) = yjTA(O)y + xjT01(T)x + B(O)yj + 02(j)xj + C(Tj).
As before, x 01(T )xj 02 (j)x -- I0 since Ixj 5 r ItI -P  and
0 p< ~ . Consequently,
0 T A* O+ C(0) F(y*,).
0 y v A(0)y + B(O)y + C(O) - F(y ,0).
10
This contradicts the assumption that VF(x,O) never vanishes on S(0), and
the proof is complete.
Using Lemma 4, we obtain from (2) that
(3) IPe (t) - P e(0) I G I +  G I
j L(t) [R (t)AR (0)] M(t)
for 0 ' p < 2, if t and s are sufficiently small. If 0 < p < 2, then
the first integral on the right-hand side of (3) approaches zero faster than
any power of t as t approaches zero. In addition, we have the following
proposition.
1 1
Proposition 1: Suppose that 0 < q - 2 and 0 1 p - q.
l+r
If t and +r are sufficiently small, or if t is sufficiently small and
s
0 - p - q, then
IGI K t l -q
[R 1 (t) R1 (0)] n M(t)
where the constant K is independent of q.
l+r
Proof: it follows from Lemma 3 that if t and are sufficiently small,
or if C is sufficiently small and 0 _p < - q, then
a -F (y(x,T),T)
S GI IG(y (x,T))I d S(T)dT
[R (t)R (0) M (t) IT!51t S(O)M ,s(t) IVF(y(x,T) ,T) Ipqrs pq,2r,
Gs )F(x,) (T) dT)J
!Tj-<it S (T)M q,3r, F(XT)
where dS(T) is the element of surface area on S(T). (See Spivak [3] for a
discussion of integration on manifolds.) It is easily seen that
J IG(x) - F(x,T)IdS(T)
p,q,3r,(t)
is bounded for ITI - Itj uniformly for t near zero. Furthermore, for fixed
s, Lemma I implies that IVF(x,T)I ; K Itl q  for x E S(T)n M s(t) and
pq,3r
IT! t. Consequently,
f I -K It 1- q .
[R1 (t)AR1 (0) ] M pqrs(t)
It is easily verified that K is independent of q, and the proof is complete.
From Proposition 1 and the preceding remarks, one sees that if 0 < p < 1,q = 0,
and a is sufficiently small, then
IP (t) - P (0  5 K Itle e
for small r, and the theorem is proved in this case.
12
Case 2: If 0 < p, then, as before, the first integral on the right-hand
side of (2) approaches zero faster than any power of t as t approaches
1
zero. In addition, Proposition I remains valid. Thus, if 0 < q < ,
0 1+r
-0 < p q and - is sufficiently small, then the second integral on
the right-hand side of (2) is bounded by Kltll -q as t approaches zero,
where the constant K is independent of q. Of course, S(T)(IN(t) 0 0 for
all T, TI ItI, and we need the following proposition.
Proposition 2: There exists a constant K, independent of q, for which
J GI 5 K Itli
N(t)
where m is the number of non-zero eigenvalues (counting multiplicity) of
-1 -1E2 (0) - E1 (0).
Proof: We have F(x,t) = x A(t)x + B(t)x + C(t), where A(t) = (E2(t)-1-1(t) -
and VF(x,t) = 2A(t)x + B(t)T. Denoting the ball of radius p about the origin
in Dn by Bp, one sees that, if x0  is any solution of VF(x0 ,O) = 0, then
N(t) _ Ix0 + y + z:y EY( (A(O)), z Eft (A(O)) Ln Bst 1  I
Now the dimension of $(A(O)) is equal to the number of non-zero eigenvalues
1 -1 -1(counting multiplicity) of A(O) = 2(2 (0) - El (0)). Denoting this dimension
by m, we obtain (with a slight abuse of notation)
13
Sf G J (x + y + z)jdyl dz
N(t) (A(0)) Bsjt q YC(A(0))
SK tjmq
for an appropriate constant K, independent of q, and the proof is complete.
From the above discussion, one sees that the best rate of decay of the
right-hand side of (2) is obtained by choosing p and q such that
1 i
q <, 0 < p - q, and 1 - q = mq. Since m > 1, a compatible choice
i m - 1is m+ and 0 <p 2(m+ This yields the desired inequalitym+l 2 (ml)
m
m+l
JP (t) - P (0)I : K It[
Case 3: In this case, one sees that S(0) is an (n-l)-dimensional hyperplane
in R n  and that VF(x,O) = 0 if and only if x E S(O). By performing a
translation of co-ordinates followed by a unitary transformation on 1n if
necessary, we may assume that S(O) = {x = (O2,x2 ... ,xn n. Then
F(x,0) = xTAx, where A has a non-zero entry in the upper left-hand corner
and only zero entries elsewhere, i.e., F(x,O) = Xx2. We will use the sets
L (t), M (t), and N s(t) in the following with p = 0 andp,r p,q,r,s p,q,r,S
I
q = ~, and we set
Kc(t) = {x (xl,...,xnT E n:Ix11 e tc i x 
.
2x .. +x)
14
1+r
Proposition 3: If is sufficiently small and c is sufficiently large, then
S
R1 (t)AR1 (0) S N(t) U K (t) whenever t is small.
This proposition follows from the two lemmas below.
Lemma 5: If + is sufficiently small, then M(t)( [R1 (t)AR1 (0)] = 0 whenever
t is small.
Proof: In M(t), F(x,0) 2 X s2 t and IFt(x,T)I K(l+r) 2  for ITI < Itl,
where the constant K is independent of r,s, and t for small t. So, for
IF(x,T) 2 -IN s 2 - K(l+r) 2 11t
in M(t) whenever t is small. If +r < r , one sees that F(x,T) # 0
in M(t) for IT! < It!. Since R(t)AR1 (0) C U S(T), the lemma follows.
ITrl5It
Lemma 6: Suppose that r is given. If C is sufficiently large, then
L(t)( [R1 (t)AR1 (0)] C Kc(t) whenever t is small.
Proof: In L(t), F(x,O)= Xx2  and IFt(x,T)I < K IxI 2  for IT- i t, where
the constant K depends on r but is independent of t for small t. So,
for IT! - Itl, one has
2 tKtl(!F(x,T) > (X-Klt )xl Kt (i E2x)
in L(t). If x E L(t)-K (t), then
2
I  > n 2
c2t M i2x
15
and
IF(x,T) 2 [ - K It - ]x2.
if c is sufficiently large, then the right-hand side is positive for small
c. Consequently,
[L(t)-K (t)]WI[RL(t)AR 1 (0)] c [L(t)-K.(t)]j([ U S(r)] = 0,cc ITllItl
and the proof is complete.
1+rFrom Proposition 3, one sees that if S is sufficiently small and c
is sufficiently large, then
IPe t)- Pe (O) , S IGI + S IGI
Kc(t) N(t)
for small t. The two integrals on the right-hand side are easily seen to be
bounded by KVriTt, and the proof of the theorem is complete.
REFERENCES
1. T. W. Anderson, An Introduction to Multivariate Statistical Analysis,
John Wiley and Sons, Inc., New York, 1958.
2. F. John, Ordinary Differential Equations, New York University (Courant
Institute of Mathematical Sciences), New York, 1966.
3. M. Spivak, Calculus on Manifolds, W. A. Benjamin, Inc., Menlo Park,
California, 1965.
4. J. Walton, A"Mathematical Question Incident to the Signature Extension
Problem of Bayesian Classification, NASA-ASEE Faculty Fellow Report,
Summer, 1974.


Some qualitative remarks on the variation of the probability of error

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server