Methods for identifying finite mixtures and sets of measures are given. Theorems and corollaries are provided for characterizing sufficient statistics for analysis

Peters, C.

English

NASA Technical Reports Server

General Disclaimer 
One or more of the Following Statements may affect this Document 
 
 This document has been reproduced from the best copy furnished by the 
organizational source. It is being released in the interest of making available as 
much information as possible. 
 
 This document may contain data, which exceeds the sheet parameters. It was 
furnished in this condition by the organizational source and is the best copy 
available. 
 
 This document may contain tone-on-tone or color graphs, charts and/or pictures, 
which have been reproduced in black and white. 
 
 This document is paginated as submitted by the original source. 
 
 Portions of this document are not fully legible due to the historical nature of some 
of the material. However, it is the best reproduction available from the original 
submission. 
 
 
 
 
 
 
 
Produced by the NASA Center for Aerospace Information (CASI) 
https://ntrs.nasa.gov/search.jsp?R=19770017943 2020-03-22T09:59:46+00:00Z
PREPARED FCR
EARTH OBSERVATION DIVISI^N, J5C
UNDER
CONTRACT NAS-9-15:.G
^.0 OF yo
n G^ DEPARTMENT OF MATHEMATICS
7^ UNIVERSITY OF HOUSTON	 HOUSTON. TEXAS
^L
(NASA-CR-151338) SUFFICIENT STATISTICS FOR
	 977-24887
HIXTUFES OF [MEASURES IN A HO M OGENEOUS FXMILY
(Houston Univ.)
	 18 p HC A02/MF AC 1 CSCL 12A
Unclas
G3/65 26140
SUFFICIENT STATISTICS
FFR	 IXTUREF CF h'EASUk--
IN A HFIMOGFNC-OUS FAMILY
FY CHARLES PFTEk-.
REPCRT #E4 V4RCH 1'477 m f
i
,,223Z4^ s^G
AY 1;;77 ^'^
NA
RECEIVED
NP ST' FACILIT
	
^.
Ur BRANCH 4
.j
^ a5^
HOUSTON. TEXAS 77004
-^--	 t 2
Sufficient Statistics for Mixtures
of Measur-s in a Homogeneous Family
I
iE
By
Charles Peters
department of Mathematics
University of Houston
March, 1977
Report 64
Sufficient Statistics for Mixtures
of Measures in a Homogeneous Family
by
Charles Peters
Department of Mathematics
University of Houston
1. Introduction:
Let (X,12) and (Y, 8) be measureable spaces and let T X Y be
surjective and measureable. Let 	 be a set of finite positive measures on
(X,Q). For each u E	 there corresponds a measure pT 1 on (Y,$) defined
for F 4 by
UT-1 (F) = U(T 1(F)).
If f is a u-integrable real valued function on X, then as a consequence of
the Radon Nikodym Theorem, there is a VT-l- integrable function e 
1
(f) on Y
satisfying
j e
U 
(f)dUT 1 = 1 fdU
F	 T-1(F)
for each F e t. Clearly eP (f) is defined only up to sets in Y of PT -1
measure 0 and f = g a.e. (u) implies eu (€) = 
e 1
(g) a.e. (UT l ). The
linear operator eu defined as above maps the space t l (X,Q ,U) to the space
J 1 (Y, -g,UT-1) and is called the conditional expectation operator. Its value
93
L.
e (f) at f c ;t
j 
(X, Q,p) is called the conditional expectation of f given
T.
The conditional probability of an event E E a is defined as
PU (E) = 
e11
(XE)
where XE is the indicator function of E. The conditional probability
functions satisfy
(a)
P11 
;a.-14(Y, °8 , PT-1
where lm $ ,uT l) is the set of all real valued ' -measureable functions
on Y, with equality defined as equality a.e. (uT 1}.
(b) For each F c k?,E c Q ,
V(E n T-1 (F) = l PP (E)dPT l
F
(c) 0 <_ Pu (E) < 1 for each E t Q and Pu M = 1.
(d) If {En}n=1 is a disjoint sequence of events in CZ,
u( COP	
n 1 E
n) = nil P1 (En) a.e. (PT-I ) .
It should be noted that P u satisfies property (c) even when u is not a
probability measure.
The transformation T is called a sufficient statistic form if
for each E c(2 there is at7 -measureable function P(E) on Y such that
for each u e-)11, Pu (E) - P(E) a.e., (NT 1 ). The set_Y4 is dominated by
a measure A -(perhaps not in 711) if for each u cll?, p is absolutely
3.
continuous with respect to A,( written U « a.) Y is homogeneous if it is
dominated by each of its members. A measure X is equivalent to 'M if
l dominates '41 and U(E) _ 0 for each P E"M implies a(E) = Q.
The notation and terminology used in this paper are taken from (Halmos
and Savage; 1949), as are the following three theorems. The notation
)T 1(	 means that there is an element of the equivalence class ^ of
dX
Radon-Nikodym derivatives which is T-1(9) measureable.
Theorem 1: If I-& is dominated, then a statistic T is sufficient for ILL if
and only if there exists a measure a equivalent to VIL such that for each
ji E7n, d1 (f )T 1(-R).
Theorem 2: If -17t is dominated, then a statistic T is sufficient for Ylt if
and only if T is sufficient for each pair (p,v} of elements of MI .
Theorem 3: If In is homogeneous, then a statistic T is sufficient for -14t if
and only if dv (E)T 1 ( ,& for each P, v E171. 
2. Homogeneous Families:
Henceforth, we will assume that )n is homogeneous. Let Coj) denote the
cone generated by'j)L , excluding the zero measure. That is, C" is the set of
all finite linear combinations, with strictly positive coefficients, -of elements
of -M . Elements of COO) are termed mixtures of elements of 'm . Clearly,
001,0 is also homogeneous; hence, the spaces j (Y ,S IjT 1 ) are all the same
for p e C(ft and may be denoted simply bye' . For )A e C('j/p, Pu maps Gt to
and it is clear from the definition of a sufficient statistic that T is
sufficient for a subset JZ of C(* if and only if the conditional probability
r
functions P^ for U c 7 are all equal.
Lemma 4: If M is dominated, c C( , and T is sufficient for , then
T is sufficient for	 .
Proof;	 Let X be that measure equivalent to Ik hose existence is assured
by Theorem 1: If P € COS, then p can be written
k
s iZl d
i vi
with di > 0 ) vi e llt for i	 1, ... ,k. Hence,
k
dX	 it ^i dXi	
{E) T-1().
Thus T is s ­ 'icient for C" and hence is sufficient for-2 .
In order to characterize sufficient statistics for A c C(M, it suffices,
by Theorem 2, to consider a pair
uI	 iEI Si ui
and
ui	 j EJ sj uj
in ^ 1 , where I and J are finite sets; a  > 0 for k E TO; and the
measures {Ui } iEI are distinct members of PI, as are the measures' {Uj}jEJ'
The set C( ) of all finite mixtures of elements of 7/t is said to be
identifiable (Teicher,1960, 1961; Yakowitz 1969) if each element of C(M
can be expressed in only one way as a linear combination with positive
coefficients of elements of `117 1 except for the order of the summands. Equivalently,
COO) is identifiable if the set 1^ is linearly independent over the real numbers.
I
The concept of identifiability is very important in establishing the
uniqueness and consistency of various estimators of the so called mixin g
parameters { :ieI} in a mixture u I (Yakowitz, 1959).
Given a mixture u I _ in COW we have for each E e O , y e 1g,
PuI (E) dull-1 	
V11 (E n T-1 (F ) }
= E Oiui ( E n T-1 (F } }
iEI
E $i Pu { E } i T-1
ieI	 F i
-1
iEI i I P^ (E) ^ i T- 1 du 1 T 1.F i
	
dl I T
{	 Let Il ,... , I r be the equivalence classes in I modulo the relation i BE k if
and only if P  = PP ; that is, if and only if T is sufficient for the
i	 k
pair { ui , uk} . Then we have
duT1
i€I. Bi J 'u (E)	
i 
-1 du lT 1
F i	 duIT
r	 d T 1
	
_	
ui	
-1
F 'j, iEI i Bi d T 1 P11 (E ) dµI T •
uI	 ,-
where Pu (E) is the common value of the Pu (E) for i e I i. Thus,I	 i
r dpIQT 1
	
Puf 
s 
	 du T 
1 PuiiI
b.
•	 uI^ iEi
k
 ^i ui
Whenever the conditional probability function P
III
 
of a mixture It  is
written in this fashion with I 1 ,...,Ir being equivalence classes modulo the
relation =, we will say that 
PuI 
is written in normal form.
Definition 5. The set C(-91) is conditionally identifiable with respect to
the statistic T if for each pair (ui,uJ) in COO
, 
whenever Pu = Pu and
I	 J
P	 P	 are expressed in normal form
u1	 uJ	
r du T-1
PuI = k.^ l
	
I___ :^__ PuI
du TI
du T-
1
_ S	 ,s
_	 k
PuJ k=1 du T 1 PPik
J
then r = s and for each i - 1,...,r there exists exactly one k = 1,...,r
dulT`1	 dPj T-1
such that	 Q_	 =	 k	 and P	 P	 . The set C(77) is
dui T 1	
duJ 
T
-1
	
111 9,
	
uJk
marginally identifiable with respect to T if the set (uT l lu C" is
linearly independent over the real numbers.
Theorem 6: If CCOO is both marginally identifiable and conditionally
identifiable with respect to a statistic T, then C(?V is identifiable.
Proof: Suppose pi = E Siu i = E sou	 uJ	 where the measures in each
iEI	 jEJ
sum are distinct members of Ili	 Then, expressed in normal form,
7.
i
i
r duI T-1
	
r dui T-1
=	
g-PuI V, 
dpi 
T-1 FuI $ iEI dp 
J T
_ 1	 P J Phi,
^
and we may assume without loss of generality that
duI 
T-1	
dui 
T-1
L	 =	 i
diAIT 1	 duiTll
and	 PuI = P11  for X
Since pIT-1 = uiT-1 , it follows that p  
T-1  
= ui 7-1 . For i,k c It,
	
k	 1L
uix 1 + ilk T 1 , for otherwise, since Pu i = Puk, we would have U  = uk'
contradicting the assumption that (Pi : ieI} are distinct. Similarly, the
Wi T 1 for j c J 
t 
are all distinct. Since C(lit) is'marginally identifiable.
I  and J 
i 
have the same number of elements and for each i c It there is
a unique j (i) c J9 such that Si = aj 1i) and u IT 1 = uj (i) T-1 . Since
P.= F
	
, it follows that u i = u j {i}	 ^for each i c I	 Therefore,ui	
uj ( } 
there is one to one map j from I onto J such that 6 J (i) _ a  and
uj(i) = Iii for each i E I. Hence, C(-M) is identifiable, and the proof
is complete.
For conditionally identifiable sets of measures, the following theorem
and its corollary provide some characterizations of sufficient statistics.
Theorem 7:	 If -/-^j is homogeneous, CCM) is conditionally identifiable
with respect to a statistic T. and li l ,ui are in C(-W , then T is
sufficient for the pair pi , Pi 
if and only if there exist partitions
I = 1 1 u ... uI r and J = J 1 u ... 0	 such that for each $ = 1,...,r'
8,
dU	 du
(a) d{ 
iCIL 
i ui)/ d{ pit j J)	 dud
	dpj
and
(b) T is sufficient for the set Ni - {uk : k e IiuJi}.
Proof: First suppose such partitions exist. By (b) T is sufficient for the
set N1 and hence, by lemma 4, it is sufficient for the pair {ui , J }. It
1 l
followa from (a) and Theorem 3 that T is sufficient for the pair
S •zppose that T is sufficient for the pair {I•uJ}, Then, expressed in
normal form,
	
r dpi T -1	 r	 duJjtT 1
LE1 
dP, 
T-1 PIA - LI 
d^ 
T-1 PuJ^,
J
and we may assume without loss of generality that
du : 1	 dp T-1
Ii-1
	
and P u	 P	 for each L.
dull	 d1a JT 	 IR	 Ji
The condition PuI - Pu	 is equivalent to (b). By Theorem 3, there exists a
X	 J^
d11	 dt:T 1
representative f e du
J	 J
which is T-1 05) measureable. If g e du
Z
T_1,
1
then g•T is T (-e) measureable and for each F e e,
I R •T auJ - I g do T 1- uIT 1 (F)
	
T-1(F)	 F
•	
- I fdp
T-1 IF)
L. -- --L
	
i
9.
It follows that vT = f a.e.(Pi). Thus,
du T-1 	 au 1	 du
T= {g,T I g E	 } C	 I
du JT-1
	
du JTl	 dui
Since T is also sufficient for the pair {u ,uJt
	
a similar argumentIR 
gives
dPI T 1	 dui
d ifi-1 
f T c 
du Ji
^J i	 f.
dui 
T-1	 du T71
for each X. Since	
^
I 
-1 
for each f,, it follows that (a)
dpi
	duJT
holds for each h and the proof is complete.
Corollary 8: If 
-M is htvvgeneous and C(-?t) is conditionally identifiable
with respect to a statistic T, then T is sufficient for a pair {ul,uJI
in C(KI) if and only if there exist subsets I 1c I and J1 c J such that:
(a) dull	 = duI
duJ	dui
1
and
(b) T is sufficient for N = {uk : k r I 1 u J1}.
Proof: That T	 sufficient implies the existence of I 1 and J 1	satisfying
•	 (a) and (b) is icsediate from Theorem 7. Conversely if 11 and J 1	satisfy
10.
(a) and (b), then T is sufficient for PI , uJ by (b) and hence, by (a),
1	 1
T is sufficient for ul,uJ.
Given a pair of mixtures u l , 1►J in :(if), we will call their
du
likelihood ratio dtj indecomposable if I 1 C-1 J l cJ and
J
dui	 du
-	 - , imply I1 a I and J1 J. It is clear from Theorem 7 that
J 1	 i
if C(-ht) is conditionally identifiable with respect to T and a pair of
mixtures ul ,
 'Pi in C(M) have an indecomposable likelihood ratio, then
T is sufficient for {ul , ui } if and only if it is sufficient for
{yk : k t I u1}. Also, it is not difficult to see that for each pair
u , u in C(i?1) there exist nonempty subsets I1 I and J1 c J such
I J
that
dull	 Jul
f	 dui	 dui
1
duI
and she likelihood ratio d^ l is indecomposable. If ul and uJ represent
iI
the probability laws for two alternative hypothcsc:,. then there would be two
advantages in being able to identify subsets I 1 and J1 satisfying these
two criteria. First,the maximu- likelihood decision procedure would be simplified,
and second, the search for a statistic sufficient for deciding between the two
hypotheses and having the property that CM) is conditionally identifiable
could be restricted to those statistics sufficient for {uk : 
I l u it}.
3. Sufficient Linear Statistics for Mixtures of liormals.
If A is a subring of the ring J- introduced in section 2, then with the
11.
usual definition of addition and multiplication by elements of the set
of all functions	 Cx ► . is a module over	 Thus, it is natural to
considerl-independence of a set 4 of such functions. To be precise, J is
-independent if whenever 1
	m is a finite set of distinct elements of
Jand yl ,...,y are elements of R such that
m
yl^l(E)  +...+ ym^m( g ) = 0 for each E c 1l ,
then y,= ... = ym = 0. If	 is a subring of J which contains all Lae
bound
	
	 don-Nikodym derivatives dW? for, v c C{} then it is Blear
dyT
that 7\-independence of the set (Pu	 c ?rj} implies that C(?1t) is
conditionally identifiable with rospect to T.
For the remainder of this section we will assume that X is n , Y is { k
(k t;n) and T	 X -+ Y is linear and full rank. &t and "/. are respectively,
the Borel fields on JK ° and lk. 	 We also assume that each k f= 'j'jt is described
by a normal density function f with mean mµ and covariance	 That is,
for each E c a
t^(E } = E fu da a
where a n is Lebesgue measure on	 R
By a suitable choice of the coordinate system, we may represent the densities
f as joint density functions f u(y,z) on P k x Ru	 while representing T
as the projection T(y,z) • y. Then the marginal densities
	
90
	 n-k U
(Y) = 1 	f (y,z)dz
^
1are normal with means Tm and covariance matrices V0T
	
V1 
	 (Anderson, 1958).
t -
	
1
WMCROCopr-*ESOLUTION TEST CHART
N&Tl6 W BUREAU OF SIAND RDS-M1 6-
-.
(2)	 SS211S1 - SY1 (TaPT 1 )-1 VIPS1.
where S is the linear operator S(y,z) = z. The conditional probabilities
P
	
} are represented by
Pu (E IY) = j hu(zly)dz
Sy(E)
where Sy (E) _ {z a IR	 (Y,z) E E }.
Theorem 9: If `j'jZ is a family of Borel measures on	 n given by n-variate
normal density functions and T 	 n ► t k is linear of rank k, then
C(M is conditionally identifiable with respect to T.
Proof: It can readily be verified that conditional identifiability of C(M)
is not affected by the change of variables just described. If u  and P 
du T-1
are in CCM), then the Radon-Nikodym derivative .^-yl is represented by a
duJT
function of- the form
8I (Y)
0J (Y)	 i I i$Pi (Y) I jCi Sjguj(Y).
I
F
--	 -
13.
i.e., a ratio of mixtures of k-variate normal density functions, which
Is continuous. Hence, by the remarks in the first paragraph of this section,
.	 it suffices to show that the set {P 	^ e'jn of conditional density
functions is =independent, where	 is the subring of j consisting of those
elements of j which have a continuous representative. To this end, let
P ,...,F	 be distinct and let Y i ,...,Yr be cont._;uous real valued
^1	^r
functions on A K such that for each E ca,
Y1 (y)PU
 ( El y) +...+ Yr(Y)p
P
 (E 1 Y) - 0
1	 r
for almost .-`1 y. In particular, choosing for E sets of the farm fKK 
X 
K,
where K i^ a borel set in n-k , we have
Yl (y) f h11 (zly)dz +...+ Yr (y) f h  (zl y)dz = 0
K 1	 K r
for almost all y. For each K, 1 h (zly)dz is a continuous function of
K pi
y. Hence,
K(Y I (y)h 11 (Zl y) +...+ Y r (y )hP (zlyl)dz = 0
for each y E 
rK k . It follows that
Yl(y)hu 
1 
( z ly) +...+ Yr (y ) hu (zlY) = 0
	
•	 r
for each y E 1[- k , z E (kn-k . Let F be the set of y C R where two or
more of the conditional & nstty functions h  (zly) are equal as functions
i
------ 
14.
of z. It is easily seen from (1) and (2) that the Lebesque measure of F is
zero. For y j F , (h (-1y),...,h { -1 y)) is d set of distinct normal
ui	
^r
density functions _ of z. Hence, (Yakowitz and Spragins; 1968), they are
linearly independent over the real numbers. Therefore, for y t F
	Y1 (y) _ ... Yr(y) _ 0. That is, Y _	 = Y  = p as elements of .
Thus, C(f?'t} is conditionally identifiable.
If uI a i€I0i P i is in C(fitt), then PI has a density function
	
f
U I
	1
= E 8. f
i6I 
	
Ui
which is a mixture of normal density functions. The following theorem is an
immediate consequence of Theorems 7 and 9.
- heorem 14:	 Given the assumptions of Theorem 9, the statistic T is
sufficient for a pair {pl, Pi } in Qjtj) if and only if there exist partitions
I = I 1 u ... ulr and J = J1 u...uJr such that for each i = 1, ... ,r,
(a) E B f (x) /	 F. ^. f {x)
ieI Q i Pi	 j ci k
	
uj
n
f W E	 f (x) for each x r f
	
ieI i ui	 j£J J 11 
(x
and
(b) T is sufficient for the family (fIl
	
k e I  t!J Q } of normal
k
density functions.
There is set of purely algebraic conditions which are equivalent to (b);
1
15.
it T1 (T 0 T1)-1
k
1
	
are all independent of k c r^ uJR (peters, Redner, and Decell; 1976).
—	
_	
--
16.
REFERENCES
1. Anderson, T.W. (1958). An Introduction to Multivariate Statistical
Analysis. John Wiley and Sons. New York.
2. Bahadur, R.R. (1954)• Sufficiency and statistical decision functions.
Ann. Math. Statist. 25, 423-463.
3. Halmos, P.R. and Savage L.J. (1949). Application of the Radon-Nikodym
theorem to the theory of sufficient statistics. Ann. Math. Statist. 20.
225-241.
4. Peters, B.C., Redner R., and Decell, H.P. (1976). Characterizations of
linear sufficient statistics. Tech. Report no. 59,	 Department of
Mathematics , University of Houston.
`	 5. Teicher, H. (1961). Identifiability of mixtures. Ann. Math. Statist.
32, 244-248.
6. Teicher, H. (1963). Identifiability of finite mixtures. Ann. Math.
Statist. 34, 1265-1269.
7. Yakowitz, S. and Spragins, J. (1968). On the identifiability of finite
mixtures. Ann. Math. Statist. 39, 209-214.
8. Yakowitz, S. (1969). A consistent estimator for the identification of
finite mixtures. Ann. Math. Statist. 41), 1728-1735.


Sufficient statistics for mixtures of measures in a homogeneous family

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server