KATO, TAKESHI

Kato, Takeshi

Institutional Repositories DataBase (IRDB)

Higher order asymptotic sufficiency of posterior densities with respect to Kullback information

Osaka University Knowledge Archive

Osaka City University Repository

Kato, T.
Osaka J. Math.
32 (1995), 313-326
HIGHER ORDER ASYMPTOTIC SUFFICIENCY OF
POSTERIOR DENSITIES WITH RESPECT TO
KULLBACK INFORMATION
TAKESHI KATO
(Received Sept. 8, 1993, Revised Feb. 28, 1994)
1. Introduction
Let (X, B) be a measurable space and {P
θ
: 0eΘ}, Θc/?, a family of probability
measures on (X, B). We suppose that there exists a σ-finite measure μ on (X, B)
which dominates P
θ
 for all θeΘ. Then we put f(x9 θ) = (dPθ/dμ) (x). Let xί9
x2,'- be independently and identically distributed in accurdance with /(x, 00) for
some Θ0e&. We write £=(xl9 x2, ) and £π = (xι, ••-,*„). Let (XN9 BN)
be the countable Cartesian product of identical components (X, B\ and P$ the
independent product of a countable number of identical components P
θ
. Then the
definition of the higher order asmptotic sufficiency of posterior densities is as follows.
Definition. Let k^ 1 be a given integer, π(θ\£
n
) a posterior density function of
θ with respect to some prior density function p( ), and ψ(θ\T
n
) a probability density
function determined by some statistic T=T
n
(£
n
). For every ΘOE& and ε>0 if
lim
then T
n
 is called the k-th order asymptotically sufficient statistic in Bayesianm
sense under P 0^, where ί/(y) is some criterion.
In this definition, it is not required that φ(θ\ T
n
) should be formed out of the density
function of T
n
 parameterized by θ and the prior density function p of θ by Bayes'
theorem. As a matter of fact, it is too difficult to obtain the density function of T
n
in general cases. Therefore, if there exists a density function approximating
π(0|x
π
) and the observations ^^(xi, •••,*„) affect it through only a statistic
T
n
 = T
n
(&
n
), then we define T
n
 as the asymptotically sufficient statistic.
When rf(v) is total variation, it can be shown from the results in Ghosh et al
[2] or Johnson [3] that 7?> = (5II, Tn2(6n), Jnk(6n)) (k^2) is the fc-th order
asymptotically sufficient statistic in Bayesian sense, where θ
n
 = 6
n
(£
n
) is the m.l.e.
of 0 and
314 T. KATO
And the idea in the case where ί/(y) is Kullback information was suggested in
Takeuchi [6] first, and the detailed argument has been done in Kato [4] in the
second order case. According to them, the m.l.e. β
n
 is second order asymptotically
sufficient with respect to Kullback information. This fact should be noted, because
the second order can be attained by using ($„, Γ
n2(βn)) with respect to total
variation. An interesting problem arises from this difference, which is how high
order the statistic T™ attains when rf(v) is Kullback information. When it comes
to this point, detailed description cannot be seen in [6]. Then the purpose of
this paper is not only to solve it clearly but also to give the precise evaluation
of the probability of the exceptional set.
2. Assumptions and the main theorem
Let Θ be an open interval in R = ( — oo, +00) and ©the closure of Θ in
R = [ — oo, -f oo]. In addition, let k(^2) be a given integer and r^ 1 a real number,
then we impose the following assumptions.
(Aj) There exist functions f( β):X-+R9 0eΘ, such that the following hold.
(1) /(*, θ)=f(x, Θ) for all θeθ and all xeX.
(2) /(*, Θ) is continous with respect to 0eΘ for every xeX.
(3) For every θe Θ and tε®, provided
\f(x, θ)-f(x, ί)\dμ(x)>0.
(4) For every Θ e Θ and t e Θ, there exists a neighborhood Vt of t such that for all
neighborhoods Vc: Vt of /,
£9[|sup{log/(jc, τ): τe F} Π < +00.
(A2) For every 0eΘ, £β[|log/(x, 0)|r]< +00.
(A3) For each is xe X,f(x, Θ) is k+ 1 times continuously differentiable with respect
to Θ in Θ.
For simplicity, we use a notation
(A4) For every 0eΘ,
E
β
[](dJ/dθί)logf(x9 0)|r] < -f oo (/=!,-. Λ),
and there exist a neighborhood U
θ
 of Θ and a measurable function G(x, Θ)
HIGHER ORDER ASYMPTOTIC SUFFICIENCY 315
such that
9 t)\£G(x9 θ)
for all t e U
θ
 and all x e X,
and also E
e
[G(x, θ)r]< + oo.
(A5) For every 0eΘ, there exist a neighborhood Uθ of θ and a measurable
function H(x, θ) such that
, t)-(dk+ί/dθk+ί)\ogf(x, τ)\^\t-τ\H(x, θ)
for all ί, τ e (70 and all Λ: e X,
and also E
θ
[H(x, 0)]<+oo.
(A6) For every θeΘ,
(A7) The function
Ak + ι(θ) =
is continuous in Θ.
Let p( ) be a prior density function of θ with respect to Lebesgue measure.
(A8) I θ2 p(θ)dθ<+ao.
J -oo
Γ+ oo
(A9) For every θeΘ, the function g(x)= p(τ)|log/(Λ:, τ)\dτ satisfies Eθ[g(x)r~]
J — oo
< +00.
When observations *
w
 = (.Xι, ••-,*„) are given, let a /^-measurable function
be the m.l.e. defined by
For given 0
w
eΘ, we put
(2.1) X [ Γ " P^n + ξ/V") Π?= 1/fo, ^ + ξ/V^WΏ - ' (6. 6 Θ)
J — oo
where we assume that the range of ξ is restricted to the set
316 T. KATO
whenever we write /fo, 6
n
 + ξ/^/n) for Θ
n
ε&. When 0
w
eΘ, π(ξ\&
n
) is a centered
and scaled posterior density of Θ with respect to a prior density function p with
a transformation ξ = ^ /^(θ — β
n
). Formally we define 0/0 = 0 and 0(— oo) = 0. On
these assumptions, our final aim is to show the following theorem.
Theorem. Let Θ0e& such that p(00)>0 and assumptions (A t) through (A9) be
all satisfied for some r^l. Moreover if p is continuous in a neighborhood of 00,
then for every ε>0, there exists an integer «0 = «0(ε, &)^1 such that
%0{xeX»: ΓXπ
J — ao
where 7?} = (^  U^ /
<ε«1~r for all
has the form
R
n
(ξ)dξ
= 2)
χ
δn
(ξ)=l if\ξ\<δ^/n, = Q if\ξ\^δ^/n, and δ = δ(ε, k)>0 is sufficiently small.
We know from this theorem that the statistic 7<k) is (2Jt-l)-th order
asymptotically sufficient with respect to Kullback information in Bayesian sense
under P$0. And this assertion holds with probability one by the Borel Cantelli
Lemma if the assumptions are satisfied for some r>2.
REMARK 1. When λk+i(θn) is omitted from Rn(ξ), it can be shown that the bound
ε n~
(k
~
l)
 of the approximation to π^l^) by ^/k(ξ\Ί^) with respect to Kullback
information should be replaced by Mn~(k~l) for some M>0, and the probability
of the exceptional set is en1"1", which is the same as the theorem. On the other
hand, if we assume furthermore that λk+l(θ) is continuously differentiable in Θ,
then it can be proved that ψk(ξ\T^ approximates π(ξ\&n) up to order O(n~k) with
respect to Kullback information with probability approaching one. In this case,
HIGHER ORDER ASYMPTOTIC SUFFICIENCY 317
it is impossible to evaluate the probability of the exceptional set with order of n.
REMARK 2. The indicator function χ
δn
 used in the definition of ψk can be
replaced by χ
δn
(ξ) = 1 if | ξ | < δ n\ = 0 if | ξ \ ^  δna, 0 < α ^  1/2. The smaller α is, the
more simply the form of ψk becomes.
REMARK 3. It is a little bit difficult to make sure of the assumption (A9).
Let Θ0 = {0:ρ(0)>0} be an open interval in R and Θ0czΘ. Then (A9) holds
if for every θ e Θ
+00.
REMARK 4. Suppose that the density function/(.x,0) can be written in the form
f(x,θ) = exp{Σί= , Σίfi , uim(x)hjm(θ)}
and the following conditions are satisfied.
(i) Every ujm(x) is a measurable function on (X, B).
(ii) For every 0eΘ and j, m, E
θ
\_\ujm(x)\r']< +00.
(iii) For every j, m, E
θ
[ujm(x)~] is continuous in Θ as a function of θ
(iv) For every y, m, hjm(θ) is k + 2 times continuously differentiate in Θ
(v) For every θ e Θ and j\ ra, there exists a neighborhood U
θ
 of θ such that
where h( ^  denotes the /-th derivative of hjm
(vi) Σt=ιΣmiι£
β
["J mW]/I^)=0 for all θe®.
for a
Λ+
;, m,
J -
(viii) For every  p(θ)\hjm(θ)\dθ< + co.
oo
Then assumptions (A2), (A3), (A4), (A5), (A6), (A7) and (A9) hold.
The following are examples of Remark 4.
EXAMPLE 1. X=R2. Θ= (— 1,1). χi = (χil^ χί2) is normally distributed with
£J>y] = 0, E
θ
ίxfj] = l (/ =1,2) and E
e
\_xtl xί2] = θ. Namely
f(x, θ) = {2π(l - θ2)1'2} ~ *exp{ - (x2, - 2θXiixi2 + x?2)
As an prior distribution, we can take a transformed Beta distribution of the form
318 T. KATO
p(θ) = {21 -β-'/*(α,/9}(l + ΘΓ \\ -βf-1 (α,jβ > 1/2).
In particular, Remark 4 is often applicable to (/,!) curved exponential families.
EXAMPLE 2. X= Rm (m ^  1). xt = (xii9 ,jcίm). Θ = (0,oo).
f(xi9Θ) = (2πΓm/2 0-m(m+1)/2 exp{-X7=1(xlV-^)2/(202j)}.
This belongs to (2m, 1) curved exponential families, and when w = l, this is a well
known example of curved exponential families N(θ,Θ2). In this case, we can take
Gamma distribution, Gamma(α, /?) (α>2m, /?>0), where /? is the scale parameter,
or /^distribution, F(vl9 v2) (v1>2w, v2>4), as a prior distribution.
These two examples also satisfy the rest of the assumptions (A^, (A8), which means
that they are taken as concrete examples of the theorem.
3. Auxiliary results
The first thing, we quote the following Lemma 1 through 4 without their
proofs. Because they are completely the same as or quite similar to their
references. Roughly speaking, Lemma 4.2 in [1] is used to evaluate exceptional
sets in each lemma.
Lemma 1 (cf. [5; Lemma 4]). Let (AA) and (A2) be satisfied for some r^l.
Then for every ΘOE& and ε0>0, there exists an integer /ι1=«1(ε0)^l such that
Pl{xεXN:\β
n
-θ0\^ε0}>ε0n1-'for all n^n,.
For simplicity, we write Γ
nj{t) = n~ί^nj:=ί(dj/(dθj)logf(xiβ)\θ=t for 7=!, ~9k -hi.
Lemma 2 (cf. [2; Lemma 4.6]). Let (A3), (A4) hold for some r^l, and (A5)
be satisfied also for some r^l when j=k+l. Then for every Θ0e®, £0>Q and
j=l,- k+ 1, there exist δ
ί
=δ
ί
(ε0)>Q and an integer «1=«1(ε0)^l such that for
\t-90\<δ}:>ε0}<ε0n1~r for all n^n^
Lemma 3 (cf. [2; Lemma 4.7] and [3; Lemma 2.2]). Let (A3), (A4) and (A6)
be satisfied for some r^l. Then for every Θ0e&, ε0>Q, there exist δί>0 and an
integer «1=n1(ε0)^l such that for all
HIGHER ORDER ASYMPTOTIC SUFFICIENCY 319
>-ξ2ί(θ0)/4for \ξ\<δ^/n, \θn-θ0\<δ/2}<ε0ni-'for all
Lemma 4 (cf. [2; Lemma 4.2] and [3; Lemma 2.3]). Let (A
x
) and (A2) be satisfied
for some r^l. Then for every 00e®> δ2>0 and ε0>0, there exist a constant
α>0 and an integer n
ί
=ni (ε0)^l such that
>-nafor \ξ\^δ2^ι, \θn-θ0\<δ2/2}<ε0nί-r for all n
For simple description, when $BeΘ, we put
(3.1)
Lemma 5. Let (A^, (A2), (A3), (A4), (A6) and (A9) be satisfied for some
^l. Then for every Θ0e® and ε0>0, there exist δi>0 and an integer
ί
=nl(εQ9k)^l such that for all δe(Q9δi']
f
•Ίίl^V
Q
n
(ξ) log{Π?-ιl/frΛ + {/ΛA)//Mj]K
r/^ all
Proof. Let ε0>0 be given and δi>0 sufficiently small. And we suppose
that the following conditions hold.
0)|:|0 - Θ0\ < δ/2} < I(θ0)/2,
where a> 0 appears in the statement of Lemma 4 and δ e (0,5,]. Then we obtain
0>
,„ Γ
320 T. KATO
£-«3/2exp(-nfl){ιι -'£?.,*(*,)
+ « ' 'Σ?- . liog/(* A)l + (0o - W.2(0*W} (|0*- 00| ^  1
> -/t3'2exp(-«α){£
βo
ΓjfM] + l)+(£
βo
[|log/(x,00)l] + l) + (<5/2)2(3/(00)/4)}
> — ε0n~'*~ " as «-»• 4- oo.
Therefore it follows from Lemma 2, 4 and Lemma 4.2 in [1] that for every Θ0e®
and ε0>0, there exist <5j>0 and an integer w1=n,(ε0)^l such that
Q
n
(ξ)
>-nα for |{|
<ε0n
1
~
r
 for all /ι^«!.
Lemma 6. Let Θ0e& such that p(Θ0)>0 and (A3), (A4) and (A6) Z>e satisfied
for some r^ 1. Moreover if p is continuous in a neighborhood of θ
ϋ
, then for every
ε0>0, ίAere exists an integer «1=n1(ε0)^l such that
Γ
J -
Q
n
(ξ)dξ<δ3, \6n-θ0\<δ3}<ε0n1-f
for some <53>0 α«ί/ all n^nί9 where δ3>0 is independent of ε0.
Proof. Let <54>0 be sufficiently small. Then we assume that
|0-0ol < 2δ4 yields \p(θ)-p(θ0)\ <P(θ0)/2.
If \θ
n
 - βol < ^ 4, then p(0
n
 4- {/^ > p(fl0)/2 for \ξ\<δ4 since 1(0, 4- ξ/JΪ) - fl0| < |0W - Θ0\
+ \ξ\<2δ4. In addition, if
|:|θ ~ 00| < 2δ4} < I(θ0)/2,
HIGHER ORDER ASYMPTOTIC SUFFICIENCY 321
then when we put <53 = min{(54, p(θQ)δ4exp{ — 3I(θQ)δl/4}}, we obtain
Λ+oo
QtfW
J -oo
^ ί p(θ
n
 + ξ/jφxp{Γ
n2(θ*)
J\ξ\<d<
exp{-3/(00)|2/4K
dξ
Consequently, by Lemma 2, there exists an integer n1=nί (ε0)^l such that
\:\θ-θ0\ <2δ4} ^
<ε0«
1
~
r
 for all w^w^
Now that we have finished proving lemmas as preparation, we will begin the
proof of the main theorem.
4. Proof of the main theorem
When 0
π
eΘ, by using (3.1), (2.1) can be rewritten as
and we obtain
(4.1) O
= Γ
 X
π(ξ\χn)\og{Q
n
(ξ)/R
n
(ξ)}dξ + log{ Γ " RJίQdξ/ Γ °° QJίQdξ}
J — oo J — oo J — oo
^ Γ "e-jioiogie./^^^^ f
J — ao J — c
322 T. KATO
_+J Q
n
(ξ)log{Q
n
(ξ)/R
a
(ξ)}dξ
ί
 K<t
'
J
"
lίl ><
ί -':
ί
ISi = «"v "
+ 2- Ί Qn(ξ){l°&{Rn(ξ)/Qn(ξ)}}2™p{v \og{R
n
(ξ)/Q
n
(ξ)}}dξ
v I κ i ^ f /H|s| ^<?v"[
J I El
+ wίj-aίί)}^/ eB(^1-00
-f 2-1 ί
Jl<
+ f
•Ίίl^V
where ^R
π
(ί) is defined in (2.2). We consider the evaluation of each term of the
right hand side of (4.1). Let ε>0 be given. We suppose that there exists (53>0
such that
HIGHER ORDER ASYMPTOTIC SUFFICIENCY 323
Let ε1 = ε1(ε, &)>0 and δ = δ(ε, k)>0 be sufficiently small. Then we assume that
<5/2><53 and
\Θ-θ0\<3δ/2 yields \P(θ)-p(Θ0)\<p(θ0),
\Θ-Θ0\< δ/2 yields \λk+1(θ)-λk+l(θ0)\<ει.
In addition, we suppose that
\θ
n
-Θ0\<δ/2,
sup{|Γ
π2(0) + /(Θ0)|:|θ - 00I < δ/2} > 7(00)/2,
sup{\T
ak+1(θ)-λk+1(Θ0)W-θ0\<3δ/2}<ει,
log{Π?= iL/fe 9, + ξ/^)/f(Xi, £,)]} ίί -/(0oK2/4 for|{| < δjn,
if
•ΊίlέίV
where α>0 is that of Lemma 4. The probability of the complements of these
events will be evaluated in the final step of the proof.
(4.2) 2-1|
<(3/4)/(θ0)exp( - «α) Γ °°ξ 2
J -co
Λ+
p(-nα){
J -
<(3/2)/(θ0)n3/2exp(-  >^p(y)rfy +(|θβ| +<5/2)2}
where we have used |0J<|00| + (5/2. Next, we will show that
(4.3) 2-4 Q
n
(ξ){^B{Rn(ξ)/Qn(ξ)}}2
J
In fact,
324 T. KATO
\log{R
n
(ξ)/Q
n
(ξ)}\ = \λk+ rfj
Therefore the left hand side of (4.3) is dominated by
4p(00)ε2/Γ<*- " ί ξ2<k+ »exp{ -I(Θ0)ξ2/4}
J
which establishes (4.3).
(4-4) ί
•Ί
^ ί•
<ε1n~
(
*~
1)
 (n^ + oo).
It follows from (4.1) through (4.4) that
<ε-<*-υ
Consequently, by Lemma 1, 2, 3, 4, 5 and 6, for every ε>0, there exist <5 = <5(ε,
and an integer «0 = Λ0(ε, k)^\ such that
HIGHER ORDER ASYMPTOTIC SUFFICIENCY 325
\6
n
-00\<δ/2}
: \6.-θ0\*δβ}
: sup{|ΓB2(0) + I(Θ0)\:\0 - Θ0\ < δ/2} £ I(Θ0)/2]
: sup{|k
 + ,(0) - λk + , (00)\:\θ - 00\ < 3.5/2} ^  ε, }
>-I(θ0)ξ2/4 for |fl<
: log{Π?-ι I/(*ι, θ.
>-na for Kl^
r:
<εnl~r for all «
This completes the proof of the theorem.
ACKNOWLEDGMENT. Γm grateful to Professor Takeru Suzuki for the
careful examination of this study. He pointed out the improvement of the
order of the approximation. I also would like to thank the referee for many
valuble comments.
References
[1] D.M. Chibisov: On the normal approximation for a certain class of statictics, Proc. Sixth
berkeley Symp. Math. Statist. Prob. 1, Univ. of California Press, 1972, 153-174.
[2] J.K.Ghosh, B.K.Shinha and S.NJoshi: Expansions for posterior probability and integrated
326 T. KATO
Bayes risk, Statistical Decision Theory and Related Topics III, Vol. 1. Academic Press, 1982,
403-456.
[3] R.A. Johnson: Asymptotic expansions associated with posterior distributions, Ann. Math.
Statist. 41, No. 3 (1970), 851-864.
[4] T. Kato: Second order asymptotic sufficiency of posterior densities with respect to Kullback
information. (1994). (to appear in Statist. Decisions. 12)
[5] R. Michel and J. Pfanzagl: The accuracy of the normal approximation for minimum contrast
estimates, Z. Wahrsch. Verw. Gebiete. 18 (1971), 73-84.
[6] K. Takeuchi: Hi-Beyes no tachiba kara mita Bayes tόkeigaku (Bayesian statistics from a
non-Bayesian point of view). Bayes tόkeigaku to sono όyό (Bayesian statistics and its applications),
Chapter 5, Univ. of Tokyo Press, (1989), 119-136. (in Japanese)
Waseda University
Department of Mathematics, School of Science and Engineering,
Waseda University
Ohkubo, Shinjuku-ku, Tokyo 169, Japan


Takeshi Kato

Crossref

Statistics & Risk Modeling

SECOND ORDER ASYMPTOTIC SUFFICIENCY OF POSTERIOR DENSITIES WITH RESPECT TO KULLBACK INFORMATION

http://dlisv03.media.osaka-cu.ac.jp/infolib/user_contents/sugaku/111F0000002-03202-5.pdf

Higher order asymptotic sufficiency of posterior densities with respect to Kullback information

Abstract

Similar works

Full text

Available Versions

Institutional Repositories DataBase (IRDB)

Osaka University Knowledge Archive

Osaka City University Repository

Crossref