A bound is given for the Bayes risk of an estimator under truncated squared error loss. The bound derives from an information inequality for the risk under this loss. It is then used to provide new proofs for some classical results of asymptotic theory

Brown, Lawrence D

English

ScholarlyCommons@Penn

University of PennsylvaniaScholarlyCommonsStatistics Papers Wharton Faculty Research1993An Information Inequality for the Bayes RiskUnder Truncated Squared Error LossLawrence D. BrownUniversity of PennsylvaniaFollow this and additional works at: http://repository.upenn.edu/statistics_papersPart of the Statistics and Probability CommonsThis paper is posted at ScholarlyCommons. http://repository.upenn.edu/statistics_papers/105For more information, please contact repository@pobox.upenn.edu.Recommended CitationBrown, L. D. (1993). An Information Inequality for the Bayes Risk Under Truncated Squared Error Loss. Proceedings of Penn StateConference on Multivariate Analysis, Retrieved from http://repository.upenn.edu/statistics_papers/105An Information Inequality for the Bayes Risk Under Truncated SquaredError LossAbstractA bound is given for the Bayes risk of an estimator under truncated squared error loss. The bound derivesfrom an information inequality for the risk under this loss. It is then used to provide new proofs for someclassical results of asymptotic theory.KeywordsInformation inequality, cramer-rao inequality, truncated squared error efficiency, local asymptotic minimaxity,superefficiencyDisciplinesStatistics and ProbabilityThis conference paper is available at ScholarlyCommons: http://repository.upenn.edu/statistics_papers/105An Information Inequality for the Bayes Risk under Truncated Squared Error Loss Lawrence D. Brown• Department of Mathematics Cornell University Ithaca, NY 14853 April 30, 1992 Abstract A bound is given {or the Bayes risk o£ an estimator under truncated squared error loss. The bound derives from an information inequality for the risk under this loss. It is then used to provide new proofs {or aome classical results o£ asymptotic theory. Introduction This paper develops a lower bound for the Bayes risk of an estimator under truncated squared error loss as defined in (1.1). The principle result is Theo-rem 2.1. This bound is then used in §3 to give new proofs for three classical theorems of asymptotic theory concerning the asymptotic Bayes property, local asymptotic minimaxity, and the set of superefficiency. The development begins with an information inequality in §1 for the risk of an estimator under this loss. Another feature of the paper is Table 2.1 which provides a comparison of the bound with the actual Bayes risk in a case in which the latter can be conveniently computed. This paper is a companion to Brown and Gajek (1990). (For convenience that paper will be referred to as BG.) Some of the relevant proofs appear in that paper; but otherwise the papers can be read independently. The results in the present paper were announced as part of my W~d Lecture in August, •Re5ean:b aupported in part by NSF OMS 8506847 and NSF OMS 8809016. AM>MOS Subject Clua.ificat.ion: Primary 62Fl0, Secondary 62F15, 62C99, 60E15. Ke1 Worda .nd Phrues: Inrormation inequalit)', Cramer-R.ao inequalit)', tnmcated aquared error efficiency, loc&1 uymptotic minim&xit)', tuperefticiency. 1 1985 except that the conclusion of (what is now) Theorem 2.1 then read only BK(9) ~ C - D instead of the better statement, BK(9) ~ C2(C + D)-1 • 1 A Bound for the Risk This section describes a lower bound for the risk of an estimator under truncated squared error loss. This bound has a peculiar and uncongenial nature. It is hard to conceive that standing alone it could be of any practical use. However it can be turned into a useful lower bound for the Bayes risk under this loss. This is accomplished in §2. Only the bound for the univariate case will be described. Generalizations to the multivariate case undoubtedly exist but appear to be notationally awkward to state and prove. Let X be an observable random variable with probability density p,,e E e, relative to some Borel measure II. Assume e c lR is an open interval. It is desired to estimate e, and so the action space is e. The Joss function is the truncated version of ordinary quadratic Joss, defined by (1.1) for given 0 < K < oo. Denote the risk function for LK by RK(e,6) and the corresponding expected risk and Bayes risk functions by BK(g, 6) and BK(g) = infBK(g,6). 6 This loss function is not convex; hence the non-randomized estimators are no longer necessarily a complete class. (See Brown, Cohen, and Strawderman (1976) for conditions under which they will be.) For notational simplicity only non-randomized estimators will be explicitly considered below. However, aU the following results are valid also for randomized estimators. Assume that {p1 } satisfies the following two regularity conditions for every eo E e: Plo(z) = 0 => p,(z) = 0 a.e. (v) for all e in a neighborhood of eo, and PI /Pio is weakly differentiable in (1.2) L2(P1odv) at e =eo with weak derivative q. (1.3) (Generally, q = lilnp,(z)l,=lo·) According to Fabian and Hannan (1977) these conditions imply the ordinary information inequality: If Var,0 (T) < oo then (e'(eo)2 Var, 0 (T) ~ !(eo) 2 (1.4) where I (eo) = E,0 (q2(X)) denotes the ordinary Fisher inform&tion &nd e(8) = E,(T ). The Appendix of BG contains several easier to verify conditions (labelled (A.7), (A.6), (A.S)) e&ch of which implies (1.3). Given &n estimater 6 let 8+K if 8+K < 6(z) 6,,K(z)=6(z) if 8-K$6(z)$8+K 8-K if 6(z)<e - K. {1.5) Note that 6,,K is not &n estimator, since it depends on e, but it is & measurable function. Then define (1.6) Under the regularity condit ions (1.2) &nd (1.3) on {p,} for every eo E 9 the function eK(e) will be &bsolutely continuous on e . In fact, e}c(e) will exist at all except at most a countable set of points in e. (The points are those for which p1(16(X) - 81 = K ) > 0.) At those points both left &nd right band derivatives el(t ) and el{r) will exist, but will not be equal . To obt&in an unambiguous st atement in (1.9) and similar expressions which follow make the convention that e}c(8) is the value between el{t)(e) and el{r)(8) for which (1.7) (If le~t) (e)l = let>{8)1 either value c:an be chosen as ejc(8).) Let 1(8) denote the Fisher information, assumed to be finite. For any cr, 0 $ cr < 1, let lK,o(O) = (1112(8) + K - 1(1- cr)-112 ) 2 . (1.8) Note that 1K,o{8) > 1(8). Here is the main result. Theorem 1.1 Make assumptions (1.£} and (1.9} on {p,). Thenforanya ,O $ cr < 1, RK(8,6) ~ lx~0(8)(eic(8))2 + abk(8 ) where hK(O) = eK(e) - 8. (1.9) Proof Under assumptions (1.2), {1.3) the right hand derivative e'}(> exists at 8o and is given by e'}(>(Oo) = j 6,0 ,K(z)q(z)p,0 (z)v(dz) + p,0 (6(X) $ 8o- K or 6(X) > 8o + K). 3 There is a symmetric expression for e~t)(9o). Bence max(let>(9o)l, le~t){9ol) = le}c(9o)l (by 1.7) satisfies lelc(9o)l $ I J 6,0 ,K(z)q(z)p,0 (z)v(dz)l +P10 (16(X)- 9oJ ~ K) = I /(6,0 ,K(z)- tK(9o))q(z)p,0 (z)v(dz)l +Pio(I6(X)- 9ol ~ K). (1.10) Apply Cauchy-Schwarz to the first term on the right of (1.10) and apply Cheby-shev's inequality to the second term to find lelc(9o)l $ I112(8o)Var!~2(6,0 ,K(X)) +(1/ K)E,o(I610 ,K(X)- 9ol). Observe that RK(8o,6) = Var,0 (6,0 ,K(X)) + 6i-(8o) and also that RK(9o ,6) ~ Ei0(I610 ,K(X)- 9ol) to get lelc(Bo)l $ (RK(9o,6) - 6J<(8o))1' 2 I 1' 2(9o) + K-1 R~r(9o, 6). (1.11) Now, for any non-negative numbers 0 <a< 1,0 < 62 < r,r ~ 0 .....:...---'----,---..- < max < 1. (r _ 62)1/2z + K-1r1/2 [ (r _ 62)1/2 r1/2(1 - a)l/2] ( _ 62)1/2 + (r-ao')l'' - (r _ Q62)1/2 ' (r _ Q62)1/2 r Q z K(l-a)lf' Bence 1elc(9o)l $ (RK(9o,6)- Q6k(8o))1f2(J112(8o) + K- 1(1- a)-112). (1.12) Squaring both sides of (1.12) and rearranging terms yields (1.9). • Inequality (1.9) should be compared to t.he ordinary information inequality, which asserts (1.13) Indeed, when R(8,6) < oo then e}c(9) - e'(8) as K - oo and (1.13) follows from (1.9) upon Jetting K- oo then a- 1. For the choice a = 0 one obtains (1.14) A variant of this inequality was obtained many years ago by B. Chernoff (private communication). Inequalities which like (1.14) do not involve a squared bias term will not suffice to yield Bayes risk bounds as in the next section. 2 Bayes Risk Lower Bound Let g be an absolutely continuous prior density. Let VK,o:(9) = Jj(~o:(9): and assume that VK,a is absolutely continuous on 9. Continue to assume (1.2) and (1.3). Let sp(g) = {9: g(O) > 0} and let csp(g) denote its closure. Theorem 2.1 If csp(g) u a compact n6set of 9 and 0 <a< 1 then C2 BK(g)~ C+D ~C-D (2.1) where C = J VK,a(8)g(8)d8 D = a- 1 J [:8(VK,a(8)g(8))r g- 1(8)d8. Proof The proof begins with (1.9) in place of (1.13) and then runs exactly parallel to the proof of Corollary 2.3 in BG. • The condition that csp(g) be a compact subset of 9 is very much stronger than necessary. Here is a formal statement which relaxes that condition. To save space, this statement refers the reader to BG. Corollary 2.1 Assume D < oo. Assume g satisfiu (£.13a) or {£.13b) of BG with VK,a(8) in place of V(8), and with g =h. Then (£.1) u still valid. Proof See the proof of Corollary 2.6 of BG. • It is possible to also carry Theorems 2.7 and 2.9 of BG over to this situation and thereby obtain somewhat improved bounds. Example 2.1: This example gives some idea of the precision of the bound in Theorem 2.1 since BK(g) ean also be exactly computed. It will be seen that the bound only becomes reasonably precise as K - oo. This is basically to be expected because of the use of Chebyshev's inequality, which introduces appreciable imprecision in the proof of Theorem 2.1 except when K is large compared to the variance of X. We nevertheless found the rate of convergence surprisingly slow. This example employs conjugate priors and so should be quite favorable to the bound. In particular, the bound is exact when K = oo. (In this regard see BG, Example 3.1).) · Consider the normal-mean location problem: so X- N(p,1). Let g be the normal density with mean 0, variance tr2• Then Theorem 2.1 and Corollary 2.1 yield (2.2) 5 The value of a which maximizes the right side of (2.2) ean easily be determined numerically. Table 2.1 gives values of the resulting maximum, labelled BOUND. Finally, for comparison the table contains values of BK(g) computed numerically from the fact that BK(g) = E(E((O- rz)2 A K 2IX = z)) = r - _1L loo (t2-K2/r)e-t'/2dt ..j2i K/..;:Y (2.3) where r = u2 /(1 + u2), since (OIX = z) - N(rz, r). (Similar computations appear in Efron and Morris (1971).) u2 1 5 10 25 100 00 BOUND . 057 .127 .156 .187 .216 .250 K = 1 BK(g) .371 .479 .497 .508 .514 .516 BOUND .161 .330 .391 .451 .506 .563 K = 3 BK(g) .500 .832 .906 .958 .985 .995 BOUND .298 .560 .641 .713 .773 .826 K= 10 BK(g) .500 .833 .909 .962 .990 1.000 BOUND .419 .570 .816 .881 .928 .961 K = 50 BK(g) .500 .833 .909 .962 .990 1.000 .BOUND .447 .770 .851 .913 .954 .980 K = 100 BK(g) .500 .833 .909 .962 .990 1.000 BOUND .500 .833 .909 .962 .990 1.000 K = oo BK(g) .500 .833 .909 .962 .990 1.000 Table f.J: Values of the bound and of BK(g) in Example 2.1. 6 Some typical values of a maximizing the right side of (2.2) are .53, .77, .93 for (K, u2) = (1, 1), (10, 1), (100, 1) resp,; .31, .47, .82 for (1, 10), (10, 10), (100, 10) resp.; and .13, .26, .55 for (1, 100), (10, 100), (100, 100). As K- oo(u2 - oo, resp.) the best a - 1 (0, resp.). 3 Asymptotic Bayes and Minimax Properties The bound in §2 provides a very convenient means of proving certain well known classical asymptotic results. Again we treat here only the one dimensional case; multivariate extensions exist for all the following results. Versions of the fol-lowing results were first formulated and proved by LeCam (1953). Lehmann (1983), Strasser (1985), or LeCam (1986) are good contemporary references. Throughout this section let X 1, ... , Xn be independent identically distributed variables satisfying the conditions of Theorem 2.1. Let V(6) = I-1(0) denote the variance bound for a single observation, X l · Let g be any prior density having compact support, or, more generally, satisfying the conditions of Corollary 2.1. Then under ordinary squared error loss, nBCn>(g) > j V(6)g(6)d0 - n-1 j [(V(O)g(O))'] 2 g(O)dO (3.1) - g(6) by BG, (Corollary 2.1) or Borovkov and Sakhanienko (1980). Here, and in similar expressions to follow , B (n ) denotes the Bayes risk based on the sample of size n. FUrthermore, under truncated squared error loss, (1.1) one bas by Theorem 2.1 nB<;}.;n(g) ~ J VK,o(O)g(O)dO -<nQ>-l j [WK.oi~jf'6>>'r g(O)do. (3.2} For what follows it is important only that for fixed 0 < Q < 1 VK,o(O)- V(O) (3.3) as K - oo, with a similar statement concerning the derivative of Vt ,o· To simplify the following proofs assume also that V(O) is continuously differentiable on 9. Here are some consequences of (3.1) and (3.2). Consequence 3.1: Aasume j [(V(:~:~O))'], g(O)dO < oo. 7 Then the "limiting Bayes risk '", lim inf nB(g), satisfies n-oo ~~f nB<">(g) ~ j V(O)g(O)d8. (3.4) The "asymptotic Bayes risk", lim lim infnBK(n/) =(9) also satisfies K-co n-oo vn lim liminfnBK(n/) =(g)~ jv(O)g(8)d8. (3.4') K-oo n-oo vn Proof (3.4) is immediate from (3.1) and (3.4') is immediate from (3.2) and (3.3). • Under suitable conditions on {p,} the maximum likelihood estimate, 0, has asymptotic risk V(O). See, e.g., Lehmann (1983). Under slightly more stringent conditions it also has limiting risk V(O). Thus, under suitable conditions the maximum likelihood estimator is asymptotically Bayes- i.e., satisfies lim liminff nRK(n/) r-(O,O)g(O)d8 = lim liminfnRK(n/) =(g). K-oon-oo vn K-oon-oo vn Under slightly more stringent conditions it is also limiting Bayes- i.e., satisfies lim jnR<">(O,O)g(8)d8 = liminf nB<">(g). n--oo n-o::> Hodges demonstrated the existence of a superefficient sequence of estimators. LeCam (1953) then proved (under suitable conditions) that the set of supereffi-ciency has Lebesgue measure zero. For our purposes a sequence of estimators, {6n}, is superefficient. at 8 E 6 if lim limsupnR<;1> r,r(6,6n) < V(O). K-oo n-oo V" (3.5} Consequence 3.2: Let {6n} be a given sequence of estimators. Under the above assumptions the set of parameter points at which {cSn} is superefficient has Lebesgue measure zero. Proof Let S C (J be the set of superefficiency and suppose the Lebesgue measure of S, ~(S), is positive. Let 0 < Ct < 1. Then, since VK,o(O)- V(fJ) as K- oo, there must exist N < oo, 1 < K < oo,O < f <·1, and an S' C S with ~(S') > 0 such that nR<;}.;n(9) < VK,o(O)- £,8 E S', (3.6} 8 for all n ~ N. Since .\(S') > 0 there must exist a (possibly small) open interval I = (a, b) such that .\(1- S') ( .\(1) < 9K2 • (3.7) Let g(O) = (12/(b- a)3){[min((9- a), (b- 8))]+}2• (3.8) Note that I [<v" ... ,~it'>)'r g(O)dB < oo. Hence (3.2) yields the existence of an N such that for n > N, nB<_;},;n(g) > j V#,a(B)g(8)d8- £/3. (3.9) On the other hand g(B) = 0 for 8 ~I and 0 5 g(B) < 3/(b - a). Note also that nB~j,;n(g) < K 2 • Hence j nR<;}.;n(0,6n)g(8)d8 5 f (VK,a(B)- £)g(O)d8 + 1<2 f g(O)dO ls• l1-s• < J VK,a(O)g(B)dO - £ f g(8)d8 + K 2 f g(O)dO ls• J1-s• < J VK,a(8)g(8)d8- £/3 (3.10) since Is• g(O)dO ~ [1<.::>r ~ (1 - £/9I<2) 3 ~ (1- 1/9)3 > 2/3, and g(O) 5 3/(b- a) so that I1_ 5 , g(O)dO 5 (3/(b - a)).\(1- S') < £/3K 2• But, (3.9) and (3.10) together claim that I nR<;).;n(O, 6n)g(O)d8 < nB<;}...;n(g), a contradiction. It follows that .\(S) = 0. • A third classic property about which (3.1) and (3.2) yield good bounds is lo-cal asymptotic minimaxity. For the purpose at hand define the local asymptotic minimax value, mo, at Bo, as mo = lim lim liminf inf sup R<;},;n(8,6n)· (3.11) K-oo D-oo n-oo {~.} 11-loi<D/Vn Consequence 3.3: mo = V(Bo). Proof Let 8o = 0, with no loss of generality, and (3.12) 9 similarly to (3.8). Then inf sup K;}..;n(9,6n) ~ B<;}..;n(gn,D) {~.} lll<D/.,fii ~ j Vx,o(O)gn,D(9)dO - (na)- 1 j [(Vx,o(:{~).D(O))'r g(O)d9 by (3.2). Now, for some C < oo, Vx,o(O) < C and V.K,0 (0) < C for 101 < Df.fii, as a consequence of the assumptions on 1(9) and the definition of Vx,o· Hence Consequently, lim liminf B<;>r-(9n,D) ~ liminfj Vx,o(9)gn,D(9)d0 D-oo n - oo vn n - oo = Vx,o(O). It follows that mo = V(O) since ·Jim Vx,0 (9) = V(O). • K-oo Under suitable conditions on {p,} the maximum likelihood estimator, 8, satisfies V(9o) = lim lim lim sup R<;1> ~n(O,B) K-ooD-oon-ooll-loi<Df.,fii vu and hence is locally asymtotically minimax. See Hajek (1972) and, again, Lehmann {1983) for a good general account of the theory plus many further references. References Brown, L.D., Cohen, A., and Strawderman, W.E. {1976). A complete class theorem for monotone likelihood ratio with applications. Ann. Statist. .j, 712-722. Brown, L.D. and Gajek L. (1990). Information inequalities for the Bayes risk (referred here to as BG). Ann. Stati$t. 18, 1578-1594. Borovkov, A.A. and Sakhanienko, A.U. (1980). On estimates of the expected quadratic risk. Prob. and Math. Statist. 1, 185-195 (in Russian). 10 Efron, B. and Morris, C. (1971). Limiting the risk of Bayes and empirical Bayes Estimators- Part 1: The Bayes case. J . .Amer. Statist . .Assoc. 66, 807-815. Fabian, V. and Bannan, J. {1977). On the Cramer Rao inequality. .Ann. Statist. 5, 197-205. Hajek, J. (1972). Local asymptotic minimaxity and admissibility in estimation. Proc. Si%th Berk. Symp. Math. . Statist. and Pro6. 1, 17~194. LeCam, L. {1953). On some asymptotic properties of maximum likelihood estimates and related Bayes estimates. Univ. of Calif Pub/. in Statist. 1, 277-330. LeCam, L. {1986) . .Asymptotic Methods in Statistical Decision Theory. Springer. Lehmann, E.L. (1983). Theory of Point Estimation. Wiley and Sons. Strasser, H. {1985). MathemD.iical Theory of Statistics-Statistical Erperiments and .Asymptotic Decision Theory. W. de Gruyter. 11 

An Information Inequality for the Bayes Risk Under Truncated Squared Error Loss

Kosmopolis

University of Pennsylvania
ScholarlyCommons
Statistics Papers Wharton Faculty Research
1993
An Information Inequality for the Bayes Risk
Under Truncated Squared Error Loss
Lawrence D. Brown
University of Pennsylvania
Follow this and additional works at: http://repository.upenn.edu/statistics_papers
Part of the Statistics and Probability Commons
This paper is posted at ScholarlyCommons. http://repository.upenn.edu/statistics_papers/105
For more information, please contact repository@pobox.upenn.edu.
Recommended Citation
Brown, L. D. (1993). An Information Inequality for the Bayes Risk Under Truncated Squared Error Loss. Proceedings of Penn State
Conference on Multivariate Analysis, Retrieved from http://repository.upenn.edu/statistics_papers/105
An Information Inequality for the Bayes Risk Under Truncated Squared
Error Loss
Abstract
A bound is given for the Bayes risk of an estimator under truncated squared error loss. The bound derives
from an information inequality for the risk under this loss. It is then used to provide new proofs for some
classical results of asymptotic theory.
Keywords
Information inequality, cramer-rao inequality, truncated squared error efficiency, local asymptotic minimaxity,
superefficiency
Disciplines
Statistics and Probability
This conference paper is available at ScholarlyCommons: http://repository.upenn.edu/statistics_papers/105
An Information Inequality for the Bayes Risk 
under Truncated Squared Error Loss 
Lawrence D. Brown• 
Department of Mathematics 
Cornell University 
Ithaca, NY 14853 
April 30, 1992 
Abstract 
A bound is given {or the Bayes risk o£ an estimator under truncated 
squared error loss. The bound derives from an information inequality for 
the risk under this loss. It is then used to provide new proofs {or aome 
classical results o£ asymptotic theory. 
Introduction 
This paper develops a lower bound for the Bayes risk of an estimator under 
truncated squared error loss as defined in (1.1). The principle result is Theo-
rem 2.1. This bound is then used in §3 to give new proofs for three classical 
theorems of asymptotic theory concerning the asymptotic Bayes property, local 
asymptotic minimaxity, and the set of superefficiency. 
The development begins with an information inequality in §1 for the risk of 
an estimator under this loss. Another feature of the paper is Table 2.1 which 
provides a comparison of the bound with the actual Bayes risk in a case in which 
the latter can be conveniently computed. 
This paper is a companion to Brown and Gajek (1990). (For convenience 
that paper will be referred to as BG.) Some of the relevant proofs appear in 
that paper; but otherwise the papers can be read independently. The results 
in the present paper were announced as part of my W~d Lecture in August, 
•Re5ean:b aupported in part by NSF OMS 8506847 and NSF OMS 8809016. AM>MOS 
Subject Clua.ificat.ion: Primary 62Fl0, Secondary 62F15, 62C99, 60E15. Ke1 Worda .nd 
Phrues: Inrormation inequalit)', Cramer-R.ao inequalit)', tnmcated aquared error efficiency, 
loc&1 uymptotic minim&xit)', tuperefticiency. 
1 
1985 except that the conclusion of (what is now) Theorem 2.1 then read only 
BK(9) ~ C - D instead of the better statement, BK(9) ~ C2(C + D)-1 • 
1 A Bound for the Risk 
This section describes a lower bound for the risk of an estimator under truncated 
squared error loss. This bound has a peculiar and uncongenial nature. It is hard 
to conceive that standing alone it could be of any practical use. However it can 
be turned into a useful lower bound for the Bayes risk under this loss. This is 
accomplished in §2. 
Only the bound for the univariate case will be described. Generalizations to 
the multivariate case undoubtedly exist but appear to be notationally awkward 
to state and prove. 
Let X be an observable random variable with probability density p,,e E e, 
relative to some Borel measure II. Assume e c lR is an open interval. It is 
desired to estimate e, and so the action space is e. The Joss function is the 
truncated version of ordinary quadratic Joss, defined by 
(1.1) 
for given 0 < K < oo. Denote the risk function for LK by RK(e,6) and the 
corresponding expected risk and Bayes risk functions by BK(g, 6) and BK(g) 
= infBK(g,6). 
6 
This loss function is not convex; hence the non-randomized estimators are 
no longer necessarily a complete class. (See Brown, Cohen, and Strawderman 
(1976) for conditions under which they will be.) For notational simplicity only 
non-randomized estimators will be explicitly considered below. However, aU the 
following results are valid also for randomized estimators. 
Assume that {p1 } satisfies the following two regularity conditions for every 
eo E e: 
Plo(z) = 0 => p,(z) = 0 a.e. (v) 
for all e in a neighborhood of eo, and 
PI /Pio is weakly differentiable in 
(1.2) 
L2(P1odv) at e =eo with weak derivative q. (1.3) 
(Generally, q = lilnp,(z)l,=lo·) According to Fabian and Hannan (1977) these 
conditions imply the ordinary information inequality: If Var,
0
(T) < oo then 
(e'(eo)2 
Var, 0 (T) ~ !(eo) 
2 
(1.4) 
where I (eo) = E,0 (q2(X)) denotes the ordinary Fisher inform&tion &nd e(8) = 
E,(T ). 
The Appendix of BG contains several easier to verify conditions (labelled 
(A.7), (A.6), (A.S)) e&ch of which implies (1.3). 
Given &n estimater 6 let 
8+K if 8+K < 6(z) 
6,,K(z)=6(z) if 8-K$6(z)$8+K 
8-K if 6(z)<e - K. 
{1.5) 
Note that 6,,K is not &n estimator, since it depends on e, but it is & measurable 
function. Then define 
(1.6) 
Under the regularity condit ions (1.2) &nd (1.3) on {p,} for every eo E 9 the 
function eK(e) will be &bsolutely continuous on e . In fact, e}c(e) will exist at all 
except at most a countable set of points in e. (The points are those for which 
p1(16(X) - 81 = K ) > 0.) At those points both left &nd right band derivatives 
el(t ) and el{r) will exist, but will not be equal . To obt&in an unambiguous 
st atement in (1.9) and similar expressions which follow make the convention 
that e}c(8) is the value between el{t)(e) and el{r)(8) for which 
(1.7) 
(If le~t) (e)l = let>{8)1 either value c:an be chosen as ejc(8).) 
Let 1(8) denote the Fisher information, assumed to be finite. For any cr, 0 $ 
cr < 1, let 
lK,o(O) = (1112(8) + K - 1(1- cr)-112 ) 2 . (1.8) 
Note that 1K,o{8) > 1(8). Here is the main result. 
Theorem 1.1 Make assumptions (1.£} and (1.9} on {p,). Thenforanya ,O $ 
cr < 1, 
RK(8,6) ~ lx~0(8)(eic(8))2 + abk(8 ) 
where hK(O) = eK(e) - 8. 
(1.9) 
Proof Under assumptions (1.2), {1.3) the right hand derivative e'}(> exists at 
8o and is given by 
e'}(>(Oo) = j 6,0 ,K(z)q(z)p,0 (z)v(dz) 
+ p,0 (6(X) $ 8o- K or 6(X) > 8o + K). 
3 
There is a symmetric expression for e~t)(9o). Bence max(let>(9o)l, le~t){9ol) = 
le}c(9o)l (by 1.7) satisfies 
lelc(9o)l $ I J 6,0 ,K(z)q(z)p,0 (z)v(dz)l 
+P10 (16(X)- 9oJ ~ K) 
= I /(6,0 ,K(z)- tK(9o))q(z)p,0 (z)v(dz)l 
+Pio(I6(X)- 9ol ~ K). (1.10) 
Apply Cauchy-Schwarz to the first term on the right of (1.10) and apply Cheby-
shev's inequality to the second term to find 
lelc(9o)l $ I112(8o)Var!~2(6,0 ,K(X)) 
+(1/ K)E,o(I610 ,K(X)- 9ol). 
Observe that RK(8o,6) = Var,0 (6,0 ,K(X)) + 6i-(8o) and also that RK(9o ,6) ~ 
Ei
0
(I610 ,K(X)- 9ol) to get 
lelc(Bo)l $ (RK(9o,6) - 6J<(8o))1' 2 I 1' 2(9o) + K-1 R~r(9o, 6). (1.11) 
Now, for any non-negative numbers 0 <a< 1,0 < 62 < r,r ~ 0 
.....:...---'----,---..- < max < 1. 
(r _ 62)1/2z + K-1r1/2 [ (r _ 62)1/2 r1/2(1 - a)l/2] 
( _ 62)1/2 + (r-ao')l'' - (r _ Q62)1/2 ' (r _ Q62)1/2 r Q z K(l-a)lf' 
Bence 
1elc(9o)l $ (RK(9o,6)- Q6k(8o))1f2(J112(8o) + K- 1(1- a)-112). (1.12) 
Squaring both sides of (1.12) and rearranging terms yields (1.9). • 
Inequality (1.9) should be compared to t.he ordinary information inequality, 
which asserts 
(1.13) 
Indeed, when R(8,6) < oo then e}c(9) - e'(8) as K - oo and (1.13) follows 
from (1.9) upon Jetting K- oo then a- 1. 
For the choice a = 0 one obtains 
(1.14) 
A variant of this inequality was obtained many years ago by B. Chernoff (private 
communication). Inequalities which like (1.14) do not involve a squared bias 
term will not suffice to yield Bayes risk bounds as in the next section. 
2 Bayes Risk Lower Bound 
Let g be an absolutely continuous prior density. Let VK,o:(9) = Jj(~o:(9): and 
assume that VK,a is absolutely continuous on 9. Continue to assume (1.2) and 
(1.3). Let sp(g) = {9: g(O) > 0} and let csp(g) denote its closure. 
Theorem 2.1 If csp(g) u a compact n6set of 9 and 0 <a< 1 then 
C2 
BK(g)~ C+D ~C-D (2.1) 
where 
C = J VK,a(8)g(8)d8 
D = a- 1 J [:8(VK,a(8)g(8))r g- 1(8)d8. 
Proof The proof begins with (1.9) in place of (1.13) and then runs exactly 
parallel to the proof of Corollary 2.3 in BG. • 
The condition that csp(g) be a compact subset of 9 is very much stronger 
than necessary. Here is a formal statement which relaxes that condition. To 
save space, this statement refers the reader to BG. 
Corollary 2.1 Assume D < oo. Assume g satisfiu (£.13a) or {£.13b) of BG 
with VK,a(8) in place of V(8), and with g =h. Then (£.1) u still valid. 
Proof See the proof of Corollary 2.6 of BG. • 
It is possible to also carry Theorems 2.7 and 2.9 of BG over to this situation 
and thereby obtain somewhat improved bounds. 
Example 2.1: This example gives some idea of the precision of the bound 
in Theorem 2.1 since BK(g) ean also be exactly computed. It will be seen 
that the bound only becomes reasonably precise as K - oo. This is basically 
to be expected because of the use of Chebyshev's inequality, which introduces 
appreciable imprecision in the proof of Theorem 2.1 except when K is large 
compared to the variance of X. We nevertheless found the rate of convergence 
surprisingly slow. This example employs conjugate priors and so should be quite 
favorable to the bound. In particular, the bound is exact when K = oo. (In this 
regard see BG, Example 3.1).) · 
Consider the normal-mean location problem: so X- N(p,1). Let g be the 
normal density with mean 0, variance tr2• Then Theorem 2.1 and Corollary 2.1 
yield 
(2.2) 
5 
The value of a which maximizes the right side of (2.2) ean easily be determined 
numerically. Table 2.1 gives values of the resulting maximum, labelled BOUND. 
Finally, for comparison the table contains values of BK(g) computed numerically 
from the fact that 
BK(g) = E(E((O- rz)2 A K 2IX = z)) 
= r - _1L loo (t2-K2/r)e-t'/2dt 
..j2i K/..;:Y 
(2.3) 
where r = u2 /(1 + u2), since (OIX = z) - N(rz, r). (Similar computations 
appear in Efron and Morris (1971).) 
u2 
1 5 10 25 100 00 
BOUND . 057 .127 .156 .187 .216 .250 
K = 1 
BK(g) .371 .479 .497 .508 .514 .516 
BOUND .161 .330 .391 .451 .506 .563 
K = 3 
BK(g) .500 .832 .906 .958 .985 .995 
BOUND .298 .560 .641 .713 .773 .826 
K= 10 
BK(g) .500 .833 .909 .962 .990 1.000 
BOUND .419 .570 .816 .881 .928 .961 
K = 50 
BK(g) .500 .833 .909 .962 .990 1.000 
.BOUND .447 .770 .851 .913 .954 .980 
K = 100 
BK(g) .500 .833 .909 .962 .990 1.000 
BOUND .500 .833 .909 .962 .990 1.000 
K = oo 
BK(g) .500 .833 .909 .962 .990 1.000 
Table f.J: Values of the bound and of BK(g) in Example 2.1. 
6 
Some typical values of a maximizing the right side of (2.2) are .53, .77, 
.93 for (K, u2) = (1, 1), (10, 1), (100, 1) resp,; .31, .47, .82 for (1, 10), (10, 
10), (100, 10) resp.; and .13, .26, .55 for (1, 100), (10, 100), (100, 100). As 
K- oo(u2 - oo, resp.) the best a - 1 (0, resp.). 
3 Asymptotic Bayes and Minimax Properties 
The bound in §2 provides a very convenient means of proving certain well known 
classical asymptotic results. Again we treat here only the one dimensional case; 
multivariate extensions exist for all the following results. Versions of the fol-
lowing results were first formulated and proved by LeCam (1953). Lehmann 
(1983), Strasser (1985), or LeCam (1986) are good contemporary references. 
Throughout this section let X 1, ... , Xn be independent identically distributed 
variables satisfying the conditions of Theorem 2.1. Let V(6) = I-1(0) denote 
the variance bound for a single observation, X l · 
Let g be any prior density having compact support, or, more generally, 
satisfying the conditions of Corollary 2.1. Then under ordinary squared error 
loss, 
nBCn>(g) > j V(6)g(6)d0 - n-1 j [(V(O)g(O))'] 2 g(O)dO (3.1) 
- g(6) 
by BG, (Corollary 2.1) or Borovkov and Sakhanienko (1980). Here, and in 
similar expressions to follow , B (n ) denotes the Bayes risk based on the sample 
of size n. FUrthermore, under truncated squared error loss, (1.1) one bas by 
Theorem 2.1 
nB<;}.;n(g) ~ J VK,o(O)g(O)dO 
-<nQ>-l j [WK.oi~jf'6>>'r g(O)do. (3.2} 
For what follows it is important only that for fixed 0 < Q < 1 
VK,o(O)- V(O) (3.3) 
as K - oo, with a similar statement concerning the derivative of Vt ,o· To 
simplify the following proofs assume also that V(O) is continuously differentiable 
on 9. 
Here are some consequences of (3.1) and (3.2). 
Consequence 3.1: Aasume 
j [(V(:~:~O))'], g(O)dO < oo. 
7 
Then the "limiting Bayes risk '", lim inf nB(g), satisfies 
n-oo 
~~f nB<">(g) ~ j V(O)g(O)d8. (3.4) 
The "asymptotic Bayes risk", lim lim infnBK(n/) =(9) also satisfies 
K-co n-oo vn 
lim liminfnBK(n/) =(g)~ jv(O)g(8)d8. (3.4') 
K-oo n-oo vn 
Proof (3.4) is immediate from (3.1) and (3.4') is immediate from (3.2) and 
(3.3). • 
Under suitable conditions on {p,} the maximum likelihood estimate, 0, has 
asymptotic risk V(O). See, e.g., Lehmann (1983). Under slightly more stringent 
conditions it also has limiting risk V(O). Thus, under suitable conditions the 
maximum likelihood estimator is asymptotically Bayes- i.e., satisfies 
lim liminff nRK(n/) r-(O,O)g(O)d8 = lim liminfnRK(n/) =(g). K-oon-oo vn K-oon-oo vn 
Under slightly more stringent conditions it is also limiting Bayes- i.e., satisfies 
lim jnR<">(O,O)g(8)d8 = liminf nB<">(g). 
n--oo n-o::> 
Hodges demonstrated the existence of a superefficient sequence of estimators. 
LeCam (1953) then proved (under suitable conditions) that the set of supereffi-
ciency has Lebesgue measure zero. For our purposes a sequence of estimators, 
{6n}, is superefficient. at 8 E 6 if 
lim limsupnR<;1> r,r(6,6n) < V(O). K-oo n-oo V" (3.5} 
Consequence 3.2: Let {6n} be a given sequence of estimators. Under the 
above assumptions the set of parameter points at which {cSn} is superefficient 
has Lebesgue measure zero. 
Proof Let S C (J be the set of superefficiency and suppose the Lebesgue 
measure of S, ~(S), is positive. Let 0 < Ct < 1. Then, since VK,o(O)- V(fJ) as 
K- oo, there must exist N < oo, 1 < K < oo,O < f <·1, and an S' C S with 
~(S') > 0 such that 
nR<;}.;n(9) < VK,o(O)- £,8 E S', (3.6} 
8 
for all n ~ N. Since .\(S') > 0 there must exist a (possibly small) open interval 
I = (a, b) such that 
.\(1- S') ( 
.\(1) < 9K2 • (3.7) 
Let 
g(O) = (12/(b- a)3){[min((9- a), (b- 8))]+}2• (3.8) 
Note that I [<v" ... ,~it'>)'r g(O)dB < oo. Hence (3.2) yields the existence of an 
N such that for n > N, 
nB<_;},;n(g) > j V#,a(B)g(8)d8- £/3. (3.9) 
On the other hand g(B) = 0 for 8 ~I and 0 5 g(B) < 3/(b - a). Note also that 
nB~j,;n(g) < K 2 • Hence 
j nR<;}.;n(0,6n)g(8)d8 
5 f (VK,a(B)- £)g(O)d8 + 1<2 f g(O)dO 
ls• l1-s• 
< J VK,a(O)g(B)dO - £ f g(8)d8 + K 2 f g(O)dO 
ls• J1-s• 
< J VK,a(8)g(8)d8- £/3 (3.10) 
since Is• g(O)dO ~ [1<.::>r ~ (1 - £/9I<2) 3 ~ (1- 1/9)3 > 2/3, and g(O) 5 
3/(b- a) so that I1_ 5 , g(O)dO 5 (3/(b - a)).\(1- S') < £/3K 2• 
But, (3.9) and (3.10) together claim that I nR<;).;n(O, 6n)g(O)d8 < nB<;}...;n(g), 
a contradiction. It follows that .\(S) = 0. • 
A third classic property about which (3.1) and (3.2) yield good bounds is lo-
cal asymptotic minimaxity. For the purpose at hand define the local asymptotic 
minimax value, mo, at Bo, as 
mo = lim lim liminf inf sup R<;},;n(8,6n)· (3.11) 
K-oo D-oo n-oo {~.} 11-loi<D/Vn 
Consequence 3.3: mo = V(Bo). 
Proof Let 8o = 0, with no loss of generality, and 
(3.12) 
9 
similarly to (3.8). Then 
inf sup K;}..;n(9,6n) ~ B<;}..;n(gn,D) 
{~.} lll<D/.,fii 
~ j Vx,o(O)gn,D(9)dO - (na)- 1 j [(Vx,o(:{~).D(O))'r g(O)d9 
by (3.2). Now, for some C < oo, Vx,o(O) < C and V.K,0 (0) < C for 101 < Df.fii, 
as a consequence of the assumptions on 1(9) and the definition of Vx,o· Hence 
Consequently, 
lim liminf B<;>r-(9n,D) ~ liminfj Vx,o(9)gn,D(9)d0 
D-oo n - oo vn n - oo 
= Vx,o(O). 
It follows that mo = V(O) since ·Jim Vx,0 (9) = V(O). • 
K-oo 
Under suitable conditions on {p,} the maximum likelihood estimator, 8, 
satisfies 
V(9o) = lim lim lim sup R<;1> ~n(O,B) K-ooD-oon-ooll-loi<Df.,fii vu 
and hence is locally asymtotically minimax. See Hajek (1972) and, again, 
Lehmann {1983) for a good general account of the theory plus many further 
references. 
References 
Brown, L.D., Cohen, A., and Strawderman, W.E. {1976). A complete class 
theorem for monotone likelihood ratio with applications. Ann. Statist. .j, 
712-722. 
Brown, L.D. and Gajek L. (1990). Information inequalities for the Bayes risk 
(referred here to as BG). Ann. Stati$t. 18, 1578-1594. 
Borovkov, A.A. and Sakhanienko, A.U. (1980). On estimates of the expected 
quadratic risk. Prob. and Math. Statist. 1, 185-195 (in Russian). 
10 
Efron, B. and Morris, C. (1971). Limiting the risk of Bayes and empirical 
Bayes Estimators- Part 1: The Bayes case. J . .Amer. Statist . .Assoc. 66, 
807-815. 
Fabian, V. and Bannan, J. {1977). On the Cramer Rao inequality. .Ann. 
Statist. 5, 197-205. 
Hajek, J. (1972). Local asymptotic minimaxity and admissibility in estimation. 
Proc. Si%th Berk. Symp. Math. . Statist. and Pro6. 1, 17~194. 
LeCam, L. {1953). On some asymptotic properties of maximum likelihood 
estimates and related Bayes estimates. Univ. of Calif Pub/. in Statist. 
1, 277-330. 
LeCam, L. {1986) . .Asymptotic Methods in Statistical Decision Theory. Springer. 
Lehmann, E.L. (1983). Theory of Point Estimation. Wiley and Sons. 
Strasser, H. {1985). MathemD.iical Theory of Statistics-Statistical Erperiments 
and .Asymptotic Decision Theory. W. de Gruyter. 
11 



https://repository.upenn.edu/cgi/viewcontent.cgi?article=1497&amp;context=statistics_papers

An Information Inequality for the Bayes Risk Under Truncated Squared Error Loss

Abstract

Similar works

Full text

Available Versions

ScholarlyCommons@Penn

Kosmopolis