14 research outputs found
Compound poisson approximation via information functionals
An information-theoretic development is given for the problem of compound Poisson approximation, which parallels earlier treatments for Gaussian and Poisson approximation. Nonasymptotic bounds are derived for the distance between the distribution of a sum of independent integer-valued random variables and an appropriately chosen compound Poisson law. In the case where all summands have the same conditional distribution given that they are non-zero, a bound on the relative entropy distance between their sum and the compound Poisson distribution is derived, based on the data-processing property of relative entropy and earlier Poisson approximation results. When the summands have arbitrary distributions, corresponding bounds are derived in terms of the total variation distance. The main technical ingredient is the introduction of two "information functionals,'' and the analysis of their properties. These information functionals play a role analogous to that of the classical Fisher information in normal approximation. Detailed comparisons are made between the resulting inequalities and related bounds
On a connection between Stein characterizations and Fisher information
We generalize the so-called density approach to Stein characterizations of
probability distributions. We prove an elementary factorization property of the
resulting Stein operator in terms of a generalized (standardized) score
function. We use this result to connect Stein characterizations with
information distances such as the generalized (standardized) Fisher
information
Log-concavity, ultra-log-concavity, and a maximum entropy property of discrete compound Poisson measures
Sufficient conditions are developed, under which the compound Poisson
distribution has maximal entropy within a natural class of probability measures
on the nonnegative integers. Recently, one of the authors [O. Johnson, {\em
Stoch. Proc. Appl.}, 2007] used a semigroup approach to show that the Poisson
has maximal entropy among all ultra-log-concave distributions with fixed mean.
We show via a non-trivial extension of this semigroup approach that the natural
analog of the Poisson maximum entropy property remains valid if the compound
Poisson distributions under consideration are log-concave, but that it fails in
general. A parallel maximum entropy result is established for the family of
compound binomial measures. Sufficient conditions for compound distributions to
be log-concave are discussed and applications to combinatorics are examined;
new bounds are derived on the entropy of the cardinality of a random
independent set in a claw-free graph, and a connection is drawn to Mason's
conjecture for matroids. The present results are primarily motivated by the
desire to provide an information-theoretic foundation for compound Poisson
approximation and associated limit theorems, analogous to the corresponding
developments for the central limit theorem and for Poisson approximation. Our
results also demonstrate new links between some probabilistic methods and the
combinatorial notions of log-concavity and ultra-log-concavity, and they add to
the growing body of work exploring the applications of maximum entropy
characterizations to problems in discrete mathematics.Comment: 30 pages. This submission supersedes arXiv:0805.4112v1. Changes in
v2: Updated references, typos correcte