2 research outputs found
Estimating Renyi Entropy of Discrete Distributions
It was recently shown that estimating the Shannon entropy of a
discrete -symbol distribution requires samples,
a number that grows near-linearly in the support size. In many applications
can be replaced by the more general R\'enyi entropy of order
, . We determine the number of samples needed to
estimate for all , showing that
requires a super-linear, roughly samples, noninteger
requires a near-linear samples, but, perhaps surprisingly, integer
requires only samples. Furthermore,
developing on a recently established connection between polynomial
approximation and estimation of additive functions of the form , we reduce the sample complexity for noninteger values of by a
factor of compared to the empirical estimator. The estimators
achieving these bounds are simple and run in time linear in the number of
samples. Our lower bounds provide explicit constructions of distributions with
different R\'enyi entropies that are hard to distinguish