13 research outputs found
Recommended from our members
On defining partition entropy by inequalities
Partition entropy is the numerical metric of uncertainty within
a partition of a finite set, while conditional entropy measures the degree of
difficulty in predicting a decision partition when a condition partition is
provided. Since two direct methods exist for defining conditional entropy
based on its partition entropy, the inequality postulates of monotonicity,
which conditional entropy satisfies, are actually additional constraints on
its entropy. Thus, in this paper partition entropy is defined as a function
of probability distribution, satisfying all the inequalities of not only partition
entropy itself but also its conditional counterpart. These inequality
postulates formalize the intuitive understandings of uncertainty contained
in partitions of finite sets.We study the relationships between these inequalities,
and reduce the redundancies among them. According to two different
definitions of conditional entropy from its partition entropy, the convenient
and unified checking conditions for any partition entropy are presented, respectively.
These properties generalize and illuminate the common nature
of all partition entropies
Variable-length compression allowing errors
This paper studies the fundamental limits of the minimum average length of
lossless and lossy variable-length compression, allowing a nonzero error
probability , for lossless compression. We give non-asymptotic bounds
on the minimum average length in terms of Erokhin's rate-distortion function
and we use those bounds to obtain a Gaussian approximation on the speed of
approach to the limit which is quite accurate for all but small blocklengths:
where is the functional
inverse of the standard Gaussian complementary cdf, and is the
source dispersion. A nonzero error probability thus not only reduces the
asymptotically achievable rate by a factor of , but this
asymptotic limit is approached from below, i.e. larger source dispersions and
shorter blocklengths are beneficial. Variable-length lossy compression under an
excess distortion constraint is shown to exhibit similar properties