Operational Interpretation of Renyi Information Measures via Composite Hypothesis Testing Against Product and Markov Distributions

We revisit the problem of asymmetric binary hypothesis testing against a composite alternative hypothesis. We introduce a general framework to treat such problems when the alternative hypothesis adheres to certain axioms. In this case we find the threshold rate, the optimal error and strong converse exponents (at large deviations from the threshold) and the second order asymptotics (at small deviations from the threshold). We apply our results to find operational interpretations of various Renyi information measures. In case the alternative hypothesis is comprised of bipartite product distributions, we find that the optimal error and strong converse exponents are determined by variations of Renyi mutual information. In case the alternative hypothesis consists of tripartite distributions satisfying the Markov property, we find that the optimal exponents are determined by variations of Renyi conditional mutual information. In either case the relevant notion of Renyi mutual information depends on the precise choice of the alternative hypothesis. As such, our work also strengthens the view that different definitions of Renyi mutual information, conditional entropy and conditional mutual information are adequate depending on the context in which the measures are used.Comment: published versio

Universal channel coding for general output alphabet

We propose two types of universal codes that are suited to two asymptotic regimes when the output alphabet is possibly continuous. The first class has the property that the error probability decays exponentially fast and we identify an explicit lower bound on the error exponent. The other class attains the epsilon-capacity the channel and we also identify the second-order term in the asymptotic expansion. The proposed encoder is essentially based on the packing lemma of the method of types. For the decoder, we first derive a R\'enyi-relative-entropy version of Clarke and Barron's formula the distance between the true distribution and the Bayesian mixture, which is of independent interest. The universal decoder is stated in terms of this formula and quantities used in the information spectrum method. The methods contained herein allow us to analyze universal codes for channels with continuous and discrete output alphabets in a unified manner, and to analyze their performances in terms of the exponential decay of the error probability and the second-order coding rate.Comment: Several typos are fixe

Equivocations, Exponents and Second-Order Coding Rates under Various R\'enyi Information Measures

We evaluate the asymptotics of equivocations, their exponents as well as their second-order coding rates under various R\'{e}nyi information measures. Specifically, we consider the effect of applying a hash function on a source and we quantify the level of non-uniformity and dependence of the compressed source from another correlated source when the number of copies of the sources is large. Unlike previous works that use Shannon information measures to quantify randomness, information or uniformity, we define our security measures in terms of a more general class of information measures--the R\'{e}nyi information measures and their Gallager-type counterparts. A special case of these R\'{e}nyi information measure is the class of Shannon information measures. We prove tight asymptotic results for the security measures and their exponential rates of decay. We also prove bounds on the second-order asymptotics and show that these bounds match when the magnitudes of the second-order coding rates are large. We do so by establishing new classes non-asymptotic bounds on the equivocation and evaluating these bounds using various probabilistic limit theorems asymptotically.Comment: 47 pages, 9 figures; Presented at the 2015 International Symposium on Information Theory (Hong Kong); Submitted to the IEEE Transactions on Information Theory; v3: fixed typos and added some clarifications to the proof