Arimoto channel coding converse and Rényi divergence

Abstract-Arimoto [1] proved a non-asymptotic upper bound on the probability of successful decoding achievable by any code on a given discrete memoryless channel. In this paper we present a simple derivation of the Arimoto converse based on the dataprocessing inequality for Rényi divergence. The method has two benefits. First, it generalizes to codes with feedback and gives the simplest proof of the strong converse for the DMC with feedback. Second, it demonstrates that the sphere-packing bound is strictly tighter than Arimoto converse for all channels, blocklengths and rates, since in fact we derive the latter from the former. Finally, we prove similar results for other (non-Rényi) divergence measures

The Sphere Packing Bound for DSPCs with Feedback a la Augustin

Establishing the sphere packing bound for block codes on the discrete stationary product channels with feedback ---which are commonly called the discrete memoryless channels with feedback--- was considered to be an open problem until recently, notwithstanding the proof sketch provided by Augustin in 1978. A complete proof following Augustin's proof sketch is presented, to demonstrate its adequacy and to draw attention to two novel ideas it employs. These novel ideas (i.e., the Augustin's averaging and the use of subblocks) are likely to be applicable in other communication problems for establishing impossibility results.Comment: 12 pages, 2 figure

The Sphere Packing Bound via Augustin's Method

A sphere packing bound (SPB) with a prefactor that is polynomial in the block length $n$ is established for codes on a length $n$ product channel $W_{[1,n]}$ assuming that the maximum order $1/2$ Renyi capacity among the component channels, i.e. $\max_{t\in[1,n]} C_{1/2,W_{t}}$, is $\mathit{O}(\ln n)$. The reliability function of the discrete stationary product channels with feedback is bounded from above by the sphere packing exponent. Both results are proved by first establishing a non-asymptotic SPB. The latter result continues to hold under a milder stationarity hypothesis.Comment: 30 pages. An error in the statement of Lemma 2 is corrected. The change is inconsequential for the rest of the pape

Sphere-packing bound for block-codes with feedback and finite memory

A lower bound bound is established on the error probability of fixed-length block-coding systems with finite memory feedback, which can be described in terms of a time dependent finite state machine. It is shown that the reliability function of such coding systems over discrete memoryless channels is upper-bounded by the sphere-packing exponent