The Capacity of Channels with Feedback

We introduce a general framework for treating channels with memory and feedback. First, we generalize Massey's concept of directed information and use it to characterize the feedback capacity of general channels. Second, we present coding results for Markov channels. This requires determining appropriate sufficient statistics at the encoder and decoder. Third, a dynamic programming framework for computing the capacity of Markov channels is presented. Fourth, it is shown that the average cost optimality equation (ACOE) can be viewed as an implicit single-letter characterization of the capacity. Fifth, scenarios with simple sufficient statistics are described

Capacity of a POST Channel with and without Feedback

We consider finite state channels where the state of the channel is its previous output. We refer to these as POST (Previous Output is the STate) channels. We first focus on POST($\alpha$) channels. These channels have binary inputs and outputs, where the state determines if the channel behaves as a $Z$ or an $S$ channel, both with parameter $\alpha$. %with parameter $\alpha.$ We show that the non feedback capacity of the POST($\alpha$) channel equals its feedback capacity, despite the memory of the channel. The proof of this surprising result is based on showing that the induced output distribution, when maximizing the directed information in the presence of feedback, can also be achieved by an input distribution that does not utilize of the feedback. We show that this is a sufficient condition for the feedback capacity to equal the non feedback capacity for any finite state channel. We show that the result carries over from the POST($\alpha$) channel to a binary POST channel where the previous output determines whether the current channel will be binary with parameters $(a,b)$ or $(b,a)$. Finally, we show that, in general, feedback may increase the capacity of a POST channel