Asymptotic Estimates in Information Theory with Non-Vanishing Error Probabilities

This monograph presents a unified treatment of single- and multi-user problems in Shannon's information theory where we depart from the requirement that the error probability decays asymptotically in the blocklength. Instead, the error probabilities for various problems are bounded above by a non-vanishing constant and the spotlight is shone on achievable coding rates as functions of the growing blocklengths. This represents the study of asymptotic estimates with non-vanishing error probabilities. In Part I, after reviewing the fundamentals of information theory, we discuss Strassen's seminal result for binary hypothesis testing where the type-I error probability is non-vanishing and the rate of decay of the type-II error probability with growing number of independent observations is characterized. In Part II, we use this basic hypothesis testing result to develop second- and sometimes, even third-order asymptotic expansions for point-to-point communication. Finally in Part III, we consider network information theory problems for which the second-order asymptotics are known. These problems include some classes of channels with random state, the multiple-encoder distributed lossless source coding (Slepian-Wolf) problem and special cases of the Gaussian interference and multiple-access channels. Finally, we discuss avenues for further research.Comment: Further comments welcom

The mother of all protocols: Restructuring quantum information's family tree

We give a simple, direct proof of the "mother" protocol of quantum information theory. In this new formulation, it is easy to see that the mother, or rather her generalization to the fully quantum Slepian-Wolf protocol, simultaneously accomplishes two goals: quantum communication-assisted entanglement distillation, and state transfer from the sender to the receiver. As a result, in addition to her other "children," the mother protocol generates the state merging primitive of Horodecki, Oppenheim and Winter, a fully quantum reverse Shannon theorem, and a new class of distributed compression protocols for correlated quantum sources which are optimal for sources described by separable density operators. Moreover, the mother protocol described here is easily transformed into the so-called "father" protocol whose children provide the quantum capacity and the entanglement-assisted capacity of a quantum channel, demonstrating that the division of single-sender/single-receiver protocols into two families was unnecessary: all protocols in the family are children of the mother.Comment: 25 pages, 6 figure

Transforming Monitoring Structures with Resilient Encoders. Application to Repeated Games

An important feature of a dynamic game is its monitoring structure namely, what the players effectively see from the played actions. We consider games with arbitrary monitoring structures. One of the purposes of this paper is to know to what extent an encoder, who perfectly observes the played actions and sends a complementary public signal to the players, can establish perfect monitoring for all the players. To reach this goal, the main technical problem to be solved at the encoder is to design a source encoder which compresses the action profile in the most concise manner possible. A special feature of this encoder is that the multi-dimensional signal (namely, the action profiles) to be encoded is assumed to comprise a component whose probability distribution is not known to the encoder and the decoder has a side information (the private signals received by the players when the encoder is off). This new framework appears to be both of game-theoretical and information-theoretical interest. In particular, it is useful for designing certain types of encoders that are resilient to single deviations and provide an equilibrium utility region in the proposed setting; it provides a new type of constraints to compress an information source (i.e., a random variable). Regarding the first aspect, we apply the derived result to the repeated prisoner's dilemma.Comment: Springer, Dynamic Games and Applications, 201

Slepian-Wolf Coding Over Cooperative Relay Networks

This paper deals with the problem of multicasting a set of discrete memoryless correlated sources (DMCS) over a cooperative relay network. Necessary conditions with cut-set interpretation are presented. A \emph{Joint source-Wyner-Ziv encoding/sliding window decoding} scheme is proposed, in which decoding at each receiver is done with respect to an ordered partition of other nodes. For each ordered partition a set of feasibility constraints is derived. Then, utilizing the sub-modular property of the entropy function and a novel geometrical approach, the results of different ordered partitions are consolidated, which lead to sufficient conditions for our problem. The proposed scheme achieves operational separation between source coding and channel coding. It is shown that sufficient conditions are indeed necessary conditions in two special cooperative networks, namely, Aref network and finite-field deterministic network. Also, in Gaussian cooperative networks, it is shown that reliable transmission of all DMCS whose Slepian-Wolf region intersects the cut-set bound region within a constant number of bits, is feasible. In particular, all results of the paper are specialized to obtain an achievable rate region for cooperative relay networks which includes relay networks and two-way relay networks.Comment: IEEE Transactions on Information Theory, accepte