Asymptotic Estimates in Information Theory with Non-Vanishing Error Probabilities

This monograph presents a unified treatment of single- and multi-user problems in Shannon's information theory where we depart from the requirement that the error probability decays asymptotically in the blocklength. Instead, the error probabilities for various problems are bounded above by a non-vanishing constant and the spotlight is shone on achievable coding rates as functions of the growing blocklengths. This represents the study of asymptotic estimates with non-vanishing error probabilities. In Part I, after reviewing the fundamentals of information theory, we discuss Strassen's seminal result for binary hypothesis testing where the type-I error probability is non-vanishing and the rate of decay of the type-II error probability with growing number of independent observations is characterized. In Part II, we use this basic hypothesis testing result to develop second- and sometimes, even third-order asymptotic expansions for point-to-point communication. Finally in Part III, we consider network information theory problems for which the second-order asymptotics are known. These problems include some classes of channels with random state, the multiple-encoder distributed lossless source coding (Slepian-Wolf) problem and special cases of the Gaussian interference and multiple-access channels. Finally, we discuss avenues for further research.Comment: Further comments welcom

Lossless Source Coding in the Point-to-Point, Multiple Access, and Random Access Scenarios

This paper treats point-to-point, multiple access and random access lossless source coding in the finite-blocklength regime. A random coding technique is developed, and its power in analyzing the third-order coding performance is demonstrated in all three scenarios. Results include a third-order-optimal characterization of the Slepian-Wolf rate region and a proof showing that for dependent sources, the independent encoders used by Slepian-Wolf codes can achieve the same third-order- optimal performance as a single joint encoder. The concept of random access source coding, which generalizes the multiple access scenario to allow for a subset of participating encoders that is unknown a priori to both the encoders and the decoder, is introduced. Contributions include a new definition of the probabilistic model for a random access-discrete multiple source, a general random access source coding scheme that employs a rateless code with sporadic feedback, and an analysis demonstrating via a random coding argument that there exists a deterministic code of the proposed structure that simultaneously achieves the third- order-optimal performance of Slepian-Wolf codes for all possible subsets of encoders

On high-efficiency optical communication and key distribution

We investigate modulation and coding techniques that approach the fundamental limits of communication and key distribution over optical channels, in the regime of simultaneously high photon and bandwidth efficiencies. First, we develop a simple and robust system design for free-space optical communication that incorporates pulse-position modulation (PPM) over multiple spatial degrees of freedom in order to achieve high photon and spectral efficiency. Further, in the context of key distribution, we determine the optimal rate using a Poisson source of entangled photon pairs and photon detectors, and show how to approach it using PPM parsing of the detected photon stream.United States. Defense Advanced Research Projects Agency. Information in a Photon Program (Contract HR0011-10-C-0159)United States. Army Research Office (Grant W911NF- 10-1-0416)United States. Air Force Office of Scientific Research (Grant FA9550-11-1-0183

Lossless Source Coding in the Point-to-Point, Multiple Access, and Random Access Scenarios

This paper treats point-to-point, multiple access and random access lossless source coding in the finite-blocklength regime. A random coding technique is developed, and its power in analyzing the third-order coding performance is demonstrated in all three scenarios. Via a connection to composite hypothesis testing, a new converse that tightens previously known converses for Slepian-Wolf source coding is established. Asymptotic results include a third-order characterization of the Slepian-Wolf rate region and a proof showing that for dependent sources, the independent encoders used by Slepian-Wolf codes can achieve the same third-order-optimal performance as a single joint encoder. The concept of random access source coding, which generalizes the multiple access scenario to allow for a subset of participating encoders that is unknown a priori to both the encoders and the decoder, is introduced. Contributions include a new definition of the probabilistic model for a random access source, a general random access source coding scheme that employs a rateless code with sporadic feedback, and an analysis demonstrating via a random coding argument that there exists a deterministic code of the proposed structure that simultaneously achieves the third-order-optimal performance of Slepian-Wolf codes for all possible subsets of encoders.Comment: 42 pages, 10 figures. Part of this work was presented at ISIT'1

Protograph LDPC Based Distributed Joint Source Channel Coding

该文提出一种基于原模图低密度奇偶校验(P-LDPC)码的分布式联合信源信道编译码系统方案。该方案编码端,分布式信源发送部分信息位及校验位以同时实; 现压缩及纠错功能;译码端,联合迭代信源信道译码的运用进一步发掘信源的相关性以获得额外的编码增益。此外,论文研究了所提方案在译码端未知相关性系数的; 译码算法。仿真结果表明,所提出的基于P-LDPC码的分布式联合信源信道编译码方案在外部迭代次数不大的情况可以获得较大的性能增益,并且相关性系数在; 译码端已知和未知系统性能基本相当。This paper proposes a Distributed Joint Source-Channel Coding (DJSCC); scheme using Protograph Low Density Parity Check (P-LDPC) code. In the; proposed scheme, the distributed source encoder sends some information; bits together with the parity bits to simultaneously achieve both; distributed compression and channel error correction. Iterative joint; decoding is introduced to further exploit the source correlation.; Moreover, the proposed scheme is investigated when the correlation; between sources is not known at the decoder. Simulation results indicate; that the proposed DJSCC scheme can obtain relatively large additional; coding gains at a relatively small number of global iterations, and the; performance for unknown correlated sources is almost the same as that; for known correlated sources since correlation can be estimated jointly; with the iterative decoding process.福建省自然科学基金; 国家自然科学基