Asymptotic Estimates in Information Theory with Non-Vanishing Error Probabilities

This monograph presents a unified treatment of single- and multi-user problems in Shannon's information theory where we depart from the requirement that the error probability decays asymptotically in the blocklength. Instead, the error probabilities for various problems are bounded above by a non-vanishing constant and the spotlight is shone on achievable coding rates as functions of the growing blocklengths. This represents the study of asymptotic estimates with non-vanishing error probabilities. In Part I, after reviewing the fundamentals of information theory, we discuss Strassen's seminal result for binary hypothesis testing where the type-I error probability is non-vanishing and the rate of decay of the type-II error probability with growing number of independent observations is characterized. In Part II, we use this basic hypothesis testing result to develop second- and sometimes, even third-order asymptotic expansions for point-to-point communication. Finally in Part III, we consider network information theory problems for which the second-order asymptotics are known. These problems include some classes of channels with random state, the multiple-encoder distributed lossless source coding (Slepian-Wolf) problem and special cases of the Gaussian interference and multiple-access channels. Finally, we discuss avenues for further research.Comment: Further comments welcom

Ultra-Reliable Short-Packet Communications: Fundamental Limits and Enabling Technologies

The paradigm shift from 4G to 5G communications, anticipated to enable ultra-reliable low-latency communications (URLLC), will enforce a radical change in the design of wireless communication systems. Unlike in 4G systems, where the main objective is to provide a large transmission rate, in URLLC, as implied by its name, the objective is to enable transmissions with low latency and, simultaneously, very high reliability. Since low latency implies the use of short data packets, the tension between blocklength and reliability is studied in URLLC.Several key enablers for URLLC communications have been designated in the literature. Of special importance are diversity-enabling technologies such as multiantenna systems and feedback protocols. Furthermore, it is not only important to introduce additional diversity by means of the above examples, one must also guarantee that thescarce number of channel uses are used in an optimal way. Therefore, it is imperative to develop design guidelines for how to enable reliable detection of incoming data, how to acquire channel-state information, and how to construct efficient short-packet channel codes. The development of such guidelines is at the heart of this thesis. This thesis focuses on the fundamental performance of URLLC-enabling technologies. Specifically, we provide converse (upper) bounds and achievability (lower) bounds on the maximum coding rate, based on finite-blocklength information theory, for systems that employ the key enablers outlined above. With focus on the wireless channel, modeled via a block-fading assumption, we are able to provide answers to questions like: howto optimally utilize spatial and frequency diversity, how far from optimal short-packet channel codes perform, how multiantenna systems should be designed to serve a given number of users, and how to design feedback schemes when the feedback link is noisy. In particular, this thesis is comprised out of four papers. In Paper A, we study the short-packet performance over the Rician block-fading channel. In particular, we present achievability bounds for pilot-assisted transmission with several different decoders that allow us to quantify the impact, on the achievable performance, of imposed pilots and mismatched decoding. Furthermore, we design short-packet channel codes that perform within 1 dB of our achievability bounds. Paper B studies multiuser massive multiple-input multiple-output systems with short packets. We provide an achievability bound on the average error probability over quasistatic spatially correlated Rayleigh-fading channels. The bound applies to arbitrary multiuser settings, pilot-assisted transmission, and mismatched decoding. This makes it suitable to assess the performance in the uplink/downlink for arbitrary linear signal processing. We show that several lessons learned from infinite-blocklength analyses carry over to the finite-blocklength regime. Furthermore, for the multicell setting with randomly placed users, pilot contamination should be avoided at all cost and minimum mean-squared error signal processing should be used to comply with the stringent requirements of URLLC.In Paper C, we consider sporadic transmissions where the task of the receiver is to both detect and decode an incoming packet. Two novel achievability bounds, and a novel converse bound are presented for joint detection-decoding strategies. It is shown that errors associated with detection deteriorates performance significantly for very short packet sizes. Numerical results also indicate that separate detection-decoding strategies are strictly suboptimal over block-fading channels.Finally, in Paper D, variable-length codes with noisy stop-feedback are studied via a novel achievability bound on the average service time and the average error probability. We use the bound to shed light on the resource allocation problem between the forward and the feedback channel. For URLLC applications, it is shown that enough resources must be assigned to the feedback link such that a NACK-to-ACK error becomes rarer than the target error probability. Furthermore, we illustrate that the variable-length stop-feedback scheme outperforms state-of-the-art fixed-length no-feedback bounds even when the stop-feedback bit is noisy

Information-Theoretic Foundations of Mismatched Decoding

Shannon's channel coding theorem characterizes the maximal rate of information that can be reliably transmitted over a communication channel when optimal encoding and decoding strategies are used. In many scenarios, however, practical considerations such as channel uncertainty and implementation constraints rule out the use of an optimal decoder. The mismatched decoding problem addresses such scenarios by considering the case that the decoder cannot be optimized, but is instead fixed as part of the problem statement. This problem is not only of direct interest in its own right, but also has close connections with other long-standing theoretical problems in information theory. In this monograph, we survey both classical literature and recent developments on the mismatched decoding problem, with an emphasis on achievable random-coding rates for memoryless channels. We present two widely-considered achievable rates known as the generalized mutual information (GMI) and the LM rate, and overview their derivations and properties. In addition, we survey several improved rates via multi-user coding techniques, as well as recent developments and challenges in establishing upper bounds on the mismatch capacity, and an analogous mismatched encoding problem in rate-distortion theory. Throughout the monograph, we highlight a variety of applications and connections with other prominent information theory problems.Comment: Published in Foundations and Trends in Communications and Information Theory (Volume 17, Issue 2-3

Polar Coding Schemes for Cooperative Transmission Systems

: In this thesis, a serially-concatenated coding scheme with a polar code as the outer code and a low density generator matrix (LDGM) code as the inner code is firstly proposed. It is shown that that the proposed scheme provides a method to improve significantly the low convergence of polar codes and the high error floor of LDGM codes while keeping the advantages of both such as the low encoding and decoding complexity. The bit error rate results show that the proposed scheme by reasonable design have the potential to approach a performance close to the capacity limit and avoid error floor effectively. Secondly, a novel transmission protocol based on polar coding is proposed for the degraded half-duplex relay channel. In the proposed protocol, the relay only needs to forward a part of the decoded source message that the destination needs according to the exquisite nested structure of polar codes. It is proved that the scheme can achieve the capacity of the half-duplex relay channel while enjoying low encoding/decoding complexity. By modeling the practical system, we verify that the proposed scheme outperforms the conventional scheme designed by low-density parity-check codes by simulations. Finally, a generalized partial information relaying protocol is proposed for degraded multiple-relay networks with orthogonal receiver components (MRN-ORCs). In such a protocol, each relay node decodes the received source message with the help of partial information from previous nodes and re-encodes part of the decoded message for transmission to satisfy the decoding requirements for the following relay node or the destination node. For the design of polar codes, the nested structures are constructed based on this protocol and the information sets corresponding to the partial messages forwarded are also calculated. It is proved that the proposed scheme achieves the theoretical capacity of the degraded MRN-ORCs while still retains the low-complexity feature of polar codes

Multiterminal Source-Channel Coding

Cooperative communication is seen as a key concept to achieve ultra-reliable communication in upcoming fifth-generation mobile networks (5G). A promising cooperative communication concept is multiterminal source-channel coding, which attracted recent attention in the research community. This thesis lays theoretical foundations for understanding the performance of multiterminal source-channel codes in a vast variety of cooperative communication networks. To this end, we decouple the multiterminal source-channel code into a multiterminal source code and multiple point-to-point channel codes. This way, we are able to adjust the multiterminal source code to any cooperative communication network without modification of the channel codes. We analyse the performance in terms of the outage probability in two steps: at first, we evaluate the instantaneous performance of the multiterminal source-channel codes for fixed channel realizations; and secondly, we average the instantaneous performance over the fading process. Based on the performance analysis, we evaluate the performance of multiterminal source-channel codes in three cooperative communication networks, namely relay, wireless sensor, and multi-connectivity networks. For all three networks, we identify the corresponding multiterminal source code and analyse its performance by the rate region for binary memoryless sources. Based on the rate region, we derive the outage probability for additive white Gaussian noise channels with quasi-static Rayleigh fading. We find results for the exact outage probability in integral form and closed-form solutions for the asymptotic outage probability at high signal-to-noise ratio. The importance of our results is fourfold: (i) we give the ultimate performance limits of the cooperative communication networks under investigation; (ii) the optimality of practical schemes can be evaluated with respect to our results, (iii) our results are suitable for link-level abstraction which reduces complexity in network-level simulation; and (iv) our results demonstrate that all three cooperative communication networks are key technologies to enable 5G applications, such as device to device and machine to machine communications, internet of things, and internet of vehicles. In addition, we evaluate the performance improvement of multiterminal source-channel codes over other (non-)cooperative communications concepts in terms of the transmit power reduction given a certain outage probability level. Moreover, we compare our theoretical results to simulated frame-error-rates of practical coding schemes. Our results manifest the superiority of multiterminal source-channel codes over other (non-)cooperative communications concepts

An Overview of Physical Layer Security with Finite-Alphabet Signaling

Providing secure communications over the physical layer with the objective of achieving perfect secrecy without requiring a secret key has been receiving growing attention within the past decade. The vast majority of the existing studies in the area of physical layer security focus exclusively on the scenarios where the channel inputs are Gaussian distributed. However, in practice, the signals employed for transmission are drawn from discrete signal constellations such as phase shift keying and quadrature amplitude modulation. Hence, understanding the impact of the finite-alphabet input constraints and designing secure transmission schemes under this assumption is a mandatory step towards a practical implementation of physical layer security. With this motivation, this article reviews recent developments on physical layer security with finite-alphabet inputs. We explore transmit signal design algorithms for single-antenna as well as multi-antenna wiretap channels under different assumptions on the channel state information at the transmitter. Moreover, we present a review of the recent results on secure transmission with discrete signaling for various scenarios including multi-carrier transmission systems, broadcast channels with confidential messages, cognitive multiple access and relay networks. Throughout the article, we stress the important behavioral differences of discrete versus Gaussian inputs in the context of the physical layer security. We also present an overview of practical code construction over Gaussian and fading wiretap channels, and we discuss some open problems and directions for future research.Comment: Submitted to IEEE Communications Surveys & Tutorials (1st Revision