Outer Bounds on the Admissible Source Region for Broadcast Channels with Correlated Sources

Two outer bounds on the admissible source region for broadcast channels with correlated sources are presented: the first one is strictly tighter than the existing outer bound by Gohari and Anantharam while the second one provides a complete characterization of the admissible source region in the case where the two sources are conditionally independent given the common part. These outer bounds are deduced from the general necessary conditions established for the lossy source broadcast problem via suitable comparisons between the virtual broadcast channel (induced by the source and the reconstructions) and the physical broadcast channel

Lecture Notes on Network Information Theory

These lecture notes have been converted to a book titled Network Information Theory published recently by Cambridge University Press. This book provides a significantly expanded exposition of the material in the lecture notes as well as problems and bibliographic notes at the end of each chapter. The authors are currently preparing a set of slides based on the book that will be posted in the second half of 2012. More information about the book can be found at http://www.cambridge.org/9781107008731/. The previous (and obsolete) version of the lecture notes can be found at http://arxiv.org/abs/1001.3404v4/

Asymptotic Estimates in Information Theory with Non-Vanishing Error Probabilities

This monograph presents a unified treatment of single- and multi-user problems in Shannon's information theory where we depart from the requirement that the error probability decays asymptotically in the blocklength. Instead, the error probabilities for various problems are bounded above by a non-vanishing constant and the spotlight is shone on achievable coding rates as functions of the growing blocklengths. This represents the study of asymptotic estimates with non-vanishing error probabilities. In Part I, after reviewing the fundamentals of information theory, we discuss Strassen's seminal result for binary hypothesis testing where the type-I error probability is non-vanishing and the rate of decay of the type-II error probability with growing number of independent observations is characterized. In Part II, we use this basic hypothesis testing result to develop second- and sometimes, even third-order asymptotic expansions for point-to-point communication. Finally in Part III, we consider network information theory problems for which the second-order asymptotics are known. These problems include some classes of channels with random state, the multiple-encoder distributed lossless source coding (Slepian-Wolf) problem and special cases of the Gaussian interference and multiple-access channels. Finally, we discuss avenues for further research.Comment: Further comments welcom

Multiple Description Quantization via Gram-Schmidt Orthogonalization

The multiple description (MD) problem has received considerable attention as a model of information transmission over unreliable channels. A general framework for designing efficient multiple description quantization schemes is proposed in this paper. We provide a systematic treatment of the El Gamal-Cover (EGC) achievable MD rate-distortion region, and show that any point in the EGC region can be achieved via a successive quantization scheme along with quantization splitting. For the quadratic Gaussian case, the proposed scheme has an intrinsic connection with the Gram-Schmidt orthogonalization, which implies that the whole Gaussian MD rate-distortion region is achievable with a sequential dithered lattice-based quantization scheme as the dimension of the (optimal) lattice quantizers becomes large. Moreover, this scheme is shown to be universal for all i.i.d. smooth sources with performance no worse than that for an i.i.d. Gaussian source with the same variance and asymptotically optimal at high resolution. A class of low-complexity MD scalar quantizers in the proposed general framework also is constructed and is illustrated geometrically; the performance is analyzed in the high resolution regime, which exhibits a noticeable improvement over the existing MD scalar quantization schemes.Comment: 48 pages; submitted to IEEE Transactions on Information Theor

Correlated Sources In Distributed Networks - Data Transmission, Common Information Characterization and Inferencing

Correlation is often present among observations in a distributed system. This thesis deals with various design issues when correlated data are observed at distributed terminals, including: communicating correlated sources over interference channels, characterizing the common information among dependent random variables, and testing the presence of dependence among observations. It is well known that separated source and channel coding is optimal for point-to-point communication. However, this is not the case for multi-terminal communications. In this thesis, we study the problem of communicating correlated sources over interference channels (IC), for both the lossless and the lossy case. For lossless case, a sufficient condition is found using the technique of random source partition and correlation preserving codeword generation. The sufficient condition reduces to the Han-Kobayashi achievable rate region for IC with independent observations. Moreover, the proposed coding scheme is optimal for transmitting a special correlated sources over a class of deterministic interference channels. We then study the general case of lossy transmission of two correlated sources over a two-user discrete memoryless interference channel (DMIC). An achievable distortion region is obtained and Gaussian examples are studied. The second topic is the generalization of Wyner\u27s definition of common information of a pair of random variables to that of N random variables. Coding theorems are obtained to show that the same operational meanings for the common information of two random variables apply to that of N random variables. We establish a monotone property of Wyner\u27s common information which is in contrast to other notions of the common information, specifically Shannon\u27s mutual information and G\u27{a}cs and K {o}rner\u27s common randomness. Later, we extend Wyner\u27s common information to that of continuous random variables and provide an operational meaning using the Gray-Wyner network with lossy source coding. We show that Wyner\u27s common information equals the smallest common message rate when the total rate is arbitrarily close to the rate-distortion function with joint decoding. Finally, we consider the problem of distributed test of statistical independence under communication constraints. Focusing on the Gaussian case because of its tractability, we study in this thesis the characteristics of optimal scalar quantizers for distributed test of independence where the optimality is both in the finite sample regime and in the asymptotic regime

Information-theoretic Secrecy in Multi-user Channels

Inherent openness of the wireless medium imposes stronger challenges on the security of wireless communications. Information-theoretic security addresses these challenges at the physical layer by using tools from wireless communication theory, signal processing and information theory. In information-theoretic security, physical layer communication is intelligently designed to exploit the characteristics of the wireless medium, such as fading, interference, cooperation, and multi-dimensional signaling, in order to provide or improve security. In this dissertation, we study the security of several fundamental wireless network configurations from an information-theoretic perspective. First, we study the Gaussian multiple-input multiple-output (MIMO) wiretap channel. In this channel, the transmitter sends a common message to both the legitimate user and the eavesdropper. In addition to the common message, a private message is sent only to the legitimate user, which needs to be kept hidden as much as possible from the eavesdropper. We obtain the entire capacity-equivocation region for this channel model. In particular, we show the sufficiency of jointly Gaussian auxiliary random variables and channel input to evaluate the existing single-letter description of the capacity-equivocation region due to Csiszar-Korner. Next, we study the secure broadcasting problem, where a transmitter wants to have secure communication with multiple legitimate users in the presence of an external eavesdropper. We study several special cases of the secure broadcasting problem. First, we consider the degraded multi-receiver wiretap channel, and establish its secrecy capacity region. Second, we consider the parallel less noisy multi-receiver wiretap channel, and obtain its common message secrecy capacity and sum secrecy capacity. Third, we consider the parallel degraded multi-receiver wiretap channel for the two-user and two-sub-channel case, and obtain its entire secrecy capacity region. Finally, we consider a parallel channel model with two sub-channels, where the transmitter can use only one of the subchannels at any time, and characterize its secrecy capacity region. Then, we study the two-user Gaussian MIMO broadcast channel with common and confidential messages. In this channel model, the transmitter sends a common message to both users, and a confidential message to each user which needs to be kept perfectly secret from the other user. We obtain the entire capacity region of this channel. We also explore the connections between this channel model and its non-confidential counterpart, i.e., the Gaussian MIMO broadcast channel with common and private message. Next, we consider the Gaussian MIMO multi-receiver wiretap channel and obtain its secrecy capacity region for the most general case. We first show that even for the single-input single-output (SISO) case, existing converse techniques fall short of proving the secrecy capacity region, to emphasize the need for a new proof technique, which we develop by using the relationships between the Fisher information and the differential entropy. Using this new proof technique, we obtain the secrecy capacity region of the degraded MIMO channel. We then establish the secrecy capacity region of the general MIMO channel by using the channel enhancement technique in conjunction with the capacity result we obtained for the degraded MIMO channel. For the general MIMO channel, we show that dirty-paper coding (DPC) combined with stochastic encoding attains the entire secrecy capacity region. Then, we study the multi-receiver wiretap channel for a more general scenario, where, in addition to confidential messages, the transmitter sends public messages to the legitimate users, on which there are no secrecy constraints. First, we consider the degraded discrete memoryless channel, and obtain inner and outer bounds for the capacity region. These inner and outer bounds match for certain cases, providing the capacity region. Second, we obtain an inner bound for the general discrete memoryless channel by using Marton's inner bound. Third, we consider the degraded Gaussian MIMO channel, and show that jointly Gaussian auxiliary random variables and channel input are sufficient to exhaust the inner and outer bounds. Finally, we provide an inner bound for the capacity region of the general Gaussian MIMO channel. Next, we focus on the multiple access wiretap (MAC-WT) channel whose capacity region is unknown. We consider a special class of MAC-WT channels which we call the weak eavesdropper class, where each user's link to the legitimate receiver is stronger than its link to the eavesdropper. For this class of channels, we develop an outer bound for the secrecy capacity region, which partially matches the achievable region in an n-letter form. We evaluate a looser version of our outer bound for the Gaussian case, and show that our outer bound is within 0.5 bits/channel use of the achievable rates along the individual secrecy rates for all weak eavesdropper Gaussian MAC-WT. Then, we investigate the effects of user cooperation on the secrecy of broadcast channels by considering the cooperative relay broadcast channel (CRBC). We propose an achievable scheme that combines Marton's coding scheme for broadcast channels and Cover and El Gamal's compress-and-forward (CAF) scheme for relay channels. For the Gaussian CRBC, we show that both users can have positive secrecy rates, which is not possible for scalar Gaussian broadcast channels without cooperation. We further investigate the effects of user cooperation on secrecy by considering the multiple access channel with generalized feedback (MAC-GF), which can be viewed as the MAC-dual of the CRBC. We propose a CAF-based achievable secrecy rate region for the MAC-GF. Specializing our results to a Gaussian MAC-GF, we present numerical results which demonstrate that cooperation can improve secrecy for the MAC-GF. Next, we study the two-user one-eavesdropper discrete memoryless compound wiretap channel, and provide the best known lower bound for the secrecy capacity of this compound channel. We evaluate this achievable secrecy rate for the Gaussian MIMO case by using DPC. We show that this achievable secrecy rate achieves at least half of the secrecy capacity of this Gaussian MIMO compound wiretap channel, and also attains the secrecy capacity when the eavesdropper is degraded with respect to one of the two users. Then, we study the degraded compound multi-receiver wiretap channel (DCMRWC), which, in addition to a group of eavesdroppers, has two groups of users, namely the stronger group and the weaker group. We study two different communication scenarios for this channel. In the first scenario, there is only one eavesdropper, and the transmitter sends a confidential message to each group of legitimate users while keeping both messages secret from the eavesdropper. In the second scenario, we study the DCMRWC with layered messages without any restriction on the number of eavesdroppers. For both scenarios, we obtain the secrecy capacity region for the discrete memoryless channel, the parallel channel, and the Gaussian parallel channel. For the Gaussian MIMO channel, we obtain the secrecy capacity region when there is only one user in the second group. Next, we study the two-user fading broadcast channel and obtain its ergodic secrecy capacity region. We show that, thanks to fading, both users can have simultaneous secure communication with the transmitter, although this is not possible in the scalar non-fading Gaussian broadcast channel where only one user can have secure communication. This simultaneous secrecy of both users is achieved by an opportunistic communication scheme, in which, at each time instant, the transmitter communicates with the user having a better channel gain. Then, we study the secure lossy transmission of a vector Gaussian source to a legitimate user in the presence of an eavesdropper, where both the legitimate user and the eavesdropper have vector Gaussian side information. We obtain an outer bound for the rate, equivocation and distortion region. Moreover, we obtain the maximum equivocation at the eavesdropper when there is no constraint on the transmission rate. By using this maximum equivocation result, we show two facts. First, for this problem, in general, Wyner-Ziv scheme is suboptimal, although, it is optimal in the absence of an eavesdropper. And, second, even when there is no transmission rate constraint, an uncoded transmission scheme is suboptimal; the presence of an eavesdropper necessitates the use of a coded scheme to attain the maximum equivocation. Finally, we revisit the secure lossy source coding problem. In all works on this problem, either the equivocation of the source at the eavesdropper or the equivocation of the legitimate user's reconstruction of the source at the eavesdropper is used to measure secrecy. We first propose the relative equivocation of the source at the eavesdropper with respect to the legitimate user as a new secrecy measure. We argue that this new secrecy measure is the one that corresponds to the natural generalization of the equivocation in a wiretap channel to the context of secure lossy source coding. Under this new secrecy measure, we provide a single-letter description of the rate, relative equivocation and distortion region, as well as its specializations to degraded and reversely degraded cases. We investigate the relationships between the optimal scheme that attains this region and the Wyner-Ziv scheme