Lecture Notes on Network Information Theory

These lecture notes have been converted to a book titled Network Information Theory published recently by Cambridge University Press. This book provides a significantly expanded exposition of the material in the lecture notes as well as problems and bibliographic notes at the end of each chapter. The authors are currently preparing a set of slides based on the book that will be posted in the second half of 2012. More information about the book can be found at http://www.cambridge.org/9781107008731/. The previous (and obsolete) version of the lecture notes can be found at http://arxiv.org/abs/1001.3404v4/

Correlated Sources In Distributed Networks - Data Transmission, Common Information Characterization and Inferencing

Correlation is often present among observations in a distributed system. This thesis deals with various design issues when correlated data are observed at distributed terminals, including: communicating correlated sources over interference channels, characterizing the common information among dependent random variables, and testing the presence of dependence among observations. It is well known that separated source and channel coding is optimal for point-to-point communication. However, this is not the case for multi-terminal communications. In this thesis, we study the problem of communicating correlated sources over interference channels (IC), for both the lossless and the lossy case. For lossless case, a sufficient condition is found using the technique of random source partition and correlation preserving codeword generation. The sufficient condition reduces to the Han-Kobayashi achievable rate region for IC with independent observations. Moreover, the proposed coding scheme is optimal for transmitting a special correlated sources over a class of deterministic interference channels. We then study the general case of lossy transmission of two correlated sources over a two-user discrete memoryless interference channel (DMIC). An achievable distortion region is obtained and Gaussian examples are studied. The second topic is the generalization of Wyner\u27s definition of common information of a pair of random variables to that of N random variables. Coding theorems are obtained to show that the same operational meanings for the common information of two random variables apply to that of N random variables. We establish a monotone property of Wyner\u27s common information which is in contrast to other notions of the common information, specifically Shannon\u27s mutual information and G\u27{a}cs and K {o}rner\u27s common randomness. Later, we extend Wyner\u27s common information to that of continuous random variables and provide an operational meaning using the Gray-Wyner network with lossy source coding. We show that Wyner\u27s common information equals the smallest common message rate when the total rate is arbitrarily close to the rate-distortion function with joint decoding. Finally, we consider the problem of distributed test of statistical independence under communication constraints. Focusing on the Gaussian case because of its tractability, we study in this thesis the characteristics of optimal scalar quantizers for distributed test of independence where the optimality is both in the finite sample regime and in the asymptotic regime

Adaptive data acquisition for communication networks

In an increasing number of communication systems, such as sensor networks or local area networks within medical, financial or military institutions, nodes communicate information sources (e.g., video, audio) over multiple hops. Moreover, nodes have, or can acquire, correlated information sources from the environment, e.g., from data bases or from measurements. Among the new design problems raised by the outlined scenarios, two key issues are addressed in this dissertation: 1) How to preserve the consistency of sensitive information across multiple hops; 2) How to incorporate the design of actuation in the form of data acquisition and network probing in the optimization of the communication network. These aspects are investigated by using information-theoretic (source and channel coding) models, obtaining fundamental insights that have been corroborated by various illustrative examples. To address point 1), the problem of cascade source coding with side information is investigated. The motivating observation is that, in this class of problems, the estimate of the source obtained at the decoder cannot be generally reproduced at the encoder if it depends directly on the side information. In some applications, such as the one mentioned above, this lack of consistency may be undesirable, and a so called Common Reconstruction (CR) requirement, whereby one imposes that the encoder be able to agree on the decoder’s estimate, may be instead in order. The rate-distortion region is here derived for some special cases of the cascade source coding problem and of the related Heegard-Berger (HB) problem under the CR constraint. As for point 2), the work is motivated by the fact that, in order to enable, or to facilitate, the exchange of information, nodes of a communication network routinely take various types of actions, such as data acquisition or network probing. For instance, sensor nodes schedule the operation of their sensing devices to measure given physical quantities of interest, and wireless nodes probe the state of the channel via training. The problem of optimal data acquisition is studied for a cascade source coding problem, a distributed source coding problem and a two-way source coding problem assuming that the side information sequences can be controlled via the selection of cost-constrained actions. It is shown that a joint design of the description of the source and of the control signals used to guide the selection of the actions at downstream nodes is generally necessary for an efficient use of the available communication links. Instead, the problem of optimal channel probing is studied for a broadcast channel and a point-to-point link in which the decoder is interested in estimating not only the message, but also the state sequence. Finally, the problem of embedding information on the actions is studied for both the source and the channel coding set-ups described above

The Road to Next-Generation Multiple Access: A 50-Year Tutorial Review

The evolution of wireless communications has been significantly influenced by remarkable advancements in multiple access (MA) technologies over the past five decades, shaping the landscape of modern connectivity. Within this context, a comprehensive tutorial review is presented, focusing on representative MA techniques developed over the past 50 years. The following areas are explored: i) The foundational principles and information-theoretic capacity limits of power-domain non-orthogonal multiple access (NOMA) are characterized, along with its extension to multiple-input multiple-output (MIMO)-NOMA. ii) Several MA transmission schemes exploiting the spatial domain are investigated, encompassing both conventional space-division multiple access (SDMA)/MIMO-NOMA systems and near-field MA systems utilizing spherical-wave propagation models. iii) The application of NOMA to integrated sensing and communications (ISAC) systems is studied. This includes an introduction to typical NOMA-based downlink/uplink ISAC frameworks, followed by an evaluation of their performance limits using a mutual information (MI)-based analytical framework. iv) Major issues and research opportunities associated with the integration of MA with other emerging technologies are identified to facilitate MA in next-generation networks, i.e., next-generation multiple access (NGMA). Throughout the paper, promising directions are highlighted to inspire future research endeavors in the realm of MA and NGMA.Comment: 43 pages, 38 figures; Submitted to Proceedings of the IEE