Lecture Notes on Network Information Theory

These lecture notes have been converted to a book titled Network Information Theory published recently by Cambridge University Press. This book provides a significantly expanded exposition of the material in the lecture notes as well as problems and bibliographic notes at the end of each chapter. The authors are currently preparing a set of slides based on the book that will be posted in the second half of 2012. More information about the book can be found at http://www.cambridge.org/9781107008731/. The previous (and obsolete) version of the lecture notes can be found at http://arxiv.org/abs/1001.3404v4/

Optimal Power Allocation for a Successive Refinable Source with Multiple Descriptions over a Fading Relay Channel Using Broadcast/Multicast Strategies

In a wireless fading relay system with multicast/broadcast transmission, one of the most crucial challenges is the optimization of a transmission rate under multiuser channel diversity. Previously reported solutions for mitigating the vicious effect due to multi-user channel diversity have been mainly based on superposition coded multicast, where an optimal power allocation to each layer of modulated signals is determined. Many previous studies investigated a harmonic interplay between the successively re nable (SR) content source and a layered modulation via superposition coding (SPC) over the multicast/broadcast channels. By jointly considering the successive re nement characteristic at the source and the dependency of the layered modulation at the channel, a graceful fexibility can be achieved on a group of users with di erent channel realizations. Here most of the receivers are supposed to obtain the base quality layer information modulated in a lower rate, while the receivers with better channel realizations will obtain more information by re ning the base quality layer information using the enhancement quality layer information. In particular, the optimal power allocation for a SR source over a fading relay channel using broadcast/multicast strategy can be determined such that the minimum distortion of total received information is produced. However, a quality layer of data in a successively refined source may not be decodable if there is any loss of channel codewords, even if the corresponding longterm channel realization is su cient for decoding. To overcome this problem, one of the previous studies introduced a framework of coded video multicast, where multiple description coding (MDC) is applied to an SR content source and is further mapped into a layered modulation via SPC at the channel. Up till now, there has not been a rigorous proof provided on the bene t of manipulating the two coding techniques, (i.e. MDC and SPC), nor has any systematic optimization approach been developed for quantifying the parameter selection. Cooperative relaying in wireless networks has recently received much attention. Because the received signal can be severely degraded due to fading in wireless communications, time, frequency and spatial diversity techniques are introduced to overcome fading. Spatial diversity is typically envisioned as having multiple transmit and/or receive antennas. Cooperation can be used here to provide higher rates and results in a more robust system. Recently proposed cooperation schemes, which take into account the practical constraint that the relay cannot transmit and receive at the same time, include amplify-forward(AF), decode-forward(DF), and compress-forward(CF). In this study, in a fading relay scenario, a proposed framework is investigated to tackle the task of layered power allocation, where an in-depth study is conducted on achieving an optimal power allocation in SPC, such that the information distortion perceived at the users can be minimized. This thesis provides a comprehensive formulation on the information distortion at the receivers and a suite of solution approaches for the developed optimization problem by jointly considering MDC and SPC parameter selection over the fading relay channel

Side information aware source and channel coding in wireless networks

Signals in communication networks exhibit significant correlation, which can stem from the physical nature of the underlying sources, or can be created within the system. Current layered network architectures, in which, based on Shannon’s separation theorem, data is compressed and transmitted over independent bit-pipes, are in general not able to exploit such correlation efficiently. Moreover, this strictly layered architecture was developed for wired networks and ignore the broadcast and highly dynamic nature of the wireless medium, creating a bottleneck in the wireless network design. Technologies that exploit correlated information and go beyond the layered network architecture can become a key feature of future wireless networks, as information theory promises significant gains. In this thesis, we study from an information theoretic perspective, three distinct, yet fundamental, problems involving the availability of correlated information in wireless networks and develop novel communication techniques to exploit it efficiently. We first look at two joint source-channel coding problems involving the lossy transmission of Gaussian sources in a multi-terminal and a time-varying setting in which correlated side information is present in the network. In these two problems, the optimality of Shannon’s separation breaks down and separate source and channel coding is shown to perform poorly compared to the proposed joint source-channel coding designs, which are shown to achieve the optimal performance in some setups. Then, we characterize the capacity of a class of orthogonal relay channels in the presence of channel side information at the destination, and show that joint decoding and compression of the received signal at the relay is required to optimally exploit the available side information. Our results in these three different scenarios emphasize the benefits of exploiting correlated side information at the destination when designing a communication system, even though the nature of the side information and the performance measure in the three scenarios are quite different.Open Acces

Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications

Communication systems to date primarily aim at reliably communicating bit sequences. Such an approach provides efficient engineering designs that are agnostic to the meanings of the messages or to the goal that the message exchange aims to achieve. Next generation systems, however, can be potentially enriched by folding message semantics and goals of communication into their design. Further, these systems can be made cognizant of the context in which communication exchange takes place, providing avenues for novel design insights. This tutorial summarizes the efforts to date, starting from its early adaptations, semantic-aware and task-oriented communications, covering the foundations, algorithms and potential implementations. The focus is on approaches that utilize information theory to provide the foundations, as well as the significant role of learning in semantics and task-aware communications.Comment: 28 pages, 14 figure

Information-theoretic Secrecy in Multi-user Channels

Inherent openness of the wireless medium imposes stronger challenges on the security of wireless communications. Information-theoretic security addresses these challenges at the physical layer by using tools from wireless communication theory, signal processing and information theory. In information-theoretic security, physical layer communication is intelligently designed to exploit the characteristics of the wireless medium, such as fading, interference, cooperation, and multi-dimensional signaling, in order to provide or improve security. In this dissertation, we study the security of several fundamental wireless network configurations from an information-theoretic perspective. First, we study the Gaussian multiple-input multiple-output (MIMO) wiretap channel. In this channel, the transmitter sends a common message to both the legitimate user and the eavesdropper. In addition to the common message, a private message is sent only to the legitimate user, which needs to be kept hidden as much as possible from the eavesdropper. We obtain the entire capacity-equivocation region for this channel model. In particular, we show the sufficiency of jointly Gaussian auxiliary random variables and channel input to evaluate the existing single-letter description of the capacity-equivocation region due to Csiszar-Korner. Next, we study the secure broadcasting problem, where a transmitter wants to have secure communication with multiple legitimate users in the presence of an external eavesdropper. We study several special cases of the secure broadcasting problem. First, we consider the degraded multi-receiver wiretap channel, and establish its secrecy capacity region. Second, we consider the parallel less noisy multi-receiver wiretap channel, and obtain its common message secrecy capacity and sum secrecy capacity. Third, we consider the parallel degraded multi-receiver wiretap channel for the two-user and two-sub-channel case, and obtain its entire secrecy capacity region. Finally, we consider a parallel channel model with two sub-channels, where the transmitter can use only one of the subchannels at any time, and characterize its secrecy capacity region. Then, we study the two-user Gaussian MIMO broadcast channel with common and confidential messages. In this channel model, the transmitter sends a common message to both users, and a confidential message to each user which needs to be kept perfectly secret from the other user. We obtain the entire capacity region of this channel. We also explore the connections between this channel model and its non-confidential counterpart, i.e., the Gaussian MIMO broadcast channel with common and private message. Next, we consider the Gaussian MIMO multi-receiver wiretap channel and obtain its secrecy capacity region for the most general case. We first show that even for the single-input single-output (SISO) case, existing converse techniques fall short of proving the secrecy capacity region, to emphasize the need for a new proof technique, which we develop by using the relationships between the Fisher information and the differential entropy. Using this new proof technique, we obtain the secrecy capacity region of the degraded MIMO channel. We then establish the secrecy capacity region of the general MIMO channel by using the channel enhancement technique in conjunction with the capacity result we obtained for the degraded MIMO channel. For the general MIMO channel, we show that dirty-paper coding (DPC) combined with stochastic encoding attains the entire secrecy capacity region. Then, we study the multi-receiver wiretap channel for a more general scenario, where, in addition to confidential messages, the transmitter sends public messages to the legitimate users, on which there are no secrecy constraints. First, we consider the degraded discrete memoryless channel, and obtain inner and outer bounds for the capacity region. These inner and outer bounds match for certain cases, providing the capacity region. Second, we obtain an inner bound for the general discrete memoryless channel by using Marton's inner bound. Third, we consider the degraded Gaussian MIMO channel, and show that jointly Gaussian auxiliary random variables and channel input are sufficient to exhaust the inner and outer bounds. Finally, we provide an inner bound for the capacity region of the general Gaussian MIMO channel. Next, we focus on the multiple access wiretap (MAC-WT) channel whose capacity region is unknown. We consider a special class of MAC-WT channels which we call the weak eavesdropper class, where each user's link to the legitimate receiver is stronger than its link to the eavesdropper. For this class of channels, we develop an outer bound for the secrecy capacity region, which partially matches the achievable region in an n-letter form. We evaluate a looser version of our outer bound for the Gaussian case, and show that our outer bound is within 0.5 bits/channel use of the achievable rates along the individual secrecy rates for all weak eavesdropper Gaussian MAC-WT. Then, we investigate the effects of user cooperation on the secrecy of broadcast channels by considering the cooperative relay broadcast channel (CRBC). We propose an achievable scheme that combines Marton's coding scheme for broadcast channels and Cover and El Gamal's compress-and-forward (CAF) scheme for relay channels. For the Gaussian CRBC, we show that both users can have positive secrecy rates, which is not possible for scalar Gaussian broadcast channels without cooperation. We further investigate the effects of user cooperation on secrecy by considering the multiple access channel with generalized feedback (MAC-GF), which can be viewed as the MAC-dual of the CRBC. We propose a CAF-based achievable secrecy rate region for the MAC-GF. Specializing our results to a Gaussian MAC-GF, we present numerical results which demonstrate that cooperation can improve secrecy for the MAC-GF. Next, we study the two-user one-eavesdropper discrete memoryless compound wiretap channel, and provide the best known lower bound for the secrecy capacity of this compound channel. We evaluate this achievable secrecy rate for the Gaussian MIMO case by using DPC. We show that this achievable secrecy rate achieves at least half of the secrecy capacity of this Gaussian MIMO compound wiretap channel, and also attains the secrecy capacity when the eavesdropper is degraded with respect to one of the two users. Then, we study the degraded compound multi-receiver wiretap channel (DCMRWC), which, in addition to a group of eavesdroppers, has two groups of users, namely the stronger group and the weaker group. We study two different communication scenarios for this channel. In the first scenario, there is only one eavesdropper, and the transmitter sends a confidential message to each group of legitimate users while keeping both messages secret from the eavesdropper. In the second scenario, we study the DCMRWC with layered messages without any restriction on the number of eavesdroppers. For both scenarios, we obtain the secrecy capacity region for the discrete memoryless channel, the parallel channel, and the Gaussian parallel channel. For the Gaussian MIMO channel, we obtain the secrecy capacity region when there is only one user in the second group. Next, we study the two-user fading broadcast channel and obtain its ergodic secrecy capacity region. We show that, thanks to fading, both users can have simultaneous secure communication with the transmitter, although this is not possible in the scalar non-fading Gaussian broadcast channel where only one user can have secure communication. This simultaneous secrecy of both users is achieved by an opportunistic communication scheme, in which, at each time instant, the transmitter communicates with the user having a better channel gain. Then, we study the secure lossy transmission of a vector Gaussian source to a legitimate user in the presence of an eavesdropper, where both the legitimate user and the eavesdropper have vector Gaussian side information. We obtain an outer bound for the rate, equivocation and distortion region. Moreover, we obtain the maximum equivocation at the eavesdropper when there is no constraint on the transmission rate. By using this maximum equivocation result, we show two facts. First, for this problem, in general, Wyner-Ziv scheme is suboptimal, although, it is optimal in the absence of an eavesdropper. And, second, even when there is no transmission rate constraint, an uncoded transmission scheme is suboptimal; the presence of an eavesdropper necessitates the use of a coded scheme to attain the maximum equivocation. Finally, we revisit the secure lossy source coding problem. In all works on this problem, either the equivocation of the source at the eavesdropper or the equivocation of the legitimate user's reconstruction of the source at the eavesdropper is used to measure secrecy. We first propose the relative equivocation of the source at the eavesdropper with respect to the legitimate user as a new secrecy measure. We argue that this new secrecy measure is the one that corresponds to the natural generalization of the equivocation in a wiretap channel to the context of secure lossy source coding. Under this new secrecy measure, we provide a single-letter description of the rate, relative equivocation and distortion region, as well as its specializations to degraded and reversely degraded cases. We investigate the relationships between the optimal scheme that attains this region and the Wyner-Ziv scheme

Progressively communicating rich telemetry from autonomous underwater vehicles via relays

Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy at the Massachusetts Institute of Technology and the Woods Hole Oceanographic Institution June 2012As analysis of imagery and environmental data plays a greater role in mission construction and execution, there is an increasing need for autonomous marine vehicles to transmit this data to the surface. Without access to the data acquired by a vehicle, surface operators cannot fully understand the state of the mission. Communicating imagery and high-resolution sensor readings to surface observers remains a significant challenge – as a result, current telemetry from free-roaming autonomous marine vehicles remains limited to ‘heartbeat’ status messages, with minimal scientific data available until after recovery. Increasing the challenge, longdistance communication may require relaying data across multiple acoustic hops between vehicles, yet fixed infrastructure is not always appropriate or possible. In this thesis I present an analysis of the unique considerations facing telemetry systems for free-roaming Autonomous Underwater Vehicles (AUVs) used in exploration. These considerations include high-cost vehicle nodes with persistent storage and significant computation capabilities, combined with human surface operators monitoring each node. I then propose mechanisms for interactive, progressive communication of data across multiple acoustic hops. These mechanisms include wavelet-based embedded coding methods, and a novel image compression scheme based on texture classification and synthesis. The specific characteristics of underwater communication channels, including high latency, intermittent communication, the lack of instantaneous end-to-end connectivity, and a broadcast medium, inform these proposals. Human feedback is incorporated by allowing operators to identify segments of data thatwarrant higher quality refinement, ensuring efficient use of limited throughput. I then analyze the performance of these mechanisms relative to current practices. Finally, I present CAPTURE, a telemetry architecture that builds on this analysis. CAPTURE draws on advances in compression and delay tolerant networking to enable progressive transmission of scientific data, including imagery, across multiple acoustic hops. In concert with a physical layer, CAPTURE provides an endto- end networking solution for communicating science data from autonomous marine vehicles. Automatically selected imagery, sonar, and time-series sensor data are progressively transmitted across multiple hops to surface operators. Human operators can request arbitrarily high-quality refinement of any resource, up to an error-free reconstruction. The components of this system are then demonstrated through three field trials in diverse environments on SeaBED, OceanServer and Bluefin AUVs, each in different software architectures.Thanks to the National Science Foundation, and the National Oceanic and Atmospheric Administration for their funding of my education and this work

Constructive Relay based Cooperative Routing in Mobile Ad hoc Networks

PhDMobile Ad hoc networks (MANETs) are flexible networks that transmit packets node-by-node along a route connecting a given source and destination. Frequent link breaks (due to node mobility) and quick exhaustion of energy (due to limited battery capacity) are two major problems impacting on the flexibility of MANETs. Cooperative communication is a key concept for improving the system lifetime and robustness and has attracted considerable attention. As a result, there is much published research concerning how to utilize cooperative communication in a MANET context. In the past few years, most cooperative technologies have focused on lower layer enhancements, such as with the Physical Layer and MAC Layer, and have become very mature. At the Network Layer, although some research has been proposed, issues still remain such as the lack of a systematically designed cooperative routing scheme (including route discovery, route reply, route enhancement and cooperative data forwarding), the use of cooperative communication for mobility resilience, and route selection (jointly considering the energy consumption, energy harvesting potential and link break probability). Driven by the above concerns, a novel Constructive Relay based CooPerative Routing (CRCPR) protocol based on a cross-layer design is proposed in this thesis. In CRCPR, we fi rst modify the traditional hello message format to carry some additional neighbour information. Based on this information, a key aspect of this protocol is to construct one or more small rhombus topologies within the MANET structure, which are stored and maintained in a COoPerative (COP) Table and Relay Table. Next, the route request procedure is re-designed to improve resilience to node mobility with a scheme called Last hop Replacement. Finally, assuming nodes are mostly battery-powered, destination node based route-decision criteria are explored that can consider energy consumption, energy harvesting and link break probability to determine an appropriate route across the MANET. As the hello message format is modi ed to carry additional information, the control overhead is increased. However, in order to improve the control message eficiency, a new generalised hello message broadcasting scheme entitled Adjust Classi ed Hello Scheme is developed, which can be deployed onto every routing protocol employing a hello mechanism. As well as designing a new routing protocol for MANETs, including route discovery, route selection, route reply, route maintenance, route enhancement and cooperative data forwarding, the proposed scheme is implemented within an Opnetbased simulation environment and evaluated under a variety of realistic conditions. The results con rm that CRCPR improves mobility resilience, saves energy via cooperative communication and reduces the control overhead associated with the hello message mechanism.Chinese Scholarship Counci