Coding for Relay Networks with Parallel Gaussian Channels

A wireless relay network consists of multiple source nodes, multiple destination nodes, and possibly many relay nodes in between to facilitate its transmission. It is clear that the performance of such networks highly depends on information for- warding strategies adopted at the relay nodes. This dissertation studies a particular information forwarding strategy called compute-and-forward. Compute-and-forward is a novel paradigm that tries to incorporate the idea of network coding within the physical layer and hence is often referred to as physical layer network coding. The main idea is to exploit the superposition nature of the wireless medium to directly compute or decode functions of transmitted signals at intermediate relays in a net- work. Thus, the coding performed at the physical layer serves the purpose of error correction as well as permits recovery of functions of transmitted signals. For the bidirectional relaying problem with Gaussian channels, it has been shown by Wilson et al. and Nam et al. that the compute-and-forward paradigm is asymptotically optimal and achieves the capacity region to within 1 bit; however, similar results beyond the memoryless case are still lacking. This is mainly because channels with memory would destroy the lattice structure that is most crucial for the compute-and-forward paradigm. Hence, how to extend compute-and-forward to such channels has been a challenging issue. This motivates this study of the extension of compute-and-forward to channels with memory, such as inter-symbol interference. The bidirectional relaying problem with parallel Gaussian channels is also studied, which is a relevant model for the Gaussian bidirectional channel with inter-symbol interference and that with multiple-input multiple-output channels. Motivated by the recent success of linear finite-field deterministic model, we first investigate the corresponding deterministic parallel bidirectional relay channel and fully characterize its capacity region. Two compute-and-forward schemes are then proposed for the Gaussian model and the capacity region is approximately characterized to within a constant gap. The design of coding schemes for the compute-and-forward paradigm with low decoding complexity is then considered. Based on the separation-based framework proposed previously by Tunali et al., this study proposes a family of constellations that are suitable for the compute-and-forward paradigm. Moreover, by using Chinese remainder theorem, it is shown that the proposed constellations are isomorphic to product fields and therefore can be put into a multilevel coding framework. This study then proposes multilevel coding for the proposed constellations and uses multistage decoding to further reduce decoding complexity

Information Theoretic Limits of State-dependent Networks

We investigate the information theoretic limits of two types of state-dependent models in this dissertation. These models capture a wide range of wireless communication scenarios where there are interference cognition among transmitters. Hence, information theoretic studies of these models provide useful guidelines for designing new interference cancellation schemes in practical wireless networks. In particular, we first study the two-user state-dependent Gaussian multiple access channel (MAC) with a helper. The channel is corrupted by an additive Gaussian state sequence known to neither the transmitters nor the receiver, but to a helper noncausally, which assists state cancellation at the receiver. Inner and outer bounds on the capacity region are first derived, which improve the state-of-the-art bounds given in the literature. Further comparison of these bounds yields either segments on the capacity region boundary or the full capacity region by considering various regimes of channel parameters. We then study the two-user Gaussian state-dependent Z-interference channel (Z-IC), in which two receivers are corrupted respectively by two correlated states that are noncausally known to transmitters, but unknown to receivers. Three interference regimes are studied, and the capacity region or the sum capacity boundary is characterized either fully or partially under various channel parameters. The impact of the correlation between the states on the cancellation of state and interference as well as the achievability of the capacity is demonstrated via numerical analysis. Finally, we extend our results on the state-dependent Z-IC to the state-dependent regular IC. As both receivers in the regular IC are interfered, more sophisticated achievable schemes are designed. For the very strong regime, the capacity region is achieved by a scheme where the two transmitters implement a cooperative dirty paper coding. For the strong but not very strong regime, the sum-rate capacity is characterized by rate splitting, layered dirty paper coding and successive cancellation. For the weak regime, the sum-rate capacity is achieved via dirty paper coding individually at two receivers as well as treating interference as noise. Numerical investigation indicates that for the regular IC, the correlation between states impacts the achievability of the channel capacity in a different way from that of the Z-IC

Optimal Feedback Communication via Posterior Matching

In this paper we introduce a fundamental principle for optimal communication over general memoryless channels in the presence of noiseless feedback, termed posterior matching. Using this principle, we devise a (simple, sequential) generic feedback transmission scheme suitable for a large class of memoryless channels and input distributions, achieving any rate below the corresponding mutual information. This provides a unified framework for optimal feedback communication in which the Horstein scheme (BSC) and the Schalkwijk-Kailath scheme (AWGN channel) are special cases. Thus, as a corollary, we prove that the Horstein scheme indeed attains the BSC capacity, settling a longstanding conjecture. We further provide closed form expressions for the error probability of the scheme over a range of rates, and derive the achievable rates in a mismatch setting where the scheme is designed according to the wrong channel model. Several illustrative examples of the posterior matching scheme for specific channels are given, and the corresponding error probability expressions are evaluated. The proof techniques employed utilize novel relations between information rates and contraction properties of iterated function systems.Comment: IEEE Transactions on Information Theor

Local to global geometric methods in information theory

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2008.Includes bibliographical references (p. 201-203).This thesis treats several information theoretic problems with a unified geometric approach. The development of this approach was motivated by the challenges encountered while working on these problems, and in turn, the testing of the initial tools to these problems suggested numerous refinements and improvements on the geometric methods. In ergodic probabilistic settings, Sanov's theorem gives asymptotic estimates on the probabilities of very rare events. The theorem also characterizes the exponential decay of the probabilities, as the sample size grows, and the exponential rate is given by the minimization of a certain divergence expression. In his seminal paper, A Mathematical Theory of Communication, Shannon introduced two influential ideas to simplify the complex task of evaluating the performance of a coding scheme: the asymptotic perspective (in the number of channel uses) and the random coding argument. In this setting, Sanov's theorem can be used to analyze ergodic information theoretic problems, and the performance of a coding scheme can be estimated by expressions involving the divergence. One would then like to use a geometric intuition to solve these problems, but the divergence is not a distance and our naive geometric intuition may lead to incorrect conclusions. In information geometry, a specific differential geometric structure is introduced by means of "dual affine connections". The approach we take in this thesis is slightly different and is based on introducing additional asymptotic regimes to analyze the divergence expressions. The following two properties play an important role. The divergence may not be a distance, but locally (i.e., when its arguments are "close to each other"), the divergence behaves like a squared distance.(cont.) Moreover, globally (i.e., when its arguments have no local restriction), it also preserves certain properties satisfied by squared distances. Therefore, we develop the Very Noisy and Hermite transformations, as techniques to map our global information theoretic problems in local ones. Through this localization, our global divergence expressions reduce in the limit to expressions defined in an inner product space. This provides us with a valuable geometric insight to the global problems, as well as a strong tool to find counter-examples. Finally, in certain cases, we have been able to "lift" results proven locally to results proven globally.(cont.) Therefore, we develop the Very Noisy and Hermite transformations, as techniques to map our global information theoretic problems in local ones. Through this localization, our global divergence expressions reduce in the limit to expressions defined in an inner product space. This provides us with a valuable geometric insight to the global problems, as well as a strong tool to find counter-examples. Finally, in certain cases, we have been able to "lift" results proven locally to results proven globally. We consider the following three problems. First, we address the problem of finding good linear decoders (maximizing additive metrics) for compound discrete memoryless channels. Known universal decoders are not linear and most of them heavily depend on the finite alphabet assumption. We show that by using a finite number of additive metrics, we can construct decoders that are universal (capacity achieving) on most compound sets. We then consider additive Gaussian noise channels. For a given perturbation of a Gaussian input distribution, we define an operator that measures how much variation is induced in the output entropy. We found that the singular functions of this operator are the Hermite polynomials, and the singular values are the powers of a signal to noise ratio. We show, in particular, how to use this structure on a Gaussian interference channel to characterize a regime where interference should not be treated as noise. Finally, we consider multi-input multi-output channels and discuss the properties of the optimal input distributions, for various random fading matrix ensembles. In particular, we prove Telatar's conjecture on the covariance structure minimizing the outage probability for output dimension one and input dimensions less than one hundred.by Emmanuel Auguste Abbe.Ph.D

Unreliable and resource-constrained decoding

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2010.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Cataloged from student submitted PDF version of thesis.Includes bibliographical references (p. 185-213).Traditional information theory and communication theory assume that decoders are noiseless and operate without transient or permanent faults. Decoders are also traditionally assumed to be unconstrained in physical resources like material, memory, and energy. This thesis studies how constraining reliability and resources in the decoder limits the performance of communication systems. Five communication problems are investigated. Broadly speaking these are communication using decoders that are wiring cost-limited, that are memory-limited, that are noisy, that fail catastrophically, and that simultaneously harvest information and energy. For each of these problems, fundamental trade-offs between communication system performance and reliability or resource consumption are established. For decoding repetition codes using consensus decoding circuits, the optimal tradeoff between decoding speed and quadratic wiring cost is defined and established. Designing optimal circuits is shown to be NP-complete, but is carried out for small circuit size. The natural relaxation to the integer circuit design problem is shown to be a reverse convex program. Random circuit topologies are also investigated. Uncoded transmission is investigated when a population of heterogeneous sources must be categorized due to decoder memory constraints. Quantizers that are optimal for mean Bayes risk error, a novel fidelity criterion, are designed. Human decision making in segregated populations is also studied with this framework. The ratio between the costs of false alarms and missed detections is also shown to fundamentally affect the essential nature of discrimination. The effect of noise on iterative message-passing decoders for low-density parity check (LDPC) codes is studied. Concentration of decoding performance around its average is shown to hold. Density evolution equations for noisy decoders are derived. Decoding thresholds degrade smoothly as decoder noise increases, and in certain cases, arbitrarily small final error probability is achievable despite decoder noisiness. Precise information storage capacity results for reliable memory systems constructed from unreliable components are also provided. Limits to communicating over systems that fail at random times are established. Communication with arbitrarily small probability of error is not possible, but schemes that optimize transmission volume communicated at fixed maximum message error probabilities are determined. System state feedback is shown not to improve performance. For optimal communication with decoders that simultaneously harvest information and energy, a coding theorem that establishes the fundamental trade-off between the rates at which energy and reliable information can be transmitted over a single line is proven. The capacity-power function is computed for several channels; it is non-increasing and concave.by Lav R. Varshney.Ph.D

Advanced Coding And Modulation For Ultra-wideband And Impulsive Noises

The ever-growing demand for higher quality and faster multimedia content delivery over short distances in home environments drives the quest for higher data rates in wireless personal area networks (WPANs). One of the candidate IEEE 802.15.3a WPAN proposals support data rates up to 480 Mbps by using punctured convolutional codes with quadrature phase shift keying (QPSK) modulation for a multi-band orthogonal frequency-division multiplexing (MB-OFDM) system over ultra wideband (UWB) channels. In the first part of this dissertation, we combine more powerful near-Shannon-limit turbo codes with bandwidth efficient trellis coded modulation, i.e., turbo trellis coded modulation (TTCM), to further improve the data rates up to 1.2 Gbps. A modified iterative decoder for this TTCM coded MB-OFDM system is proposed and its bit error rate performance under various impulsive noises over both Gaussian and UWB channel is extensively investigated, especially in mismatched scenarios. A robust decoder which is immune to noise mismatch is provided based on comparison of impulsive noises in time domain and frequency domain. The accurate estimation of the dynamic noise model could be very difficult or impossible at the receiver, thus a significant performance degradation may occur due to noise mismatch. In the second part of this dissertation, we prove that the minimax decoder in \cite, which instead of minimizing the average bit error probability aims at minimizing the worst bit error probability, is optimal and robust to certain noise model with unknown prior probabilities in two and higher dimensions. Besides turbo codes, another kind of error correcting codes which approach the Shannon capacity is low-density parity-check (LDPC) codes. In the last part of this dissertation, we extend the density evolution method for sum-product decoding using mismatched noises. We will prove that as long as the true noise type and the estimated noise type used in the decoder are both binary-input memoryless output symmetric channels, the output from mismatched log-likelihood ratio (LLR) computation is also symmetric. We will show the Shannon capacity can be evaluated for mismatched LLR computation and it can be reduced if the mismatched LLR computation is not an one-to-one mapping function. We will derive the Shannon capacity, threshold and stable condition of LDPC codes for mismatched BIAWGN and BIL noise types. The results show that the noise variance estimation errors will not affect the Shannon capacity and stable condition, but the errors do reduce the threshold. The mismatch in noise type will only reduce Shannon capacity when LLR computation is based on BIL

Coding for Relay Networks with Parallel Gaussian Channels

A wireless relay network consists of multiple source nodes, multiple destination nodes, and possibly many relay nodes in between to facilitate its transmission. It is clear that the performance of such networks highly depends on information for- warding strategies adopted at the relay nodes. This dissertation studies a particular information forwarding strategy called compute-and-forward. Compute-and-forward is a novel paradigm that tries to incorporate the idea of network coding within the physical layer and hence is often referred to as physical layer network coding. The main idea is to exploit the superposition nature of the wireless medium to directly compute or decode functions of transmitted signals at intermediate relays in a net- work. Thus, the coding performed at the physical layer serves the purpose of error correction as well as permits recovery of functions of transmitted signals. For the bidirectional relaying problem with Gaussian channels, it has been shown by Wilson et al. and Nam et al. that the compute-and-forward paradigm is asymptotically optimal and achieves the capacity region to within 1 bit; however, similar results beyond the memoryless case are still lacking. This is mainly because channels with memory would destroy the lattice structure that is most crucial for the compute-and-forward paradigm. Hence, how to extend compute-and-forward to such channels has been a challenging issue. This motivates this study of the extension of compute-and-forward to channels with memory, such as inter-symbol interference. The bidirectional relaying problem with parallel Gaussian channels is also studied, which is a relevant model for the Gaussian bidirectional channel with inter-symbol interference and that with multiple-input multiple-output channels. Motivated by the recent success of linear finite-field deterministic model, we first investigate the corresponding deterministic parallel bidirectional relay channel and fully characterize its capacity region. Two compute-and-forward schemes are then proposed for the Gaussian model and the capacity region is approximately characterized to within a constant gap. The design of coding schemes for the compute-and-forward paradigm with low decoding complexity is then considered. Based on the separation-based framework proposed previously by Tunali et al., this study proposes a family of constellations that are suitable for the compute-and-forward paradigm. Moreover, by using Chinese remainder theorem, it is shown that the proposed constellations are isomorphic to product fields and therefore can be put into a multilevel coding framework. This study then proposes multilevel coding for the proposed constellations and uses multistage decoding to further reduce decoding complexity