Achievable Rates for Noisy Channels with Synchronization Errors

Cataloged from PDF version of article.We develop several lower bounds on the capacity of binary input symmetric output channels with synchronization errors, which also suffer from other types of impairments such as substitutions, erasures, additive white Gaussian noise (AWGN), etc. More precisely, we show that if a channel suffering from synchronization errors as well as other type of impairments can be decomposed into a cascade of two component channels where the first one is another channel with synchronization errors and the second one is a memoryless channel (with no synchronization errors), a lower bound on the capacity of the original channel in terms of the capacity of the component synchronization error channel can be derived. A primary application of our results is that we can employ any lower bound derived on the capacity of the component synchronization error channel to find lower bounds on the capacity of the (original) noisy channel with synchronization errors. We apply the general ideas to several specific classes of channels such as synchronization error channels with erasures and substitutions, with symmetric q-ary outputs and with AWGN explicitly, and obtain easy-to-compute bounds. We illustrate that, with our approach, it is possible to derive tighter capacity lower bounds compared to the currently available bounds in the literature for certain classes of channels, e.g., deletion/substitution channels and deletion/AWGN channels (for certain signal-to-noise ratio (SNR) ranges). © 2014 IEEE

Bounds on the Capacity of Random Insertion and Deletion-Additive Noise Channels

We develop several analytical lower bounds on the capacity of binary insertion and deletion channels by considering independent uniformly distributed (i.u.d.) inputs and computing lower bounds on the mutual information between the input and output sequences. For the deletion channel, we consider two different models: independent and identically distributed (i.i.d.) deletion-substitution channel and i.i.d. deletion channel with additive white Gaussian noise (AWGN). These two models are considered to incorporate effects of the channel noise along with the synchronization errors. For the insertion channel case we consider the Gallager's model in which the transmitted bits are replaced with two random bits and uniform over the four possibilities independently of any other insertion events. The general approach taken is similar in all cases, however the specific computations differ. Furthermore, the approach yields a useful lower bound on the capacity for a wide range of deletion probabilities for the deletion channels, while it provides a beneficial bound only for small insertion probabilities (less than 0.25) for the insertion model adopted. We emphasize the importance of these results by noting that 1) our results are the first analytical bounds on the capacity of deletion-AWGN channels, 2) the results developed are the best available analytical lower bounds on the deletion-substitution case, 3) for the Gallager insertion channel model, the new lower bound improves the existing results for small insertion probabilities.Comment: Accepted for publication in IEEE Transactions on Information Theor

On Asynchronous Communication Systems: Capacity Bounds and Relaying Schemes

abstract: Practical communication systems are subject to errors due to imperfect time alignment among the communicating nodes. Timing errors can occur in different forms depending on the underlying communication scenario. This doctoral study considers two different classes of asynchronous systems; point-to-point (P2P) communication systems with synchronization errors, and asynchronous cooperative systems. In particular, the focus is on an information theoretic analysis for P2P systems with synchronization errors and developing new signaling solutions for several asynchronous cooperative communication systems. The first part of the dissertation presents several bounds on the capacity of the P2P systems with synchronization errors. First, binary insertion and deletion channels are considered where lower bounds on the mutual information between the input and output sequences are computed for independent uniformly distributed (i.u.d.) inputs. Then, a channel suffering from both synchronization errors and additive noise is considered as a serial concatenation of a synchronization error-only channel and an additive noise channel. It is proved that the capacity of the original channel is lower bounded in terms of the synchronization error-only channel capacity and the parameters of both channels. On a different front, to better characterize the deletion channel capacity, the capacity of three independent deletion channels with different deletion probabilities are related through an inequality resulting in the tightest upper bound on the deletion channel capacity for deletion probabilities larger than 0.65. Furthermore, the first non-trivial upper bound on the 2K-ary input deletion channel capacity is provided by relating the 2K-ary input deletion channel capacity with the binary deletion channel capacity through an inequality. The second part of the dissertation develops two new relaying schemes to alleviate asynchronism issues in cooperative communications. The first one is a single carrier (SC)-based scheme providing a spectrally efficient Alamouti code structure at the receiver under flat fading channel conditions by reducing the overhead needed to overcome the asynchronism and obtain spatial diversity. The second one is an orthogonal frequency division multiplexing (OFDM)-based approach useful for asynchronous cooperative systems experiencing excessive relative delays among the relays under frequency-selective channel conditions to achieve a delay diversity structure at the receiver and extract spatial diversity.Dissertation/ThesisPh.D. Electrical Engineering 201

Maximum Likelihood Upper Bounds on the Capacities of Discrete Information Stable Channels

Motivated by generating information stable processes greedily, we prove a universal maximum likelihood (ML) upper bound on the capacities of discrete information stable channels. The bound is derived leveraging a system of equations obtained via the Karush-Kuhn-Tucker (KKT) conditions. Intriguingly, for some discrete memoryless channels (DMCs), for instance, the BEC and BSC, the associated upper bounds are tight and equal to their capacities. Furthermore, for discrete channels with memory, as a particular example, we apply the ML bound to the BDC. The derived upper bound is a sum-max function related to counting the number of possible ways that a length-m binary subsequence that can be obtained by deleting n – m bits (with n – m close to nd and d denotes the deletion probability) of a length-n binary sequence. A full version of this paper is accessible at [1]

Maximum Likelihood Upper Bounds on the Capacities of Discrete Information Stable Channels

Motivated by a greedy approach for generating {\it{information stable}} processes, we prove a universal maximum likelihood (ML) upper bound on the capacities of discrete information stable channels, including the binary erasure channel (BEC), the binary symmetric channel (BSC) and the binary deletion channel (BDC). The bound is derived leveraging a system of equations obtained via the Karush-Kuhn-Tucker conditions. Intriguingly, for some memoryless channels, e.g., the BEC and BSC, the resulting upper bounds are tight and equal to their capacities. For the BDC, the universal upper bound is related to a function counting the number of possible ways that a length-\lo binary subsequence can be obtained by deleting $n-m$ bits (with $n-m$ close to $nd$ and $d$ denotes the {\it{deletion probability}}) of a length-$n$ binary sequence. To get explicit upper bounds from the universal upper bound, it requires to compute a maximization of the matching functions over a Hamming cube containing all length-$n$ binary sequences. Calculating the maximization exactly is hard. Instead, we provide a combinatorial formula approximating it. Under certain assumptions, several approximations and an {\it{explicit}} upper bound for deletion probability $d\geq 1/2$ are derived.Comment: 14 pages, 3 figure

Capacity Bounds and Concatenated Codes Over Segmented Deletion Channels

Cataloged from PDF version of article.We develop an information theoretic characterization and a practical coding approach for segmented deletion channels. Compared to channels with independent and identically distributed (i.i.d.) deletions, where each bit is independently deleted with an equal probability, the segmentation assumption imposes certain constraints, i.e., in a block of bits of a certain length, only a limited number of deletions are allowed to occur. This channel model has recently been proposed and motivated by the fact that for practical systems, when a deletion error occurs, it is more likely that the next one will not appear very soon. We first argue that such channels are information stable, hence their channel capacity exists. Then, we introduce several upper and lower bounds with two different methods in an attempt to understand the channel capacity behavior. The first scheme utilizes certain information provided to the transmitter and/or receiver while the second one explores the asymptotic behavior of the bounds when the average bit deletion rate is small. In the second part of the paper, we consider a practical channel coding approach over a segmented deletion channel. Specifically, we utilize outer LDPC codes concatenated with inner marker codes, and develop suitable channel detection algorithms for this scenario. Different maximum-a-posteriori (MAP) based channel synchronization algorithms operating at the bit and symbol levels are introduced, and specific LDPC code designs are explored. Simulation results clearly indicate the advantages of the proposed approach. In particular, for the entire range of deletion probabilities less than unity, our scheme offers a significantly larger transmission rate compared to the other existing solutions in the literature