An Upper Bound on the Capacity of non-Binary Deletion Channels

We derive an upper bound on the capacity of non-binary deletion channels. Although binary deletion channels have received significant attention over the years, and many upper and lower bounds on their capacity have been derived, such studies for the non-binary case are largely missing. The state of the art is the following: as a trivial upper bound, capacity of an erasure channel with the same input alphabet as the deletion channel can be used, and as a lower bound the results by Diggavi and Grossglauser are available. In this paper, we derive the first non-trivial non-binary deletion channel capacity upper bound and reduce the gap with the existing achievable rates. To derive the results we first prove an inequality between the capacity of a 2K-ary deletion channel with deletion probability $d$, denoted by $C_{2K}(d)$, and the capacity of the binary deletion channel with the same deletion probability, $C_2(d)$, that is, $C_{2K}(d)\leq C_2(d)+(1-d)\log(K)$. Then by employing some existing upper bounds on the capacity of the binary deletion channel, we obtain upper bounds on the capacity of the 2K-ary deletion channel. We illustrate via examples the use of the new bounds and discuss their asymptotic behavior as $d \rightarrow 0$.Comment: accepted for presentation in ISIT 201

A Note on the Deletion Channel Capacity

Memoryless channels with deletion errors as defined by a stochastic channel matrix allowing for bit drop outs are considered in which transmitted bits are either independently deleted with probability $d$ or unchanged with probability $1-d$. Such channels are information stable, hence their Shannon capacity exists. However, computation of the channel capacity is formidable, and only some upper and lower bounds on the capacity exist. In this paper, we first show a simple result that the parallel concatenation of two different independent deletion channels with deletion probabilities $d_1$ and $d_2$, in which every input bit is either transmitted over the first channel with probability of $\lambda$ or over the second one with probability of $1-\lambda$, is nothing but another deletion channel with deletion probability of $d=\lambda d_1+(1-\lambda)d_2$. We then provide an upper bound on the concatenated deletion channel capacity $C(d)$ in terms of the weighted average of $C(d_1)$, $C(d_2)$ and the parameters of the three channels. An interesting consequence of this bound is that $C(\lambda d_1+(1-\lambda))\leq \lambda C(d_1)$ which enables us to provide an improved upper bound on the capacity of the i.i.d. deletion channels, i.e., $C(d)\leq 0.4143(1-d)$ for $d\geq 0.65$. This generalizes the asymptotic result by Dalai as it remains valid for all $d\geq 0.65$. Using the same approach we are also able to improve upon existing upper bounds on the capacity of the deletion/substitution channel.Comment: Submitted to the IEEE Transactions on Information Theor

Write Channel Model for Bit-Patterned Media Recording

We propose a new write channel model for bit-patterned media recording that reflects the data dependence of write synchronization errors. It is shown that this model accommodates both substitution-like errors and insertion-deletion errors whose statistics are determined by an underlying channel state process. We study information theoretic properties of the write channel model, including the capacity, symmetric information rate, Markov-1 rate and the zero-error capacity.Comment: 11 pages, 12 figures, journa

Bounds on the Capacity of Random Insertion and Deletion-Additive Noise Channels

We develop several analytical lower bounds on the capacity of binary insertion and deletion channels by considering independent uniformly distributed (i.u.d.) inputs and computing lower bounds on the mutual information between the input and output sequences. For the deletion channel, we consider two different models: independent and identically distributed (i.i.d.) deletion-substitution channel and i.i.d. deletion channel with additive white Gaussian noise (AWGN). These two models are considered to incorporate effects of the channel noise along with the synchronization errors. For the insertion channel case we consider the Gallager's model in which the transmitted bits are replaced with two random bits and uniform over the four possibilities independently of any other insertion events. The general approach taken is similar in all cases, however the specific computations differ. Furthermore, the approach yields a useful lower bound on the capacity for a wide range of deletion probabilities for the deletion channels, while it provides a beneficial bound only for small insertion probabilities (less than 0.25) for the insertion model adopted. We emphasize the importance of these results by noting that 1) our results are the first analytical bounds on the capacity of deletion-AWGN channels, 2) the results developed are the best available analytical lower bounds on the deletion-substitution case, 3) for the Gallager insertion channel model, the new lower bound improves the existing results for small insertion probabilities.Comment: Accepted for publication in IEEE Transactions on Information Theor

Directly Lower Bounding the Information Capacity for Channels with I.I.D. Deletions and Duplications

We directly lower bound the information capacity for channels with i.i.d. deletions and duplications. Our approach differs from previous work in that we focus on the information capacity using ideas from renewal theory, rather than focusing on the transmission capacity by analyzing the error probability of some randomly generated code using a combinatorial argument. Of course, the transmission and information capacities are equal, but our change of perspective allows for a much simpler analysis that gives more general theoretical results. We then apply these results to the binary deletion channel to improve existing lower bounds on its capacity