Computable Lower Bounds for Capacities of Input-Driven Finite-State Channels

This paper studies the capacities of input-driven finite-state channels, i.e., channels whose current state is a time-invariant deterministic function of the previous state and the current input. We lower bound the capacity of such a channel using a dynamic programming formulation of a bound on the maximum reverse directed information rate. We show that the dynamic programming-based bounds can be simplified by solving the corresponding Bellman equation explicitly. In particular, we provide analytical lower bounds on the capacities of $(d, k)$-runlength-limited input-constrained binary symmetric and binary erasure channels. Furthermore, we provide a single-letter lower bound based on a class of input distributions with memory.Comment: 9 pages, 8 figures, submitted to International Symposium on Information Theory, 202

Finite-State Channels with Feedback and State Known at the Encoder

We consider finite state channels (FSCs) with feedback and state information known causally at the encoder. This setting is quite general and includes: a memoryless channel with i.i.d. state (the Shannon strategy), Markovian states that include look-ahead (LA) access to the state and energy harvesting. We characterize the feedback capacity of the general setting as the directed information between auxiliary random variables with memory to the channel outputs. We also propose two methods for computing the feedback capacity: (i) formulating an infinite-horizon average-reward dynamic program; and (ii) a single-letter lower bound based on auxiliary directed graphs called $Q$-graphs. We demonstrate our computation methods on several examples. In the first example, we introduce a channel with LA and derive a closed-form, analytic lower bound on its feedback capacity. Furthermore, we show that the mentioned methods achieve the feedback capacity of known unifilar FSCs such as the trapdoor channel, the Ising channel and the input-constrained erasure channel. Finally, we analyze the feedback capacity of a channel whose state is stochastically dependent on the input.Comment: 39 pages, 10 figures. The material in this paper was presented in part at the 56th Annual Allerton Conference on Communication, Control, and Computing, Monticello, IL, USA, October 2018, and at the IEEE International Symposium on Information Theory, Los Angeles, CA, USA, June 202

The Duality Upper Bound for Finite-State Channels with Feedback

This paper investigates the capacity of finite-state channels (FSCs) with feedback. We derive an upper bound on the feedback capacity of FSCs by extending the duality upper bound method from mutual information to the case of directed information. The upper bound is expressed as a multi-letter expression that depends on a test distribution on the sequence of channel outputs. For any FSC, we show that if the test distribution is structured on a $Q$-graph, the upper bound can be formulated as a Markov decision process (MDP) whose state being a belief on the channel state. In the case of FSCs and states that are either unifilar or have a finite memory, the MDP state simplifies to take values in a finite set. Consequently, the MDP consists of a finite number of states, actions, and disturbances. This finite nature of the MDP is of significant importance, as it ensures that dynamic programming algorithms can solve the associated Bellman equation to establish analytical upper bounds, even for channels with large alphabets. We demonstrate the simplicity of computing bounds by establishing the capacity of a broad family of Noisy Output is the State (NOST) channels as a simple closed-form analytical expression. Furthermore, we introduce novel, nearly optimal analytical upper bounds on the capacity of the Noisy Ising channel

Achievable Rates and Low-Complexity Encoding of Posterior Matching for the BSC

Horstein, Burnashev, Shayevitz and Feder, Naghshvar et al. and others have studied sequential transmission of a K-bit message over the binary symmetric channel (BSC) with full, noiseless feedback using posterior matching. Yang et al. provide an improved lower bound on the achievable rate using martingale analysis that relies on the small-enough difference (SED) partitioning introduced by Naghshvar et al. SED requires a relatively complex encoder and decoder. To reduce complexity, this paper replaces SED with relaxed constraints that admit the small enough absolute difference (SEAD) partitioning rule. The main analytical results show that achievable-rate bounds higher than those found by Yang et al. are possible even under the new constraints, which are less restrictive than SED. The new analysis does not use martingale theory for the confirmation phase and applies a surrogate channel technique to tighten the results. An initial systematic transmission further increases the achievable rate bound. The simplified encoder associated with SEAD has a complexity below order O(K^2) and allows simulations for message sizes of at least 1000 bits. For example, simulations achieve 99% of of the channel's 0.50-bit capacity with an average block size of 200 bits for a target codeword error rate of 10^(-3).Comment: This paper consists of 26 pages and contains 6 figures. An earlier version of the algorithm included in this paper was published at the 2020 IEEE International Symposium on Information Theory (ISIT), (DOI: 10.1109/ISIT44484.2020.9174232