Variable-Length Coding of Two-Sided Asymptotically Mean Stationary Measures

We collect several observations that concern variable-length coding of two-sided infinite sequences in a probabilistic setting. Attention is paid to images and preimages of asymptotically mean stationary measures defined on subsets of these sequences. We point out sufficient conditions under which the variable-length coding and its inverse preserve asymptotic mean stationarity. Moreover, conditions for preservation of shift-invariant $\sigma$-fields and the finite-energy property are discussed and the block entropies for stationary means of coded processes are related in some cases. Subsequently, we apply certain of these results to construct a stationary nonergodic process with a desired linguistic interpretation.Comment: 20 pages. A few typos corrected after the journal publicatio

Block-Fading Channels with Delayed CSIT at Finite Blocklength

In many wireless systems, the channel state information at the transmitter (CSIT) can not be learned until after a transmission has taken place and is thereby outdated. In this paper, we study the benefits of delayed CSIT on a block-fading channel at finite blocklength. First, the achievable rates of a family of codes that allows the number of codewords to expand during transmission, based on delayed CSIT, are characterized. A fixed-length and a variable-length characterization of the rates are provided using the dependency testing bound and the variable-length setting introduced by Polyanskiy et al. Next, a communication protocol based on codes with expandable message space is put forth, and numerically, it is shown that higher rates are achievable compared to coding strategies that do not benefit from delayed CSIT.Comment: Extended version of a paper submitted to ISIT'1

Efficient Universal Noiseless Source Codes

Although the existence of universal noiseless variable-rate codes for the class of discrete stationary ergodic sources has previously been established, very few practical universal encoding methods are available. Efficient implementable universal source coding techniques are discussed in this paper. Results are presented on source codes for which a small value of the maximum redundancy is achieved with a relatively short block length. A constructive proof of the existence of universal noiseless codes for discrete stationary sources is first presented. The proof is shown to provide a method for obtaining efficient universal noiseless variable-rate codes for various classes of sources. For memoryless sources, upper and lower bounds are obtained for the minimax redundancy as a function of the block length of the code. Several techniques for constructing universal noiseless source codes for memoryless sources are presented and their redundancies are compared with the bounds. Consideration is given to possible applications to data compression for certain nonstationary sources

Error Exponents for Variable-length Block Codes with Feedback and Cost Constraints

Variable-length block-coding schemes are investigated for discrete memoryless channels with ideal feedback under cost constraints. Upper and lower bounds are found for the minimum achievable probability of decoding error $P_{e,\min}$ as a function of constraints R, \AV, and $\bar \tau$ on the transmission rate, average cost, and average block length respectively. For given $R$ and \AV, the lower and upper bounds to the exponent $-(\ln P_{e,\min})/\bar \tau$ are asymptotically equal as $\bar \tau \to \infty$. The resulting reliability function, $\lim_{\bar \tau\to \infty} (-\ln P_{e,\min})/\bar \tau$, as a function of $R$ and \AV, is concave in the pair (R, \AV) and generalizes the linear reliability function of Burnashev to include cost constraints. The results are generalized to a class of discrete-time memoryless channels with arbitrary alphabets, including additive Gaussian noise channels with amplitude and power constraints

Less redundant codes for variable size dictionaries

We report on a family of variable-length codes with less redundancy than the flat code used in most of the variable size dictionary-based compression methods. The length of codes belonging to this family is still bounded above by [log_2/ |D|] where |D| denotes the dictionary size. We describe three of these codes, namely, the balanced code, the phase-in-binary code (PB), and the depth-span code (DS). As the name implies, the balanced code is constructed by a height balanced tree, so it has the shortest average codeword length. The corresponding coding tree for the PB code has an interesting property that it is made of full binary phases, and thus the code can be computed efficiently using simple binary shifting operations. The DS coding tree is maintained in such a way that the coder always finds the longest extendable codeword and extends it until it reaches the maximum length. It is optimal with respect to the code-length contrast. The PB and balanced codes have almost similar improvements, around 3% to 7% which is very close to the relative redundancy in flat code. The DS code is particularly good in dealing with files with a large amount of redundancy, such as a running sequence of one symbol. We also did some empirical study on the codeword distribution in the LZW dictionary and proposed a scheme called dynamic block shifting (DBS) to further improve the codes' performance. Experiments suggest that the DBS is helpful in compressing random sequences. From an application point of view, PB code with DBS is recommended for general practical usage

Numerical Study on Joint Quantization and Control under Block-Coding

This paper addresses the joint quantization and control problem for hidden Markov chains with variable-length block-coding. The aim is to understand the impact of communication bandwidth and information delay (due to block-coding) on the control performance. A heuristic algorithm is developed to solve the dynamic programming (DP) equation through the introduction of a metric on the discrete observation space. Numerical results are presented demonstrating the attention division in block-coding and the tradeoffs between control performance and communication bandwidth

Variable-Length Coding with Feedback: Finite-Length Codewords and Periodic Decoding

Theoretical analysis has long indicated that feedback improves the error exponent but not the capacity of single-user memoryless channels. Recently Polyanskiy et al. studied the benefit of variable-length feedback with termination (VLFT) codes in the non-asymptotic regime. In that work, achievability is based on an infinite length random code and decoding is attempted at every symbol. The coding rate backoff from capacity due to channel dispersion is greatly reduced with feedback, allowing capacity to be approached with surprisingly small expected latency. This paper is mainly concerned with VLFT codes based on finite-length codes and decoding attempts only at certain specified decoding times. The penalties of using a finite block-length $N$ and a sequence of specified decoding times are studied. This paper shows that properly scaling $N$ with the expected latency can achieve the same performance up to constant terms as with $N = \infty$. The penalty introduced by periodic decoding times is a linear term of the interval between decoding times and hence the performance approaches capacity as the expected latency grows if the interval between decoding times grows sub-linearly with the expected latency.Comment: 8 pages. A shorten version is submitted to ISIT 201