Data expansion with Huffman codes

The following topics were dealt with: Shannon theory; universal lossless source coding; CDMA; turbo codes; broadband networks and protocols; signal processing and coding; coded modulation; information theory and applications; universal lossy source coding; algebraic geometry codes; modelling analysis and stability in networks; trellis structures and trellis decoding; channel capacity; recording channels; fading channels; convolutional codes; neural networks and learning; estimation; Gaussian channels; rate distortion theory; constrained channels; 2D channel coding; nonparametric estimation and classification; data compression; synchronisation and interference in communication systems; cyclic codes; signal detection; group codes; multiuser systems; entropy and noiseless source coding; dispersive channels and equalisation; block codes; cryptography; image processing; quantisation; random processes; wavelets; sequences for synchronisation; iterative decoding; optical communications

The Rényi Redundancy of Generalized Huffman Codes

Huffman's algorithm gives optimal codes, as measured by average codeword length, and the redundancy can be measured as the difference between the average codeword length and Shannon's entropy. If the objective function is replaced by an exponentially weighted average, then a simple modification of Huffman's algorithm gives optimal codes. The redundancy can now be measured as the difference between this new average and A. Renyi's (1961) generalization of Shannon's entropy. By decreasing some of the codeword lengths in a Shannon code, the upper bound on the redundancy given in the standard proof of the noiseless source coding theorem is improved. The lower bound is improved by randomizing between codeword lengths, allowing linear programming techniques to be used on an integer programming problem. These bounds are shown to be asymptotically equal. The results are generalized to the Renyi case and are related to R.G. Gallager's (1978) bound on the redundancy of Huffman codes

Algorithmic counting of nonequivalent compact Huffman codes

It is known that the following five counting problems lead to the same integer sequence~$f_t(n)$: the number of nonequivalent compact Huffman codes of length~$n$ over an alphabet of $t$ letters, the number of `nonequivalent' canonical rooted $t$-ary trees (level-greedy trees) with $n$~leaves, the number of `proper' words, the number of bounded degree sequences, and the number of ways of writing $1= \frac{1}{t^{x_1}}+ \dots + \frac{1}{t^{x_n}}$ with integers $0 \leq x_1 \leq x_2 \leq \dots \leq x_n$. In this work, we show that one can compute this sequence for \textbf{all} $n<N$ with essentially one power series division. In total we need at most $N^{1+\varepsilon}$ additions and multiplications of integers of $cN$ bits, $c<1$, or $N^{2+\varepsilon}$ bit operations, respectively. This improves an earlier bound by Even and Lempel who needed $O(N^3)$ operations in the integer ring or $O(N^4)$ bit operations, respectively

Maximal codeword lengths in Huffman codes

The following question about Huffman coding, which is an important technique for compressing data from a discrete source, is considered. If p is the smallest source probability, how long, in terms of p, can the longest Huffman codeword be? It is shown that if p is in the range 0 less than p less than or equal to 1/2, and if K is the unique index such that 1/F(sub K+3) less than p less than or equal to 1/F(sub K+2), where F(sub K) denotes the Kth Fibonacci number, then the longest Huffman codeword for a source whose least probability is p is at most K, and no better bound is possible. Asymptotically, this implies the surprising fact that for small values of p, a Huffman code's longest codeword can be as much as 44 percent larger than that of the corresponding Shannon code