Information-Distilling Quantizers

Let $X$ and $Y$ be dependent random variables. This paper considers the problem of designing a scalar quantizer for $Y$ to maximize the mutual information between the quantizer's output and $X$, and develops fundamental properties and bounds for this form of quantization, which is connected to the log-loss distortion criterion. The main focus is the regime of low $I(X;Y)$, where it is shown that, if $X$ is binary, a constant fraction of the mutual information can always be preserved using $\mathcal{O}(\log(1/I(X;Y)))$ quantization levels, and there exist distributions for which this many quantization levels are necessary. Furthermore, for larger finite alphabets $2 < |\mathcal{X}| < \infty$, it is established that an $\eta$-fraction of the mutual information can be preserved using roughly $(\log(| \mathcal{X} | /I(X;Y)))^{\eta\cdot(|\mathcal{X}| - 1)}$ quantization levels

Mutual Information-Maximizing Quantized Belief Propagation Decoding of Regular LDPC Codes

In mutual information-maximizing lookup table (MIM-LUT) decoding of low-density parity-check (LDPC) codes, table lookup operations are used to replace arithmetic operations. In practice, large tables need to be decomposed into small tables to save the memory consumption, at the cost of degraded error performance. In this paper, we propose a method, called mutual information-maximizing quantized belief propagation (MIM-QBP) decoding, to remove the lookup tables used for MIM-LUT decoding. Our method leads to a very efficient decoder, namely the MIM-QBP decoder, which can be implemented based only on simple mappings and fixed-point additions. Simulation results show that the MIM-QBP decoder can always considerably outperform the state-of-the-art MIM-LUT decoder, mainly because it can avoid the performance loss due to table decomposition. Furthermore, the MIM-QBP decoder with only 3 bits per message can outperform the floating-point belief propagation (BP) decoder at high signal-to-noise ratio (SNR) regions when testing on high-rate codes with a maximum of 10-30 iterations