A Novel Stochastic Decoding of LDPC Codes with Quantitative Guarantees

Low-density parity-check codes, a class of capacity-approaching linear codes, are particularly recognized for their efficient decoding scheme. The decoding scheme, known as the sum-product, is an iterative algorithm consisting of passing messages between variable and check nodes of the factor graph. The sum-product algorithm is fully parallelizable, owing to the fact that all messages can be update concurrently. However, since it requires extensive number of highly interconnected wires, the fully-parallel implementation of the sum-product on chips is exceedingly challenging. Stochastic decoding algorithms, which exchange binary messages, are of great interest for mitigating this challenge and have been the focus of extensive research over the past decade. They significantly reduce the required wiring and computational complexity of the message-passing algorithm. Even though stochastic decoders have been shown extremely effective in practice, the theoretical aspect and understanding of such algorithms remains limited at large. Our main objective in this paper is to address this issue. We first propose a novel algorithm referred to as the Markov based stochastic decoding. Then, we provide concrete quantitative guarantees on its performance for tree-structured as well as general factor graphs. More specifically, we provide upper-bounds on the first and second moments of the error, illustrating that the proposed algorithm is an asymptotically consistent estimate of the sum-product algorithm. We also validate our theoretical predictions with experimental results, showing we achieve comparable performance to other practical stochastic decoders.Comment: This paper has been submitted to IEEE Transactions on Information Theory on May 24th 201

Noisy Gradient Descent Bit-Flip Decoding for LDPC Codes

A modified Gradient Descent Bit Flipping (GDBF) algorithm is proposed for decoding Low Density Parity Check (LDPC) codes on the binary-input additive white Gaussian noise channel. The new algorithm, called Noisy GDBF (NGDBF), introduces a random perturbation into each symbol metric at each iteration. The noise perturbation allows the algorithm to escape from undesirable local maxima, resulting in improved performance. A combination of heuristic improvements to the algorithm are proposed and evaluated. When the proposed heuristics are applied, NGDBF performs better than any previously reported GDBF variant, and comes within 0.5 dB of the belief propagation algorithm for several tested codes. Unlike other previous GDBF algorithms that provide an escape from local maxima, the proposed algorithm uses only local, fully parallelizable operations and does not require computing a global objective function or a sort over symbol metrics, making it highly efficient in comparison. The proposed NGDBF algorithm requires channel state information which must be obtained from a signal to noise ratio (SNR) estimator. Architectural details are presented for implementing the NGDBF algorithm. Complexity analysis and optimizations are also discussed.Comment: 16 pages, 22 figures, 2 table

Fault Tolerance of Stochastic Decoders for Error Correcting Codes

Low-density Parity-check (LDPC) codes are very powerful linear error-correcting codes, first introduced by Gallager in 1963. They are now used in many communication standards due to their ability to achieve near Shannon-capacity performance. Stochastic decoding is a hardware-efficient method of iterative decoding of LDPC codes. In this work, we investigate the capability of stochastic decoding to tolerate circuit soft errors while maintaining good bit error rate performance and low error floor. Soft errors can be intended faults as a result of either supply voltage scaling to reduce power consumption or overclocking the system to achieve a higher throughput. They can also be unintended faults as a result of temperature or process variations. We develop two models to emulate these circuit errors at the system level. We apply our models to two standardized LDPC codes (10GBASE-T and WiMAX). Simulation results show that stochastic decoding is very tolerant to faults and errors, where it can tolerate a probability of setup time violation of 0.1 in the wires of the decoder. Hence, stochastic decoding can be very useful in systems with very low power or high performance requirements where we can push the limits of power or speed by lowering the supply voltage or highly overclocking the system while maintaining good performance. In addition, a chip has been designed and sent to fabrication to do post-silicon validation and verify our models

Lossy Image Compression with Conditional Diffusion Models

Denoising diffusion models have recently marked a milestone in high-quality image generation. One may thus wonder if they are suitable for neural image compression. This paper outlines an end-to-end optimized image compression framework based on a conditional diffusion model, drawing on the transform-coding paradigm. Besides the latent variables inherent to the diffusion process, this paper introduces an additional discrete "content" latent variable to condition the denoising process on. This variable is equipped with a hierarchical prior for entropy coding. The remaining "texture" latent variables characterizing the diffusion process are synthesized (either stochastically or deterministically) at decoding time. We furthermore show that the performance can be tuned toward perceptual metrics of interest. Our extensive experiments involving five datasets and 16 image perceptual quality assessment metrics show that our approach not only compares favorably in terms of rate and perceptual distortion tradeoffs but also shows robust performance under all metrics while other baselines show less consistent behavior.Comment: Accepted at the ECCV 2022 Workshop on Uncertainty Quantification for Computer Visio

Stochastic multiple-stream decoding of Cortex codes

International audienceCortex codes are short length block codes having a large Hamming distance. Their modular construction, based on simple and very short block codes, yield to difficulties in efficiently decoding them with digital decoders implementing the Sum-Product algorithm. However, this construction lends itself to analog decoding with performance close to ML decoding as was recently demonstrated. A digital decoding method close to analog decoding is stochastic decoding. This paper brings the two together to design a Cortex stochastic architecture with good decoding performance. Moreover, the proposed stochastic decoder architecture is simplified when compared to the customary one. Instead of edge or tracking forecast memories the proposed architecture uses multiple streams to represent the same probability and deterministic shufflers. This results in a more efficient architecture in terms of ratio between data throughput and hardware complexity. Finally, the proposed method offers decoding performance similar to a Min-Sum decoder with 50 iterations