Since irregular low-density parity-check (LDPC) codes are known to perform better than regular ones, and to exhibit, like them, the so-called \u2018threshold phenomenon\u2019, this Letter investigates a low-complexity upper bound on belief-propagation decoding thresholds for this class of codes on memoryless binary input additive white Gaussian noise channels, with sum-product decoding. A simplified analysis of the belief-propagation decoding algorithm is used, i.e. consider a Gaussian approximation for message densities under density evolution, and a simple algorithmic method, defined recently, to estimate the decoding thresholds for regular and irregular LDPC codes

Babich, F.

Soranzo, A.

Vatta, F.

English

Since irregular low-density parity-check (LDPC) codes are known to perform better than regular ones, and to exhibit, like them, the so-called ‘threshold phenomenon’, this Letter investigates a low-complexity upper bound on belief-propagation decoding thresholds for this class of codes on memoryless binary input additive white Gaussian noise channels, with sum-product decoding. A simplified analysis of the belief-propagation decoding algorithm is used, i.e. consider a Gaussian approximation for message densities under density evolution, and a simple algorithmic method, defined recently, to estimate the decoding thresholds for regular and irregular LDPC codes

Archivio istituzionale della ricerca - Università di Trieste

Low complexity bound on irregular LDPCbelief-propagation decoding thresholdsusing a Gaussian approximationF. Vatta, A. Soranzo, and F. BabichSince irregular low-density parity-check (LDPC) codes are known toperform better than regular ones, and to exhibit, like them, the so called“threshold phenomenon”, this letter investigates a low complexity upperbound on belief-propagation decoding thresholds for this class of codeson memoryless BI-AWGN (Binary Input - Additive White GaussianNoise) channels, with sum-product decoding. We use a simplifiedanalysis of the belief-propagation decoding algorithm, i.e., consider aGaussian approximation for message densities under density evolution,and a simple algorithmic method, defined recently, to estimate thedecoding thresholds for regular and irregular LDPC codes.Introduction: As first noticed by Gallager in his introductory workto regular LDPC codes [1], these exhibit the so called “thresholdphenomenon”. Namely, an upper bound for the channel noise can bedefined by the noise threshold so that, if the channel noise is maintainedbelow this threshold, the probability of lost information can be made assmall as desired. Later it was shown in [2] that irregular LDPC codesperform better than regular ones, and exhibit this phenomenon, too.LDPC codes are capacity-approaching codes, which means thatpractical constructions exist that allow the noise threshold to be set veryclose to the theoretical maximum (the Shannon limit) for a symmetricmemoryless channel. Thus, the problem of an easy evaluation of thethreshold, and, in general, of the performance of belief propagationdecoding (see, e.g., [3] and [4]) is important to allow the design ofcapacity-approaching codes, based on noise threshold maximization.Maximum Likelihood decoding of LDPC codes is in general notfeasible [3]. Instead, Gallager proposed an iterative soft decodingalgorithm, also called belief propagation [5]. Gallager also noted that, forany given channel conditions, it is possible to evaluate the performanceof belief propagation by following the evolution of the distribution ofthe messages. This idea was extended in [6], where it was shown howto apply density evolution efficiently. One difficulty encountered whenapplying density evolution is given by the continuous nature of themessages which makes them hard to analyze. As an alternative, in [7]a Gaussian approximation for the message distribution was proposed,reducing the evolution of the infinite dimensional density space to theevolution of a single parameter. In this way, the mean value of a genericcheck node output message at the l-th iteration is simply described as afunction of the check node output message mean value at the (l − 1)-th iteration, thus obtaining a recurrent sequence. With this simplifieddescription, the threshold can be calculated as the last value such thatthe recurrent sequence converges but no mathematical methods wereprovided in [7] to determine it.In [8] it was presented a mathematical method to allow the noisethresholds evaluation using the quadratic degeneracy theory, thustransforming a recurrence relation convergence problem in a problemof mathematical analysis. Applying the result of [8] to the asymptoticalbehavior of the recurrent sequence thereby defined, a low complexityupper bound to the exact belief-propagation decoding thresholds can bederived. This analysis gives rise to a simple algebraic expression of theupper bound on irregular LDPC belief-propagation decoding thresholdsusing a Gaussian approximation, valid in the most common case (i.e.,when the first non-zero coefficient of the dl-uple {λi} is λ2), thusallowing its simple determination from the codes parameters.Low complexity approximation of the exact belief-propagation decodingthresholds: Irregular LDPC codes [6] are defined by specifying thedistribution of the node degrees in their Tanner graphs. In particular,in the edge-perspective degree distribution, λi (respectively ρj) isthe fraction of edges in the Tanner graph connecting to a degree-i(respectively degree-j) variable (respectively check) node. To specify thedegree distribution, the polynomials λ(x) and ρ(x) are defined, havingdegree dl − 1 and dr − 1, respectively. The dl-uple {λi} and dr-uple{ρj} both add up to 1.Since in [7] it is shown that, without great sacrifice in accuracy, aone-dimensional quantity, namely the mean of Gaussian densities, canact as faithful surrogate for the message densities themselves, which isan infinite-dimensional vector, we assume that LDPC codes messagedistributions for AWGN channels are approximately Gaussian and denotethe means of u and v by m(l)u and m(l)v at the l-th iteration, respectively.Moreover, the LLR message u0 from the channel is assumed to beGaussian with mean mu0 = 2/σ2n and variance 4/σ2n , where σ2n is thevariance of the channel noise. Defining, as in [8],gs(t) =dr∑j=2ρjgs,j(t) (1)wheregs,j(t) = φ−1(1−[1−dl∑i=i1λiφ(s+ (i− 1)t)]j−1) (2)(being the function φ(x) defined in [7] and s=mu0 ), and applyingthe method therewith defined, instead of searching the last value of theparameter s granting the convergence of the sequencetl = gs(tl−1) (3)where tl corresponds to m(l)u , we solve a problem of quadraticdegeneracy which can be assigned to a standard software.When the second derivative g′′s (t) 6=0 the problem of quadraticdegeneracy is the system of equations{gs(t) = tg′s(t) = 1(4)Its solution (t∗, s∗) determines an approximation σ∗ of the exactbelief-propagation decoding threshold defined as σ∗ :=√2s∗.To find the solution of (4), an explicitly invertible approximationof φ(x) is needed. In [7], an “ad hoc” approximation by elementaryfunctions is given (see Eq. (8) in [7]) for 0<x< 10, which is explicitlyinvertible. A graph of φ(x) may be found in [9], where the approximationof φ(x) and its inverse were derived using the analysis outlined in [10].Upper bound: To determine the upper bound on threshold we need firstof all to determine the asymptotical behaviour of (4) for t→∞. For thispurpouse, we have to compute φ(x) in (4) with x≥ 10. To this end, weadd a further invertible approximation of the function φ(x), the derivationof which is given in the Appendix. With this approximation, definingzs(t) :=s+(i1−1)t2and Aj := 1(j−1)2λ2i1, we can write the followingLemma: As t→∞, gs(t) becomes:gs(t) = 2dr∑j=2ρjW(Ajzs(t)ezs(t)) (5)Proof: When t→∞, since, as shown in [9], the function φ(x) has arapid decrease in x, only the first terms of∑dli=i1λiφ(s+ (i− 1)t) areimportant in Eq. (1). Thus, we can simplify that sum to:dl∑i=i1λiφ(s+ (i− 1)t) = λi1φ(s+ (i1 − 1)t) + O(λi2φ(s+ (i2 − 1)t))(6)being λi1 and λi2 the first and second nonzero λi’s, respectively, andobserving that, in most practical cases, never λi2 is largely greater thanλi1 , and that they are both very significant. By the development(1 − x)n =1− nx+ O(x2), x→ 0 (7)gs,j(t) = φ−1((j − 1)λi1φ(s+ (i1 − 1)t)[1 + O(φ(s+ (i1 − 1)t))])(8)where we used φ(s+ (i2 − 1)t)≪ φ(s+ (i1 − 1)t). Thenφ(gs,j(t)) = (j − 1)λi1φ(s+ (i1 − 1)t)[1 + O(φ(s+ (i1 − 1)t))](9)Using the approximation (21) and ignoring the O(·) term:φ(gs,j(t)) =√pis+ (i1 − 1)te−s+(i1−1)t−4log((j−1)λi1)4 (10)ELECTRONICS LETTERS Vol. 00 No. 00and, applying (22),gs,j(t) = 2W( s+ (i1 − 1)t2es+(i1−1)t−4log((j−1)λi1)2)(11)This can be rewritten asgs,j(t) = 2W(Ajzs(t)ezs(t)) (12)and applying (1) we get the result.With gs(t) given in (5) and remembering that dW (x)dx = 1x+eW(x) :g′s(t) = 2dr∑j=2ρjz′s(t)ezs(t)(1 + zs(t))zs(t)ezs(t) + eW(Ajzs(t)ezs(t))−logAj(13)Applying (4) to (5) and (13) we get:2∑drj=2 ρjW(Ajzs(t)ezs(t)) = t2∑drj=2 ρjz′s(t)ezs(t)(1+zs(t))zs(t)ezs(t)+eW(Ajzs(t)ezs(t))−logAj= 1(14)Its solution (s∗, t∗), obtained applying the instruction set produced in[8], determines the bound σ∗ =√2s∗, which is valid ∀i1.With aj :=−2log((j − 1)λi1 ), Eq. (11) can be rewritten asgs,j(t) = 2W(zs(t)ezs(t)+aj)(15)Assuming that the following simplified approximation holdsgs,j(t)≃ 2W((zs(t) + aj)ezs(t)+aj), (16)calling x := zs(t) + aj , and being W (xex)≡ x for x> 0, we findanother asymptotical expression for gs,j(t) which is much simpler thanthe one obtained in (12):gs,j(t)≃ 2x=2(zs(t) + aj) = s+ (i1 − 1)t − 4log((j − 1)λi1 )(17)Applying (1) and getting its first derivative, we may rewrite (14) as:{s+ (i1 − 1)t − 4logλi1 − 4∑drj=2 ρj log(j − 1) = ti1 − 1 = 1(18)Its solution is:s∗approx = 4logλi1 + 4dr∑j=2ρj log(j − 1) (19)Using the Jensen’s inequality∏drj=2(j − 1)ρj ≤∑drj=2 ρj(j − 1), wemay write the following upper bound on (19):s∗approx ≤ s∗Jensen = 4logλi1 + 4log( dr∑j=2(j − 1)ρj)(20)Taking the simple algebrical expressions (19) and (20), σ∗ =√2s∗gives two further bounds on threshold, both valid only for i1 =2.Numeric results: For the irregular rate-1/2 LDPC codes reported in [6],we report in Table 1 the σ∗ values found with density evolution in [6]together with the upper bounds for σ we found in the present work,namely σ∗approx =√2/s∗approx (from (19)), σ∗Jensen =√2/s∗Jensen(from (20)), and σ∗ =√2/s∗ (solution of (14)).Conclusions: In this letter, the derivation of low complexity upperbounds on belief-propagation decoding thresholds was addressed, usingthe algorithmic method proposed in [8]. The results showed goodagreement with the threshold values found with density evolution in [6].Table 1: Bounds on decoding thresholds of good rate-1/2 codes listed in [6].dl 5 15 20 30i λi λi λi λi2 0.32660 0.23802 0.21991 0.196063 0.11960 0.20997 0.23328 0.240394 0.18393 0.03492 0.020585 0.36988 0.120156 0.08543 0.002287 0.01587 0.06540 0.055168 0.04767 0.166029 0.01912 0.0408810 0.0106414 0.0048015 0.3762719 0.0806420 0.2279828 0.0022130 0.28636j ρj ρj ρj ρj6 0.785557 0.214458 0.98013 0.64854 0.007499 0.01987 0.34747 0.9910110 0.00399 0.00150σ∗ 0.9194 0.9622 0.9649 0.9690σ∗approx 0.971728 0.987091 1.02193 1.05493σ∗Jensen 0.969081 0.986917 1.01966 1.05485σ∗ 0.97173 0.987094 1.02193 1.05493Appendix: Approximation of φ(x) for x≥ 10: In [7], the followingapproximation (called here φˆ(x)) was used for φ(x) when x is largeφ(x)≃ φˆ(x) :=√pixe−x4 (21)which we found invertible by means of the Lambert-W function. BeingW(·) the Lambert-W function, the inverse of φˆ(x) isφ−1(y)≃ φˆ−1(y) = 2W( pi2y2)(22)References1 R. G. Gallager, ‘Low-density parity-check codes’, IRE Transactions onInformation Theory, 1962, 8, pp. 21-28.2 M. G. Luby, M. Mitzenmacher, M. A. Shokrollahi, and D. A. Spielman,‘Analysis of Low Density Codes and Improved Design Using IrregularGraphs’, Proc.1998 ACM Symp. Theory of Computing, pp. 249-258.3 D. Burshtein and G. Miller, ‘Bounds on the Performance of BeliefPropagation Decoding”, IEEE Trans. on Inf. Theory, 2002, 48, pp. 112-122.4 L. Geller and D. Burshtein, ‘Bounds on the Belief PropagationThreshold of Non-Binary LDPC Codes”, Proc. of the 2012 IEEEInformation Theory Workshop, pp. 357-361.5 J. Pearl, ‘Probabilistic Reasoning in Intelligent Systems: Networks ofPlausible Inference’, Morgan Kaufmann Publishers, 1988.6 T. Richardson and R. Urbanke, ‘The Capacity of Low-Density ParityCheck Codes Under Message-Passing Decoding’, IEEE Trans. on Inf.Theory, 2001, 47, pp. 599-618.7 S.-Y. Chung, T. J. Richardson, and R. Urbanke, ‘Analysis of Sum-Product Decoding of Low-Density Parity-Check Codes Using aGaussian Approximation’, IEEE Trans. on Inf. Theory, 2001, 47, pp.657-670.8 F. Babich, A. Soranzo, and F. Vatta, ‘Useful mathematical tools forcapacity approaching codes design’, IEEE Communications Letters,2017, 21, pp. 1949-1952.9 F. Babich, M. Noschese, A. Soranzo, and F. Vatta,‘Low ComplexityRate Compatible Puncturing Patterns Design for LDPC Codes’, Proc.of the 2017 Int. Conf. SoftCOM, Split, Croatia, Sept. 21-23, 2017.10 F. Babich, M. Noschese, and F. Vatta, ‘Analysis and Design of RateCompatible LDPC Codes’, Proc. of the 27th IEEE InternationalSymposium on Personal, Indoor and Mobile Radio Communications -PIMRC ’16, Valencia, Spain, September 4-8, 2016, pp. 1-6.2

Low-complexity bound on irregular LDPC belief-propagation decoding thresholds using a Gaussian approximation

https://arts.units.it/retrieve/handle/11368/2928430/236395/BSV-LDPC-EL.pdf

Low-complexity bound on irregular LDPC belief-propagation decoding thresholds using a Gaussian approximation

Abstract

Similar works

Full text

Available Versions

Archivio istituzionale della ricerca - Università di Trieste