Hehn, Thorsten

Hoeher, Peter Adam

Huber, Johannes

Land, Ingmar

    Aalborg UniversitetMutual Information Profile of a BISMC with ApplicationsHuber, Johannes; Hehn, Thorsten; Land, Ingmar Rüdiger; Hoeher, Peter AdamPublished in:Proceedings Conference in Information Sciences and Systems (CISS)Publication date:2005Document VersionPublisher's PDF, also known as Version of recordLink to publication from Aalborg UniversityCitation for published version (APA):Huber, J., Hehn, T., Land, I., & Hoeher, P. A. (2005). Mutual Information Profile of a BISMC with Applications. InProceedings Conference in Information Sciences and Systems (CISS) The Johns Hopkins University,Baltimore, MD.General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.            ? Users may download and print one copy of any publication from the public portal for the purpose of private study or research.            ? You may not further distribute the material or use it for any profit-making activity or commercial gain            ? You may freely distribute the URL identifying the publication in the public portal ?Take down policyIf you believe that this document breaches copyright please contact us at vbn@aub.aau.dk providing details, and we will remove access tothe work immediately and investigate your claim.Downloaded from vbn.aau.dk on: April 25, 20172005 Conference on Information Sciences and Systems, The Johns Hopkins University, March 16–18, 2005Mutual Information Proﬁleof a BISMC with ApplicationsJohannes Huber, Thorsten HehnInst. f. Information TransmissionUniv. of Erlangen, Germanye-mail: huber,hehn@LNT.deIngmar Land, Peter A. HoeherInformation & Coding Theory LabUniv. of Kiel, Germanye-mail: ph,il@tf.uni-kiel.deAbstract — It is shown how to decomposeany Binary Input Symmetric Memoryless Channel(BISMC) into BSCs. We explain why BEC and BSCare the two extreme cases of any BISMC with respectto the Mutual Information Proﬁle (MIP). Finally, thepaper points out useful applications for MIP.I. IntroductionBy introducing the Mutual Information Proﬁle (MIP) there isa new, alternative way of completely characterizing a BISMCwhich is equivalent to the traditional one given by the chan-nels transition probabilities but describes the behaviour of achannel in a more evident way and thus makes some theoremseasier to be proofed. Thinking in mutual information pro-ﬁles is especially interesting when talking about informationcombining.II. Decomposing BISMCsIt is shown that any Binary Input Symmetric MemorylessChannel (BISMC) can be decomposed into BSCs (see also[1]). A subchannel of a BISMC which is not further decompos-able has an output alphabet with one or two output symbols.This will be proofed by contradiction where we assume thatthe subchannel a has more than two diﬀerent output symbols.As the subchannel is strongly symmetric, the transition ma-trix has only two probabilities p1, p2 and all of its columnsare permutations of each other (note that there are only twodiﬀerent permutations). If p1 = p2, the transition matrixof the considered subchannel can be further decomposed intosingle columns, each of them corresponding to a BSC witherror probability 1/2. If p1 = p2 we group the two diﬀerentcolumns to submatrices and decompose the BISMC furtherinto channels corresponding to these matrices (decompositioninto BSCs). Decomposition of subchannel a is a contradictionto our assumption (q. e. d.).Along with the decomposition into BSCs we introduce theMIP wI(i), which is the probability density function of mutualinformation. Note that the average mutual information overthe BISMC I is given by the ﬁrst moment of the MIP.An AWGN channel with binary input ±1 is a BISMC andcan be decomposed using a = |y| as a subchannel-indicator (yis the continuous output signal). The pdf of a isfA(a) =1√2πσn(e− (a+1)22σ2n + e− (a−1)22σ2n), (1)where σ2n =12· N0Esis the noise power. The mutual in-formation I(a) of a subchannel and the MIP wI(i) are givenbyI(a) = 1− h(11 + e2a/σ2n)(2)wI(i) =fA(a)dI(a)/da∣∣∣∣a=I−1(i),dI(a)da=2 · e2a·log2(e2a)(1 + e2a)2(3)The ﬁgure shows MIPs for an AWGN channel at diﬀerentvalues 10 log10(Es/N0). For ﬁxed capacity BEC and BSC0 0.2 0.4 0.6 0.8 1012345678910i→wI(i)−4 dB 0 dB +5dBare the two extreme cases w. r. t. the MIP, having maximumor minimum (zero) second moment, respectively. Thus, BECand BSC are extreme cases for all bounds derived by means ofJensen’s inequalities, e. g. information combining [2, 3]. Usingthese extreme cases, a lot of bounds like Fano’s and Hellman-Raviv-bound can be derived in a very illustrative way. Forhigh SNR, the AWGN channel can be approximated very ac-curately by a BEC with equal capacity (see also the plot for10 log10(Es/N0) = 5 dB, well approximated by two Dirac δ-functions for i = 0 and i = 1).III. ApplicationsHigh-rate codes are used in high signal to noise ratio sce-narios. As for high SNR an AWGN channel can be well ap-proximated by a BEC, high-rate codes can be designed forBECs, e. g. by very eﬃcient density evolution and used forAWGN channels, like shown in the following table:Rate 0.2 0.5 0.99Rel. AWGN Cap. Gap 0.00582 0.00357 0.00078In [4] another case has been studied: Nonoptimized codesperform equally well for AWGN and BEC at any SNR.References[1] I. Land, “Reliability Information in Channel Decoding - Practi-cal Aspects and Information Theoretic Bounds”, Ph.D. Thesis,University of Kiel, Kiel, August 2004.[2] I. Land, S. Huettinger, P. A. Hoeher, and J. Huber, “Boundson Information Combining”, accepted for IEEE Transactionson Information Theory.[3] I. Land, S. Huettinger, P. A. Hoeher, and J. Huber, “Boundson mutual Information for simple Codes using information com-bining”, Annales des te´le´communications, vol. 60, no. 1-2, pp.191-221, 2005 (to appear)[4] M. Franceschini, G. Ferrari, and R. Raheli, “Does the Perfor-mance of LDPC Codes depend on the Channel?”, Proceedingson ISITA 2004, pp. 55-59, Parma, Italy, 2004.

Mutual Information Profile of a BISMC with Applications

Abstract

Similar works

Full text

Available Versions

VBN