Thesis (Ph.D.)--University of Adelaide, School of Mathematical Sciences, Discipline of Applied Mathematics, 200

Glus̆c̆ević, Vido

Adelaide Research & Scholarship

OFz2(onThe University of AdelaideSchool of Applied MathematicsFaculty of Engineering, Computer and Mathematical SciencesVido GlusðevióApproximation for Csiszár/-divergenceThesis submitted for the degree ofDoctor of Philosophyat The University of AdelaideAdelaide, March, 2004.CONTENTSSigned statement .AcknowledgementAbstract1 Introduction2 /-divergence2.I Introduction2.2 Properties of /-divergence2.3 Particular instances of Csiszár /-divergence measures2.4 Some inequalities for /-divergence measures3 Shannon entropy and mutual information3.1 Shannon entropy3.1.1 Introduction .3.I.2 Properties of Shannon entropy3.1.3 Inequalities reiated to Shannon's entropy3.2 Mutual information4 Inequalities required4.L Ostrowski integral inequality for an absolutely continuous function4.2 Ostrowski type inequality for mappings of bounded variation .4.3 Diaz and Metcalf inequalitY4.4 Iyengar inequality4.5 Thapezoid inequality4.6 Generalisedtriangle inequality5 Approximation for Csiszár /-divergence5.1 Introduction5.2 Csiszár /-divergence, Ostrowski's inequality and mutual information5.2.I Introduction .5.2.2 An inequality for Csisz¿ír /-divergence5.2.3 Applications to some common divergence mea,sures5.2.4 Some numerical examPlesIiiiii244468IIII1316L717192I2l222223232323242737CO]VTE]VTS 2373838394346545.35.45.2.5 Application to mutual informationApproximations for Csiszár /-divergence via midpoint inequalities5.3.1 Introduction .5.3.2 Preliminaries5.3.3 Basic results5.3.4 Applications to some common divergence measures5.3.5 Application to mutual informationThe approximations for Csiszár /-divergence for absolutely continuousmapplngs5.4.I Introduction .5.4.2 Basic theorems5.4.3 Applications to some common divergence measuresThe approximation of csiszár /-divergence for mappings of boundedvariation5.5.1 Introduction5.5.2 Main results5.5.3 Application to some common divergence mea"sures5.56 References545454É. ¡-7(Jt6464656774123456taI8I1011t213T4LIST OF FIGURESKullback-Leibler distance - Theorem 1 . .Negative Kultback-Leibler distance - Theorem 1 . . . .Hellinger discriminationVariational distance - Theorem 1' .Tliangular discrimination - Theorem LKullback-Leibler distance - Theorem 2 and 3' . . . .Negative Kultback-Leibler distance - Theorem 2 and 3Hellinger discrimination - Theorem 2 and 3Renyi distance - Theorem 2 and 3 . . .Kullback-Leiblerdistance-Theorem4andS ' . ' . . .Negative Kullback-Leibler distance - Theorem 4 and 5 'Helinger discrimination - Theorem 4 and 5Kullback-Leibler distance - Theorem 6Variational distance - Theorem 63233343536505152536162637273SIGNEDThis work contains no material which has been accepted for the award of any otherdegree or diploma in any University or other tertiary institution and, to the best of-f kno*l"dge and beiief, contains no material previously published or written byanother person' except where due reference has been made in the text'I consent to this copy of my thesis, when deposited in the University Library,being available for loan and photocopying.SIGNED ll,*/, ,0 ...DATEACKNO\ATLEDGEMENTMy deepest appreciation go to my supervisors, Prof. Charles Pearce and Prof' SeverDiagomir, whose guidance, encouragement and intellectual support have sustainedme throughout creation of this thesis.This doctorate would not have been possible without the continued supportof my wife Jadranka and my daughter Matea.SUMMARYA variety of divergences have been introduced in the literature for measuring thedifference between two probability distributions. A unifying concept is the Csiszár/-divergence, which subsumes many of these as special cases.In many applications, we have only limited information about the distributionsconcerned. In this thesis we address this issue in the case of distributions with finitesupport. We make substantial use of the theory of inequalities, in particular, theDiaz-Metcalf inequality, Iyengar's inequality and the trapezoidal inequaiity'Our main results are approximations for Csiszár /-divergence, with applicationto mutual information. Special cases for various choices of the function / are aiso ofinterest. A number of our results are considerable improvements on existing results,in one case being eight times sharper than the best pre-existing result.Some of the Ãaterial, in the thesis has been published in seven papers [3S]- [44].

Approximation for Csiszár f-divergence

https://digital.library.adelaide.edu.au/dspace/bitstream/2440/103206/2/01front.pdf

Approximation for Csiszár f-divergence

Abstract

Similar works

Full text

Available Versions

Adelaide Research & Scholarship