2,916 research outputs found
Relative entropy via non-sequential recursive pair substitutions
The entropy of an ergodic source is the limit of properly rescaled 1-block
entropies of sources obtained applying successive non-sequential recursive
pairs substitutions (see P. Grassberger 2002 ArXiv:physics/0207023 and D.
Benedetto, E. Caglioti and D. Gabrielli 2006 Jour. Stat. Mech. Theo. Exp. 09
doi:10.1088/1742.-5468/2006/09/P09011). In this paper we prove that the cross
entropy and the Kullback-Leibler divergence can be obtained in a similar way.Comment: 13 pages , 2 figure
- …