The Asymptotic Redundancy of Bayes Rules for Markov Chains

Abstract

Abstract-- We derive the asymptotics of the redundancy of Bayes rules for Markov chains with known order, extending the work of Barron and Clarke[6, 5] on i.i.d. sources. These asymptotics are derived when the actual source is in the class of OE-mixing sources which includes Markov chains and functions of Markov chains. These results can be used to derive minimax asymptotic rates of convergence for universal codes when a Markov chain of known order is used as a model. Index terms-- universal coding, Markov chains, Bayesian statistics, asymptotics. 1 Introduction Given data generated by a known stochastic process, methods of encoding the data to achieve the minimal average coding length, such as Huffman and arithmetic coding, are known[7]. Universal codes[15, 8] encode data such that, asymptotically, the average per-symbol code length is equal to its minimal value (the entropy rate) for any source within a wide class. For the well-known Lempel-Ziv code, the average per-symbol code l..