Information-Preserving Markov Aggregation

We present a sufficient condition for a non-injective function of a Markov chain to be a second-order Markov chain with the same entropy rate as the original chain. This permits an information-preserving state space reduction by merging states or, equivalently, lossless compression of a Markov source on a sample-by-sample basis. The cardinality of the reduced state space is bounded from below by the node degrees of the transition graph associated with the original Markov chain. We also present an algorithm listing all possible information-preserving state space reductions, for a given transition graph. We illustrate our results by applying the algorithm to a bi-gram letter model of an English text.Comment: 7 pages, 3 figures, 2 table

Optimal Kullback-Leibler Aggregation via Information Bottleneck

In this paper, we present a method for reducing a regular, discrete-time Markov chain (DTMC) to another DTMC with a given, typically much smaller number of states. The cost of reduction is defined as the Kullback-Leibler divergence rate between a projection of the original process through a partition function and a DTMC on the correspondingly partitioned state space. Finding the reduced model with minimal cost is computationally expensive, as it requires an exhaustive search among all state space partitions, and an exact evaluation of the reduction cost for each candidate partition. Our approach deals with the latter problem by minimizing an upper bound on the reduction cost instead of minimizing the exact cost; The proposed upper bound is easy to compute and it is tight if the original chain is lumpable with respect to the partition. Then, we express the problem in the form of information bottleneck optimization, and propose using the agglomerative information bottleneck algorithm for searching a sub-optimal partition greedily, rather than exhaustively. The theory is illustrated with examples and one application scenario in the context of modeling bio-molecular interactions.Comment: 13 pages, 4 figure

Markov State Space Aggregation via the Information Bottleneck Method

Consider the problem of approximating a Markov chain by another Markov chain with a smaller state space that is obtained by partitioning the original state space. An information-theoretic cost function is proposed that is based on the relative entropy rate between the original Markov chain and a Markov chain defined by the partition. The state space aggregation problem can be sub-optimally solved by using the information bottleneck method

Метод и оценка сложности вычисления предельного распределения марковских функций, представляемых укрупненными цепями Маркова

Предложен метод вычисления предельного распределения марковских функций из класса укрупненных цепей Маркова, уменьшающий вычислительную сложность по сравнению с известным методом. Дана сравнительная оценка вычислительной сложности

An aggregation method for large-scale dynamic games

It is a well known fact that many dynamic games are subject to the curse of dimensionality, limiting the ability to use them in the study of real-world problems. I propose a new method to solve complex large-scale dynamic games using aggregation as an approximate solution. I obtain two fundamental characterization results. First, approximations with small within-state variation in the primitives have a smaller maximum error bound. I provide numerical results which compare the exact errors and the bound. Second, I find that for monotone games, order preserving aggregation is a necessary condition of any optimal aggregation. I suggest using quantiles as a straightforward implementation of an order preserving aggregation architecture for industry distributions. I conclude with an illustration, by solving and estimating a stylized dynamic reputation game for the hotel industry. Simulation results show maximal errors between the exact and approximated solutions below 6%, with average errors below 1%.publishersversionpublishe

Lumpings of Markov chains, entropy rate preservation, and higher-order lumpability

A lumping of a Markov chain is a coordinate-wise projection of the chain. We characterise the entropy rate preservation of a lumping of an aperiodic and irreducible Markov chain on a finite state space by the random growth rate of the cardinality of the realisable preimage of a finite-length trajectory of the lumped chain and by the information needed to reconstruct original trajectories from their lumped images. Both are purely combinatorial criteria, depending only on the transition graph of the Markov chain and the lumping function. A lumping is strongly k-lumpable, iff the lumped process is a k-th order Markov chain for each starting distribution of the original Markov chain. We characterise strong k-lumpability via tightness of stationary entropic bounds. In the sparse setting, we give sufficient conditions on the lumping to both preserve the entropy rate and be strongly k-lumpable