4 research outputs found
Approximation of Markov Processes by Lower Dimensional Processes via Total Variation Metrics
The aim of this paper is to approximate a finite-state Markov process by
another process with fewer states, called herein the approximating process. The
approximation problem is formulated using two different methods.
The first method, utilizes the total variation distance to discriminate the
transition probabilities of a high dimensional Markov process and a reduced
order Markov process. The approximation is obtained by optimizing a linear
functional defined in terms of transition probabilities of the reduced order
Markov process over a total variation distance constraint. The transition
probabilities of the approximated Markov process are given by a water-filling
solution.
The second method, utilizes total variation distance to discriminate the
invariant probability of a Markov process and that of the approximating
process. The approximation is obtained via two alternative formulations: (a)
maximizing a functional of the occupancy distribution of the Markov process,
and (b) maximizing the entropy of the approximating process invariant
probability. For both formulations, once the reduced invariant probability is
obtained, which does not correspond to a Markov process, a further
approximation by a Markov process is proposed which minimizes the
Kullback-Leibler divergence. These approximations are given by water-filling
solutions.
Finally, the theoretical results of both methods are applied to specific
examples to illustrate the methodology, and the water-filling behavior of the
approximations.Comment: 38 pages, 17 figures, submitted to IEEE-TA