research

AN ANALYTICAL METHOD TO CALCULATE THE ERGODIC AND DIFFERENCE MATRICES OF THE DISCOUNTED MARKOV DECISION PROCESSES

Abstract

In the paper, a theorem about the existence of the ergodic and difference matrices of the finite state discounted Markov decision processes had been formulated and proved. On the basis of this theorem an analytical method to calculate these matrices is presented. The theorem allows the distribution of the overall value into two parts: the so-called "constant" part, which represents a part of the value related to the ergodic matrix and the "variable" part which represents a sum of the difference matrices. On the basis of the mentioned analytical method a new performance index to the discounted optimal control Markov problem is proposed and some interpretation of the received results is given. The proposed new performance index is formulated as a quotient of the distinguished parts of the overall value. The method is illustrated by two simple examples.

Similar works

Full text

thumbnail-image

Research Papers in Economics

Provided a free PDF
Last time updated on 7/6/2012View original full text link

This paper was published in Research Papers in Economics.

Having an issue?

Is data on this page outdated, violates copyrights or anything else? Report the problem now and we will take corresponding actions after reviewing your request.