This paper considers the two-person zero-sum Markov game with finite state and action spaces at the criterion of average reward per unit time. For two types of Markov games, the communicating game and the simply connected game, it is shown that the method of successive approximations provides good bounds on the value of the game and nearly-optimal stationary strategies for the two players
