Finite-Step Algorithms for Single-Controller and Perfect Information Stochastic Games

Abstract. After a brief survey of iterative algorithms for general stochas-tic games, we concentrate on finite-step algorithms for two special classes of stochastic games. They are Single-Controller Stochastic Games and Per-fect Information Stochastic Games. In the case of single-controller games, the transition probabilities depend on the actions of the same player in all states. In perfect information stochastic games, one of the players has exactly one action in each state. Single-controller zero-sum games are effi-ciently solved by linear programming. Non-zero-sum single-controller stochastic games are reducible to linear complementary problems (LCP). In the discounted case they can be modified to fit into the so-called LCPs of Eave’s class L. In the undiscounted case the LCP’s are reducible to Lemke’s copositive plus class. In either case Lemke’s algorithm can be used to find a Nash equilibrium. In the case of discounted zero-sum perfect informa-tion stochastic games, a policy improvement algorithm is presented. Many other classes of stochastic games with orderfield property still await efficient finite-step algorithms. 1

Candidate to Job Allocation Problem with a Lexicographic Objective

Allocation of candidates to jobs is required in a situation where each candidate can only be allocated to certain jobs, and every job's "length" decreases as more candidates are allocated to it. The objective is to minimize the "ranked length vector," which is derived by arranging the job-lengths in nonincreasing order, in the lexicographic sense. Necessary and sufficient conditions for optimality are derived, and an algorithm for obtaining optimal allocations is presented, along with computational experience.networks/graphs, programming: multiple criteria

Algorithms for Stochastic Games with Geometrical Interpretation

The paper presents a new approach, based on analysis and geometrical interpretation, to the solution of Markov stochastic games. The proposed algorithm, using iterations in policy space, turns out to be a Newton-Raphson type procedure. Several numerical examples are given, covering the terminating and non-terminating cases respectively and illustrating the advantages of the proposed algorithm compared with other known algorithms. Special attention is given to Howard's sequential decision problem with discrete and continuous policy spaces.