1 research outputs found
Value Iteration for Simple Stochastic Games: Stopping Criterion and Learning Algorithm
Simple stochastic games can be solved by value iteration (VI), which yields a
sequence of under-approximations of the value of the game. This sequence is
guaranteed to converge to the value only in the limit. Since no stopping
criterion is known, this technique does not provide any guarantees on its
results. We provide the first stopping criterion for VI on simple stochastic
games. It is achieved by additionally computing a convergent sequence of
over-approximations of the value, relying on an analysis of the game graph.
Consequently, VI becomes an anytime algorithm returning the approximation of
the value and the current error bound. As another consequence, we can provide a
simulation-based asynchronous VI algorithm, which yields the same guarantees,
but without necessarily exploring the whole game graph.Comment: CAV201