5,803 research outputs found
Controlled diffusion processes
This article gives an overview of the developments in controlled diffusion
processes, emphasizing key results regarding existence of optimal controls and
their characterization via dynamic programming for a variety of cost criteria
and structural assumptions. Stochastic maximum principle and control under
partial observations (equivalently, control of nonlinear filters) are also
discussed. Several other related topics are briefly sketched.Comment: Published at http://dx.doi.org/10.1214/154957805100000131 in the
Probability Surveys (http://www.i-journals.org/ps/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Policy iteration for perfect information stochastic mean payoff games with bounded first return times is strongly polynomial
Recent results of Ye and Hansen, Miltersen and Zwick show that policy
iteration for one or two player (perfect information) zero-sum stochastic
games, restricted to instances with a fixed discount rate, is strongly
polynomial. We show that policy iteration for mean-payoff zero-sum stochastic
games is also strongly polynomial when restricted to instances with bounded
first mean return time to a given state. The proof is based on methods of
nonlinear Perron-Frobenius theory, allowing us to reduce the mean-payoff
problem to a discounted problem with state dependent discount rate. Our
analysis also shows that policy iteration remains strongly polynomial for
discounted problems in which the discount rate can be state dependent (and even
negative) at certain states, provided that the spectral radii of the
nonnegative matrices associated to all strategies are bounded from above by a
fixed constant strictly less than 1.Comment: 17 page
Hypergraph conditions for the solvability of the ergodic equation for zero-sum games
The ergodic equation is a basic tool in the study of mean-payoff stochastic
games. Its solvability entails that the mean payoff is independent of the
initial state. Moreover, optimal stationary strategies are readily obtained
from its solution. In this paper, we give a general sufficient condition for
the solvability of the ergodic equation, for a game with finite state space but
arbitrary action spaces. This condition involves a pair of directed hypergraphs
depending only on the ``growth at infinity'' of the Shapley operator of the
game. This refines a recent result of the authors which only applied to games
with bounded payments, as well as earlier nonlinear fixed point results for
order preserving maps, involving graph conditions.Comment: 6 pages, 1 figure, to appear in Proc. 54th IEEE Conference on
Decision and Control (CDC 2015
- …