3 research outputs found
Entropy-Regularized Stochastic Games
In zero-sum stochastic games, where two competing players make decisions under uncertainty, a pair of optimal strategies is traditionally described by Nash equilibrium and computed under the assumption that the players have perfect information about the stochastic transition model of the environment. However, implementing such strategies may make the players vulnerable to unforeseen changes in the environment. In this paper, we introduce entropy-regularized stochastic games where each player aims to maximize the causal entropy of its strategy in addition to its expected payoff. The regularization term balances each player's rationality with its belief about the level of misinformation about the transition model. We consider both entropy-regularized N-stage and entropy-regularized discounted stochastic games, and establish the existence of a value in both games. Moreover, we prove the sufficiency of Markovian and stationary mixed strategies to attain the value, respectively, in N-stage and discounted games. Finally, we present algorithms, which are based on convex optimization problems, to compute the optimal strategies. In a numerical example, we demonstrate the proposed method on a motion planning scenario and illustrate the effect of the regularization term on the expected payoff
Entropy-Regularized Stochastic Games
In two-player zero-sum stochastic games, where two competing players make
decisions under uncertainty, a pair of optimal strategies is traditionally
described by Nash equilibrium and computed under the assumption that the
players have perfect information about the stochastic transition model of the
environment. However, implementing such strategies may make the players
vulnerable to unforeseen changes in the environment. In this paper, we
introduce entropy-regularized stochastic games where each player aims to
maximize the causal entropy of its strategy in addition to its expected payoff.
The regularization term balances each player's rationality with its belief
about the level of misinformation about the transition model. We consider both
entropy-regularized -stage and entropy-regularized discounted stochastic
games, and establish the existence of a value in both games. Moreover, we prove
the sufficiency of Markovian and stationary mixed strategies to attain the
value, respectively, in -stage and discounted games. Finally, we present
algorithms, which are based on convex optimization problems, to compute the
optimal strategies. In a numerical example, we demonstrate the proposed method
on a motion planning scenario and illustrate the effect of the regularization
term on the expected payoff.Comment: Corrected typo
Entropy-Regularized Stochastic Games
In zero-sum stochastic games, where two competing players make decisions under uncertainty, a pair of optimal strategies is traditionally described by Nash equilibrium and computed under the assumption that the players have perfect information about the stochastic transition model of the environment. However, implementing such strategies may make the players vulnerable to unforeseen changes in the environment. In this paper, we introduce entropy-regularized stochastic games where each player aims to maximize the causal entropy of its strategy in addition to its expected payoff. The regularization term balances each player's rationality with its belief about the level of misinformation about the transition model. We consider both entropy-regularized N-stage and entropy-regularized discounted stochastic games, and establish the existence of a value in both games. Moreover, we prove the sufficiency of Markovian and stationary mixed strategies to attain the value, respectively, in N-stage and discounted games. Finally, we present algorithms, which are based on convex optimization problems, to compute the optimal strategies. In a numerical example, we demonstrate the proposed method on a motion planning scenario and illustrate the effect of the regularization term on the expected payoff