Robust policy iteration for continuous-time stochastic H∞H_\infty control problem with unknown dynamics

In this article, we study a continuous-time stochastic $H_\infty$ control problem based on reinforcement learning (RL) techniques that can be viewed as solving a stochastic linear-quadratic two-person zero-sum differential game (LQZSG). First, we propose an RL algorithm that can iteratively solve stochastic game algebraic Riccati equation based on collected state and control data when all dynamic system information is unknown. In addition, the algorithm only needs to collect data once during the iteration process. Then, we discuss the robustness and convergence of the inner and outer loops of the policy iteration algorithm, respectively, and show that when the error of each iteration is within a certain range, the algorithm can converge to a small neighborhood of the saddle point of the stochastic LQZSG problem. Finally, we applied the proposed RL algorithm to two simulation examples to verify the effectiveness of the algorithm

An iterative algorithm to solve state-perturbed stochastic algebraic Riccati equations in LQ zero-sum games

An iterative algorithm to solve a kind of state-perturbed stochastic algebraic Riccati equation (SARE) in LQ zero-sum game problems is proposed. In our algorithm, we replace the problem of solving a SARE with an indefinite quadratic term by the problem of solving a sequence of SAREs with a negative semidefinite quadratic term, which can be solved by existing methods. Under some appropriate conditions, we prove that our algorithm is globally convergent. We give a numerical example to show the effectiveness of our algorithm. Our algorithm also has a natural game theoretic interpretation