Pursuit-Evasion Games and Zero-sum Two-person Differential Games

International audienceDifferential games arose from the investigation, by Rufus Isaacs in the 50's, of pursuit-evasion problems. In these problems, closed-loop strategies are of the essence, although defining what is exactly meant by this phrase, and what is the Value of a differential game, is difficult. For closed-loop strategies, there is no such thing as a " two-sided Maximum Principle " , and one must resort to the analysis of Isaacs' equation, a Hamilton Jacobi equation. The concept of viscosity solutions of Hamilton-Jacobi equations has helped solve several of these issues

Nonlinear zero-sum differential game analysis by singular perturbation methods

A class of nonlinear, zero-sum differential games, exhibiting time-scale separation properties, can be analyzed by singular-perturbation techniques. The merits of such an analysis, leading to an approximate game solution, as well as the 'well-posedness' of the formulation, are discussed. This approach is shown to be attractive for investigating pursuit-evasion problems; the original multidimensional differential game is decomposed to a 'simple pursuit' (free-stream) game and two independent (boundary-layer) optimal-control problems. Using multiple time-scale boundary-layer models results in a pair of uniformly valid zero-order composite feedback strategies. The dependence of suboptimal strategies on relative geometry and own-state measurements is demonstrated by a three dimensional, constant-speed example. For game analysis with realistic vehicle dynamics, the technique of forced singular perturbations and a variable modeling approach is proposed. Accuracy of the analysis is evaluated by comparison with the numerical solution of a time-optimal, variable-speed 'game of two cars' in the horizontal plane

A formulation and analysis of combat games

Combat which is formulated as a dynamical encounter between two opponents, each of whom has offensive capabilities and objectives is outlined. A target set is associated with each opponent in the event space in which he endeavors to terminate the combat, thereby winning. If the combat terminates in both target sets simultaneously, or in neither, a joint capture or a draw, respectively, occurs. Resolution of the encounter is formulated as a combat game; as a pair of competing event constrained differential games. If exactly one of the players can win, the optimal strategies are determined from a resulting constrained zero sum differential game. Otherwise the optimal strategies are computed from a resulting nonzero sum game. Since optimal combat strategies may frequently not exist, approximate or delta combat games are also formulated leading to approximate or delta optimal strategies. The turret game is used to illustrate combat games. This game is sufficiently complex to exhibit a rich variety of combat behavior, much of which is not found in pursuit evasion games

A formulation and analysis of combat games

Combat is formulated as a dynamical encounter between two opponents, each of whom has offensive capabilities and objectives. With each opponent is associated a target in the event space in which he endeavors to terminate the combat, thereby winning. If the combat terminates in both target sets simultaneously or in neither, a joint capture or a draw, respectively, is said to occur. Resolution of the encounter is formulated as a combat game; namely, as a pair of competing event-constrained differential games. If exactly one of the players can win, the optimal strategies are determined from a resulting constrained zero-sum differential game. Otherwise the optimal strategies are computed from a resulting non-zero-sum game. Since optimal combat strategies frequencies may not exist, approximate of delta-combat games are also formulated leading to approximate or delta-optimal strategies. To illustrate combat games, an example, called the turret game, is considered. This game may be thought of as a highly simplified model of air combat, yet it is sufficiently complex to exhibit a rich variety of combat behavior, much of which is not found in pursuit-evasion games

Linear-quadratic stochastic pursuit-evasion games

A linear-quadratic differential game in which the system state is affected by disturbance and both players have access to different measurements is solved. The problem is first converted to an optimization problem in infinite-dimensional state space and then solved using standard techniques. For convenience, “L2-white noise” instead of “Wiener process” setup is used

Tauberian theorem for value functions

For two-person dynamic zero-sum games (both discrete and continuous settings), we investigate the limit of value functions of finite horizon games with long run average cost as the time horizon tends to infinity and the limit of value functions of $\lambda$-discounted games as the discount tends to zero. We prove that the Dynamic Programming Principle for value functions directly leads to the Tauberian Theorem---that the existence of a uniform limit of the value functions for one of the families implies that the other one also uniformly converges to the same limit. No assumptions on strategies are necessary. To this end, we consider a mapping that takes each payoff to the corresponding value function and preserves the sub- and super- optimality principles (the Dynamic Programming Principle). With their aid, we obtain certain inequalities on asymptotics of sub- and super- solutions, which lead to the Tauberian Theorem. In particular, we consider the case of differential games without relying on the existence of the saddle point; a very simple stochastic game model is also considered

Risk-Minimizing Two-Player Zero-Sum Stochastic Differential Game via Path Integral Control

This paper addresses a continuous-time risk-minimizing two-player zero-sum stochastic differential game (SDG), in which each player aims to minimize its probability of failure. Failure occurs in the event when the state of the game enters into predefined undesirable domains, and one player's failure is the other's success. We derive a sufficient condition for this game to have a saddle-point equilibrium and show that it can be solved via a Hamilton-Jacobi-Isaacs (HJI) partial differential equation (PDE) with Dirichlet boundary condition. Under certain assumptions on the system dynamics and cost function, we establish the existence and uniqueness of the saddle-point of the game. We provide explicit expressions for the saddle-point policies which can be numerically evaluated using path integral control. This allows us to solve the game online via Monte Carlo sampling of system trajectories. We implement our control synthesis framework on two classes of risk-minimizing zero-sum SDGs: a disturbance attenuation problem and a pursuit-evasion game. Simulation studies are presented to validate the proposed control synthesis framework.Comment: 8 pages, 4 figures, CDC 202