On the numerical solution of a free end-time homicidal chauffeur game*

A functional formulation of the classical homicidal chauffeur Nash game is presented and a numerical framework for its solution is discussed. This methodology combines a Hamiltonian based scheme with proximal penalty to determine the time horizon where the game takes place with a Lagrangian optimal control approach and relaxation to solve the Nash game at a fixed end-time

An All-Against-One Game Approach for the Multi-Player Pursuit-Evasion Problem

The traditional pursuit-evasion game considers a situation where one pursuer tries to capture an evader, while the evader is trying to escape. A more general formulation of this problem is to consider multiple pursuers trying to capture one evader. This general multi-pursuer one-evader problem can also be used to model a system of systems in which one of the subsystems decides to dissent (evade) from the others while the others (the pursuer subsystems) try to pursue a strategy to prevent it from doing so. An important challenge in analyzing these types of problems is to develop strategies for the pursuers along with the advantages and disadvantages of each. In this thesis, we investigate three possible and conceptually different strategies for pursuers: (1) act non-cooperatively as independent pursuers, (2) act cooperatively as a unified team of pursuers, and (3) act individually as greedy pursuers. The evader, on the other hand, will consider strategies against all possible strategies by the pursuers. We assume complete uncertainty in the game i.e. no player knows which strategies the other players are implementing and none of them has information about any of the parameters in the objective functions of the other players. To treat the three pursuers strategies under one general framework, an all-against-one linear quadratic dynamic game is considered and the corresponding closed-loop Nash solution is discussed. Additionally, different necessary and sufficient conditions regarding the stability of the system, and existence and definiteness of the closed-loop Nash strategies under different strategy assumptions are derived. We deal with the uncertainties in the strategies by first developing the Nash strategies for each of the resulting games for all possible options available to both sides. Then we deal with the parameter uncertainties by performing a Monte Carlo analysis to determine probabilities of capture for the pursuers (or escape for the evader) for each resulting game. Results of the Monte Carlo simulation show that in general, pursuers do not always benefit from cooperating as a team and that acting as non-cooperating players may yield a higher probability of capturing of the evader

Dynamic network analysis of a target defense differential game with limited observations

In this paper, we study a Target-Attacker-Defender (TAD) differential game involving one attacker, one target and multiple defenders. We consider two variations where (a) the attacker and the target have unlimited observation range and the defenders are visibility constrained (b) only the attacker has unlimited observation range and the remaining players are visibility constrained. We model the players' interactions as a dynamic game with asymmetric information. Here, the visibility constraints of the players induce a visibility network which encapsulates the visibility information during the evolution of the game. Based on this observation, we introduce network adapted feedback or implementable strategies for visibility constrained players. Using inverse game theory approach we obtain network adapted feedback Nash equilibrium strategies. We introduce a consistency criterion for selecting a subset (or refinement) of network adapted feedback Nash strategies, and provide an optimization based approach for computing them. Finally, we illustrate our results with numerical experiments.Comment: 8 figure

Nash Strategies For Pursuit-Evasion Differential Games Involving Limited Observations

A linear-quadratic N-pursuer single-evader differential game is considered. The evader can observe all the pursuers but pursuers have limited observations of themselves and the evader. The evader implements the conventional feedback Nash strategy and the pursuers implement Nash strategies based on a novel concept of best achievable performance indices. This problem has potential applications in situations where a well-equipped unmanned vehicle is evading several weakly equipped pursuing vehicles. An illustrative example is solved, and several scenarios are presented

Large-scale Multi-agent Decision-making Using Mean Field Game Theory and Reinforcement Learning

The Multi-agent system (MAS) optimal control problem is a recently emerging research topic that benefits industries such as robotics, communication, and power systems. The traditional MAS control algorithms are developed by extending the single agent optimal controllers, requiring heavy information exchange. Moreover, the information exchanged within the MAS needs to be used to compute the optimal control resulting in the coupling between the computational complexity and the agent number. With the increasing need for large-scale MAS in practical applications, the existing MAS optimal control algorithms suffer from the ``curse of dimensionality" problem and limited communication resources. Therefore, a new type of MAS optimal control framework that features a decentralized and computational friendly decision process is desperately needed. To deal with the aforementioned problems, the mean field game theory is introduced to generate a decentralized optimal control framework named the Actor-critic-mass (ACM). Moreover, the ACM algorithm is improved by eliminating constraints such as homogeneous agents and cost functions. Finally, the ACM algorithm is utilized in two applications