Toward multi-target self-organizing pursuit in a partially observable Markov game

The multiple-target self-organizing pursuit (SOP) problem has wide applications and has been considered a challenging self-organization game for distributed systems, in which intelligent agents cooperatively pursue multiple dynamic targets with partial observations. This work proposes a framework for decentralized multi-agent systems to improve intelligent agents' search and pursuit capabilities. We model a self-organizing system as a partially observable Markov game (POMG) with the features of decentralization, partial observation, and noncommunication. The proposed distributed algorithm: fuzzy self-organizing cooperative coevolution (FSC2) is then leveraged to resolve the three challenges in multi-target SOP: distributed self-organizing search (SOS), distributed task allocation, and distributed single-target pursuit. FSC2 includes a coordinated multi-agent deep reinforcement learning method that enables homogeneous agents to learn natural SOS patterns. Additionally, we propose a fuzzy-based distributed task allocation method, which locally decomposes multi-target SOP into several single-target pursuit problems. The cooperative coevolution principle is employed to coordinate distributed pursuers for each single-target pursuit problem. Therefore, the uncertainties of inherent partial observation and distributed decision-making in the POMG can be alleviated. The experimental results demonstrate that distributed noncommunicating multi-agent coordination with partial observations in all three subtasks are effective, and 2048 FSC2 agents can perform efficient multi-target SOP with almost 100% capture rates

Intelligent Escape of Robotic Systems: A Survey of Methodologies, Applications, and Challenges

Intelligent escape is an interdisciplinary field that employs artificial intelligence (AI) techniques to enable robots with the capacity to intelligently react to potential dangers in dynamic, intricate, and unpredictable scenarios. As the emphasis on safety becomes increasingly paramount and advancements in robotic technologies continue to advance, a wide range of intelligent escape methodologies has been developed in recent years. This paper presents a comprehensive survey of state-of-the-art research work on intelligent escape of robotic systems. Four main methods of intelligent escape are reviewed, including planning-based methodologies, partitioning-based methodologies, learning-based methodologies, and bio-inspired methodologies. The strengths and limitations of existing methods are summarized. In addition, potential applications of intelligent escape are discussed in various domains, such as search and rescue, evacuation, military security, and healthcare. In an effort to develop new approaches to intelligent escape, this survey identifies current research challenges and provides insights into future research trends in intelligent escape.Comment: This paper is accepted by Journal of Intelligent and Robotic System

An All-Against-One Game Approach for the Multi-Player Pursuit-Evasion Problem

The traditional pursuit-evasion game considers a situation where one pursuer tries to capture an evader, while the evader is trying to escape. A more general formulation of this problem is to consider multiple pursuers trying to capture one evader. This general multi-pursuer one-evader problem can also be used to model a system of systems in which one of the subsystems decides to dissent (evade) from the others while the others (the pursuer subsystems) try to pursue a strategy to prevent it from doing so. An important challenge in analyzing these types of problems is to develop strategies for the pursuers along with the advantages and disadvantages of each. In this thesis, we investigate three possible and conceptually different strategies for pursuers: (1) act non-cooperatively as independent pursuers, (2) act cooperatively as a unified team of pursuers, and (3) act individually as greedy pursuers. The evader, on the other hand, will consider strategies against all possible strategies by the pursuers. We assume complete uncertainty in the game i.e. no player knows which strategies the other players are implementing and none of them has information about any of the parameters in the objective functions of the other players. To treat the three pursuers strategies under one general framework, an all-against-one linear quadratic dynamic game is considered and the corresponding closed-loop Nash solution is discussed. Additionally, different necessary and sufficient conditions regarding the stability of the system, and existence and definiteness of the closed-loop Nash strategies under different strategy assumptions are derived. We deal with the uncertainties in the strategies by first developing the Nash strategies for each of the resulting games for all possible options available to both sides. Then we deal with the parameter uncertainties by performing a Monte Carlo analysis to determine probabilities of capture for the pursuers (or escape for the evader) for each resulting game. Results of the Monte Carlo simulation show that in general, pursuers do not always benefit from cooperating as a team and that acting as non-cooperating players may yield a higher probability of capturing of the evader

Surveillance of a Faster Fixed-Course Target

The maximum surveillance of a target which is holding course is considered, wherein an observer vehicle aims to maximize the time that a faster target remains within a fixed-range of the observer. This entails two coupled phases: an approach phase and observation phase. In the approach phase, the observer strives to make contact with the faster target, such that in the observation phase, the observer is able to maximize the time where the target remains within range. Using Pontryagin's Minimum Principle, the optimal control laws for the observer are found in closed-form. Example scenarios highlight various aspects of the engagement.Comment: 12 pages, 8 figure