23 research outputs found

    Cooperative Pursuit with Multi-Pursuer and One Faster Free-moving Evader

    Full text link
    This paper addresses a multi-pursuer single-evader pursuit-evasion game where the free-moving evader moves faster than the pursuers. Most of the existing works impose constraints on the faster evader such as limited moving area and moving direction. When the faster evader is allowed to move freely without any constraint, the main issues are how to form an encirclement to trap the evader into the capture domain, how to balance between forming an encirclement and approaching the faster evader, and what conditions make the capture possible. In this paper, a distributed pursuit algorithm is proposed to enable pursuers to form an encirclement and approach the faster evader. An algorithm that balances between forming an encirclement and approaching the faster evader is proposed. Moreover, sufficient capture conditions are derived based on the initial spatial distribution and the speed ratios of the pursuers and the evader. Simulation and experimental results on ground robots validate the effectiveness and practicability of the proposed method

    Toward multi-target self-organizing pursuit in a partially observable Markov game

    Full text link
    The multiple-target self-organizing pursuit (SOP) problem has wide applications and has been considered a challenging self-organization game for distributed systems, in which intelligent agents cooperatively pursue multiple dynamic targets with partial observations. This work proposes a framework for decentralized multi-agent systems to improve intelligent agents' search and pursuit capabilities. We model a self-organizing system as a partially observable Markov game (POMG) with the features of decentralization, partial observation, and noncommunication. The proposed distributed algorithm: fuzzy self-organizing cooperative coevolution (FSC2) is then leveraged to resolve the three challenges in multi-target SOP: distributed self-organizing search (SOS), distributed task allocation, and distributed single-target pursuit. FSC2 includes a coordinated multi-agent deep reinforcement learning method that enables homogeneous agents to learn natural SOS patterns. Additionally, we propose a fuzzy-based distributed task allocation method, which locally decomposes multi-target SOP into several single-target pursuit problems. The cooperative coevolution principle is employed to coordinate distributed pursuers for each single-target pursuit problem. Therefore, the uncertainties of inherent partial observation and distributed decision-making in the POMG can be alleviated. The experimental results demonstrate that distributed noncommunicating multi-agent coordination with partial observations in all three subtasks are effective, and 2048 FSC2 agents can perform efficient multi-target SOP with almost 100% capture rates

    Multiagent Cooperative Learning Strategies for Pursuit-Evasion Games

    Get PDF
    This study examines the pursuit-evasion problem for coordinating multiple robotic pursuers to locate and track a nonadversarial mobile evader in a dynamic environment. Two kinds of pursuit strategies are proposed, one for agents that cooperate with each other and the other for agents that operate independently. This work further employs the probabilistic theory to analyze the uncertain state information about the pursuers and the evaders and uses case-based reasoning to equip agents with memories and learning abilities. According to the concepts of assimilation and accommodation, both positive-angle and bevel-angle strategies are developed to assist agents in adapting to their environment effectively. The case study analysis uses the Recursive Porous Agent Simulation Toolkit (REPAST) to implement a multiagent system and demonstrates superior performance of the proposed approaches to the pursuit-evasion game

    Reachable sets analysis in the cooperative control of pursuer vehicles

    Full text link
    This thesis is concerned with the Pursuit-and-Evasion (PE) problem where the pursuer aims to minimize the time to capture the evader while the evader tries to prevent capture. In the problem, the evader has two advantages: a higher manoeuvrability and that the pursuer is uncertain about the evader's state. Cooperation among multiple pursuer vehicles can thus be used to overcome the evader’s advantages. The focus here is on the formulation and development of frameworks and algorithms for cooperation amongst pursuers, aiming at feasible implementation on real and autonomous vehicles. The thesis is split into Parts I and II. Part I considers the problem of capturing an evader of higher manoeuvrability in a deterministic PE game. The approach is the employment of Forward Reachable Set (FRS) analysis in the pursuers’ control. The analysis considers the coverage of the evader’s FRS, which is the set of reachable states at a future time, with the pursuer’s FRS and assumes that the chance of capturing the evader is dependent on the degree of the coverage. Using the union of multiple pursuers’ FRSs intuitively leads to more evader FRS coverage and this forms the mechanism of cooperation. A framework for cooperative control based on the FRS coverage, or FRS-based control, is proposed. Two control algorithms were developed within this framework. Part II additionally introduces the problem of evader state uncertainty due to noise and limited field-of-view of the pursuers’ sensors. A search-and-capture (SAC) problem is the result and a hybrid architecture, which includes multi-sensor estimation using the Particle Filter as well as FRS-based control, is proposed to accomplish the SAC task. The two control algorithms in Part I were tested in simulations against an optimal guidance algorithm. The results show that both algorithms yield a better performance in terms of time and miss distance. The results in Part II demonstrate the effectiveness of the hybrid architecture for the SAC task. The proposed frameworks and algorithms provide insights for the development of effective and more efficient control of pursuer vehicles and can be useful in the practical applications such as defence systems and civil law enforcement

    Multi-Agent Pursuit of a Faster Evader with Application to Unmanned Aerial Vehicles

    Get PDF
    Robotic applications like search and rescue missions, surveillance, police missions, patrolling, and warfare can all be modeled as a Pursuit-Evasion Game (PEG). Most of these tasks are multi-agent problems, often including a cooperation between team members and a conflict between adversarial teams. In order to realize such a situation with robots, two major problems have to be solved. Initially, a decomposition of the PEG has to be performed for getting results in reasonable time. Present embedded computers lack the computational power enabling them to process the highly complex solution algorithm of the non-decomposed game fast enough. Secondly, a framework has to be defined, enabling the computation of optimal actions for both the pursuers and the evaders when a cooperation within the teams is possible. It is intended to develop strategies, that allow the team of pursuers to capture a faster evader in a visibility-based PEG setup due to cooperation. For tackling the first problem a game structure is sought, aiming to considerably reduce the time complexity of the solution process. The first step is the decomposition of the action space, and the second is the change of the game structure itself. The latter is reached by defining a two-pursuer one-evader PEG with three different game structures, which are the Non-Decomposed Game, the Multiple Two-Player Game Decomposition (MTPGD) game, and the Team-Subsumption Two-Player Game (TSTPG). Several simulation results demonstrate, that both methods yield close results in respect to the full game. With increasing cardinality of each player’s strategy space, the MTPGD yields a relevant decrease of the run-time. Otherwise, the TSTPG does not minimize the time complexity, but enables the use of more sophisticated algorithms for two-player games, resulting in a decreased runtime. The cooperation within a team is enabled by introducing a hierarchical decomposition of the game. On a superordinate collaboration level, the pursuers choose their optimal behavioral strategy (e.g. pursuit and battue) resulting in the case of a two-pursuer one-evader PEG in a three-player noncooperative dynamic game, which is solved in a subordinate level of the overall game. This structure enables an intelligent behavior change for the pursuers based on game-theoretical solution methods. Depending on the state of the game, which behavioral strategy yields the best results for the pursuers within a predefined time horizon has to be evaluated. It is shown that the pursuer’s outcome can be improved by using a superordinate cooperation. Moreover, conditions are presented under which a capture of a faster evader by a group of two pursuers is possible in a visibility-based PEG with imperfect information. Since Unmanned Aerial Vehicles (UAVs) are increasingly a common platform used in the aforementioned applications, this work focuses only on PEGs with multi-rotor UAVs. Furthermore, the realization of the concepts in this thesis are applied on a real hex rotor. The feasibility of the approach is proven in experiments, while all implementations on the UAV are running in real-time. This framework provides a solution concept for all types of dynamic games with an 1-M or N-1 setup, that have a non-cooperative and cooperative nature. At this stage a N-M dynamic game is not applicable. Nevertheless, an approach to extend this framework to the N-M case is proposed in the last chapter of this work

    A Methodology to Enhance Quantitative Technology Evaluation Through Exploration of Employment Concepts in Engagement Analysis

    Get PDF
    The process of designing a new system has often been treated as a purely technological problem, where the infusion or synthesis of new technologies forms the basis of progress. However, recent trends in design and analysis methodologies have tried to shift away from the narrow scope of technology-centric approaches. One such trend is the increase in analysis scope from the level of an isolated system to that of multiple interacting systems. Analysis under this broader scope allows for the exploration of non-materiel solutions to existing or future problems. Solutions of this type can reduce the cost of closing capability gaps by mitigating the need to procure new systems to achieve desired levels of performance. In particular, innovations in the employment concepts can enhance existing, evolutionary, or revolutionary materiel solutions. The task of experimenting with non-materiel solutions often falls to operators after the system has been designed and produced. This begs the question as to whether the chosen design adequately accounted for the possibility of innovative employment concepts which operators might discover. Attempts can be made to bring the empirical knowledge possessed by skilled operators upstream in the design process. However, care must be taken to ensure such attempts do not introduce unwanted bias, and there can be significant difficulty in translating human intuition into an appropriate modeling paradigm for analysis. Furthermore, the capacity for human operators to capitalize on the potential benefits of a given technology may be limited or otherwise infeasible in design space explorations where the number of alternatives becomes very large. This is especially relevant to revolutionary concepts to which prior knowledge may not be applicable. Each of these complicating factors is exacerbated by interactions between systems, where changes in the decision-making processes of individual entities can greatly influence outcomes. This necessitates exploration and analysis of employment concepts for all relevant entities, not only that or those to which the technology applies. This research sought to address the issues of exploring employment concepts in the early phases of the system design process. A characterization of the problem identified several gaps in existing methodologies, particularly with respect to the representation, generation, and evaluation of alternative employment concepts. Relevant theories, including behavioral psychology, control theory, and game theory were identified to facilitate closure of these gaps. However, these theories also introduced technical challenges which had to be overcome. These challenges stemmed from systematic problems such as the curse of dimensionality, temporal credit assignment, and the complexities of entity interactions. A candidate approach was identified through thorough review of available literature: Multi-agent reinforcement learning. Experiments show the proposed approach can be used to generate highly effective models of behavior which could out-perform existing models on a representative problem. It was further shown that models produced by this new method can achieve consistently high levels of performance in competitive scenarios. Additional experimentation demonstrated how incorporation of design variables into the state space allowed models to learn policies which were effective across a continuous design space and outperformed their respective baselines. All of these results were obtained without reliance on prior knowledge, mitigating risks in and enhancing the capabilities of the analysis process. Lastly, the completed methodology was applied to the design of a fighter aircraft for one-on-one, gun-only air combat engagements to demonstrate its efficacy on and applicability to more complex problems.Ph.D

    Coordinating Team Tactics for Swarm-vs.-Swarm Adversarial Games

    Get PDF
    While swarms of UAVs have received much attention in the last few years, adversarial swarms (i.e., competitive, swarm-vs.-swarm games) have been less well studied. In this dissertation, I investigate the factors influential in team-vs.-team UAV aerial combat scenarios, elucidating the impacts of force concentration and opponent spread in the engagement space. Specifically, this dissertation makes the following contributions: (1) Tactical Analysis: Identifies conditions under which either explicitly-coordinating tactics or decentralized, greedy tactics are superior in engagements as small as 2-vs.-2 and as large as 10-vs.-10, and examines how these patterns change with the quality of the teams' weapons; (2) Coordinating Tactics: Introduces and demonstrates a deep-reinforcement-learning framework that equips agents to learn to use their own and their teammates' situational context to decide which pre-scripted tactics to employ in what situations, and which teammates, if any, to coordinate with throughout the engagement; the efficacy of agents using the neural network trained within this framework outperform baseline tactics in engagements against teams of agents employing baseline tactics in N-vs.-N engagements for N as small as two and as large as 64; and (3) Bio-Inspired Coordination: Discovers through Monte-Carlo agent-based simulations the importance of prioritizing the team's force concentration against the most threatening opponent agents, but also of preserving some resources by deploying a smaller defense force and defending against lower-penalty threats in addition to high-priority threats to maximize the remaining fuel within the defending team's fuel reservoir.Ph.D

    Collision Free Navigation of a Multi-Robot Team for Intruder Interception

    Full text link
    In this report, we propose a decentralised motion control algorithm for the mobile robots to intercept an intruder entering (k-intercepting) or escaping (e-intercepting) a protected region. In continuation, we propose a decentralized navigation strategy (dynamic-intercepting) for a multi-robot team known as predators to intercept the intruders or in the other words, preys, from escaping a siege ring which is created by the predators. A necessary and sufficient condition for the existence of a solution of this problem is obtained. Furthermore, we propose an intelligent game-based decision-making algorithm (IGD) for a fleet of mobile robots to maximize the probability of detection in a bounded region. We prove that the proposed decentralised cooperative and non-cooperative game-based decision-making algorithm enables each robot to make the best decision to choose the shortest path with minimum local information. Then we propose a leader-follower based collision-free navigation control method for a fleet of mobile robots to traverse an unknown cluttered environment where is occupied by multiple obstacles to trap a target. We prove that each individual team member is able to traverse safely in the region, which is cluttered by many obstacles with any shapes to trap the target while using the sensors in some indefinite switching points and not continuously, which leads to saving energy consumption and increasing the battery life of the robots consequently. And finally, we propose a novel navigation strategy for a unicycle mobile robot in a cluttered area with moving obstacles based on virtual field force algorithm. The mathematical proof of the navigation laws and the computer simulations are provided to confirm the validity, robustness, and reliability of the proposed methods
    corecore