23 research outputs found
Cooperative Pursuit with Multi-Pursuer and One Faster Free-moving Evader
This paper addresses a multi-pursuer single-evader pursuit-evasion game where
the free-moving evader moves faster than the pursuers. Most of the existing
works impose constraints on the faster evader such as limited moving area and
moving direction. When the faster evader is allowed to move freely without any
constraint, the main issues are how to form an encirclement to trap the evader
into the capture domain, how to balance between forming an encirclement and
approaching the faster evader, and what conditions make the capture possible.
In this paper, a distributed pursuit algorithm is proposed to enable pursuers
to form an encirclement and approach the faster evader. An algorithm that
balances between forming an encirclement and approaching the faster evader is
proposed. Moreover, sufficient capture conditions are derived based on the
initial spatial distribution and the speed ratios of the pursuers and the
evader. Simulation and experimental results on ground robots validate the
effectiveness and practicability of the proposed method
Toward multi-target self-organizing pursuit in a partially observable Markov game
The multiple-target self-organizing pursuit (SOP) problem has wide
applications and has been considered a challenging self-organization game for
distributed systems, in which intelligent agents cooperatively pursue multiple
dynamic targets with partial observations. This work proposes a framework for
decentralized multi-agent systems to improve intelligent agents' search and
pursuit capabilities. We model a self-organizing system as a partially
observable Markov game (POMG) with the features of decentralization, partial
observation, and noncommunication. The proposed distributed algorithm: fuzzy
self-organizing cooperative coevolution (FSC2) is then leveraged to resolve the
three challenges in multi-target SOP: distributed self-organizing search (SOS),
distributed task allocation, and distributed single-target pursuit. FSC2
includes a coordinated multi-agent deep reinforcement learning method that
enables homogeneous agents to learn natural SOS patterns. Additionally, we
propose a fuzzy-based distributed task allocation method, which locally
decomposes multi-target SOP into several single-target pursuit problems. The
cooperative coevolution principle is employed to coordinate distributed
pursuers for each single-target pursuit problem. Therefore, the uncertainties
of inherent partial observation and distributed decision-making in the POMG can
be alleviated. The experimental results demonstrate that distributed
noncommunicating multi-agent coordination with partial observations in all
three subtasks are effective, and 2048 FSC2 agents can perform efficient
multi-target SOP with almost 100% capture rates
Multiagent Cooperative Learning Strategies for Pursuit-Evasion Games
This study examines the pursuit-evasion problem for coordinating multiple robotic pursuers to locate and track a nonadversarial mobile evader in a dynamic environment. Two kinds of pursuit strategies are proposed, one for agents that cooperate with each other and the other for agents that operate independently. This work further employs the probabilistic theory to analyze the uncertain state information about the pursuers and the evaders and uses case-based reasoning to equip agents with memories and learning abilities. According to the concepts of assimilation and accommodation, both positive-angle and bevel-angle strategies are developed to assist agents in adapting to their environment effectively. The case study analysis uses the Recursive Porous Agent Simulation Toolkit (REPAST) to implement a multiagent system and demonstrates superior performance of the proposed approaches to the pursuit-evasion game
Reachable sets analysis in the cooperative control of pursuer vehicles
This thesis is concerned with the Pursuit-and-Evasion (PE) problem where the pursuer aims to minimize the time to capture the evader while the evader tries to prevent capture. In the problem, the evader has two advantages: a higher manoeuvrability and that the pursuer is uncertain about the evader's state. Cooperation among multiple pursuer vehicles can thus be used to overcome the evader’s advantages. The focus here is on the formulation and development of frameworks and algorithms for cooperation amongst pursuers, aiming at feasible implementation on
real and autonomous vehicles.
The thesis is split into Parts I and II. Part I considers the problem of capturing an evader of higher manoeuvrability in a deterministic PE game. The approach is the employment of Forward Reachable Set (FRS) analysis in the pursuers’
control. The analysis considers the coverage of the evader’s FRS, which is the set of reachable states at a future time, with the pursuer’s FRS and assumes that the chance of capturing the evader is dependent on the degree of the coverage. Using the union of multiple pursuers’ FRSs intuitively leads to more evader FRS coverage and this forms the mechanism of cooperation. A framework for cooperative control based on the FRS coverage, or FRS-based control, is proposed. Two control algorithms were developed within this framework. Part II additionally introduces the problem of evader state uncertainty due to noise and limited field-of-view of the pursuers’ sensors. A search-and-capture (SAC) problem is the result and a hybrid architecture, which includes multi-sensor estimation using the Particle Filter as well as FRS-based control, is proposed to accomplish the SAC task.
The two control algorithms in Part I were tested in simulations against an optimal guidance algorithm. The results show that both algorithms yield a better performance in terms of time and miss distance. The results in Part II demonstrate the effectiveness of the hybrid architecture for the SAC task. The proposed frameworks and algorithms provide insights for the development of effective and more efficient control of pursuer vehicles and can be useful in the practical applications such as defence systems and civil law enforcement
Multi-Agent Pursuit of a Faster Evader with Application to Unmanned Aerial Vehicles
Robotic applications like search and rescue missions, surveillance, police missions, patrolling, and warfare can all be modeled as a Pursuit-Evasion Game
(PEG). Most of these tasks are multi-agent problems, often including a cooperation between team members and a conflict between adversarial teams.
In order to realize such a situation with robots, two major problems have
to be solved. Initially, a decomposition of the PEG has to be performed for
getting results in reasonable time. Present embedded computers lack the
computational power enabling them to process the highly complex solution
algorithm of the non-decomposed game fast enough. Secondly, a framework
has to be defined, enabling the computation of optimal actions for both the
pursuers and the evaders when a cooperation within the teams is possible. It
is intended to develop strategies, that allow the team of pursuers to capture
a faster evader in a visibility-based PEG setup due to cooperation.
For tackling the first problem a game structure is sought, aiming to considerably reduce the time complexity of the solution process. The first step is the decomposition of the action space, and the second is the change of the game structure itself. The latter is reached by defining a two-pursuer one-evader PEG with three different game structures, which are the Non-Decomposed Game, the Multiple Two-Player Game Decomposition (MTPGD) game, and the Team-Subsumption Two-Player Game (TSTPG). Several simulation results demonstrate, that both methods yield close results in respect to the full game. With increasing cardinality of each player’s strategy space, the
MTPGD yields a relevant decrease of the run-time. Otherwise, the TSTPG
does not minimize the time complexity, but enables the use of more sophisticated algorithms for two-player games, resulting in a decreased runtime.
The cooperation within a team is enabled by introducing a hierarchical decomposition of the game. On a superordinate collaboration level, the pursuers choose their optimal behavioral strategy (e.g. pursuit and battue)
resulting in the case of a two-pursuer one-evader PEG in a three-player noncooperative dynamic game, which is solved in a subordinate level of the
overall game. This structure enables an intelligent behavior change for the pursuers based on game-theoretical solution methods. Depending on the
state of the game, which behavioral strategy yields the best results for the
pursuers within a predefined time horizon has to be evaluated. It is shown
that the pursuer’s outcome can be improved by using a superordinate cooperation. Moreover, conditions are presented under which a capture of a
faster evader by a group of two pursuers is possible in a visibility-based PEG
with imperfect information.
Since Unmanned Aerial Vehicles (UAVs) are increasingly a common platform
used in the aforementioned applications, this work focuses only on PEGs with
multi-rotor UAVs. Furthermore, the realization of the concepts in this thesis
are applied on a real hex rotor. The feasibility of the approach is proven in
experiments, while all implementations on the UAV are running in real-time.
This framework provides a solution concept for all types of dynamic games
with an 1-M or N-1 setup, that have a non-cooperative and cooperative
nature. At this stage a N-M dynamic game is not applicable. Nevertheless,
an approach to extend this framework to the N-M case is proposed in the
last chapter of this work
A Methodology to Enhance Quantitative Technology Evaluation Through Exploration of Employment Concepts in Engagement Analysis
The process of designing a new system has often been treated as a purely technological problem, where the infusion or synthesis of new technologies forms the basis of progress. However, recent trends in design and analysis methodologies have tried to shift away from the narrow scope of technology-centric approaches. One such trend is the increase in analysis scope from the level of an isolated system to that of multiple interacting systems. Analysis under this broader scope allows for the exploration of non-materiel solutions to existing or future problems. Solutions of this type can reduce the cost of closing capability gaps by mitigating the need to procure new systems to achieve desired levels of performance. In particular, innovations in the employment concepts can enhance existing, evolutionary, or revolutionary materiel solutions.
The task of experimenting with non-materiel solutions often falls to operators after the system has been designed and produced. This begs the question as to whether the chosen design adequately accounted for the possibility of innovative employment concepts which operators might discover. Attempts can be made to bring the empirical knowledge possessed by skilled operators upstream in the design process. However, care must be taken to ensure such attempts do not introduce unwanted bias, and there can be significant difficulty in translating human intuition into an appropriate modeling paradigm for analysis. Furthermore, the capacity for human operators to capitalize on the potential benefits of a given technology may be limited or otherwise infeasible in design space explorations where the number of alternatives becomes very large. This is especially relevant to revolutionary concepts to which prior knowledge may not be applicable. Each of these complicating factors is exacerbated by interactions between systems, where changes in the decision-making processes of individual entities can greatly influence outcomes. This necessitates exploration and analysis of employment concepts for all relevant entities, not only that or those to which the technology applies.
This research sought to address the issues of exploring employment concepts in the early phases of the system design process. A characterization of the problem identified several gaps in existing methodologies, particularly with respect to the representation, generation, and evaluation of alternative employment concepts. Relevant theories, including behavioral psychology, control theory, and game theory were identified to facilitate closure of these gaps. However, these theories also introduced technical challenges which had to be overcome. These challenges stemmed from systematic problems such as the curse of dimensionality, temporal credit assignment, and the complexities of entity interactions. A candidate approach was identified through thorough review of available literature: Multi-agent reinforcement learning. Experiments show the proposed approach can be used to generate highly effective models of behavior which could out-perform existing models on a representative problem. It was further shown that models produced by this new method can achieve consistently high levels of performance in competitive scenarios. Additional experimentation demonstrated how incorporation of design variables into the state space allowed models to learn policies which were effective across a continuous design space and outperformed their respective baselines. All of these results were obtained without reliance on prior knowledge, mitigating risks in and enhancing the capabilities of the analysis process. Lastly, the completed methodology was applied to the design of a fighter aircraft for one-on-one, gun-only air combat engagements to demonstrate its efficacy on and applicability to more complex problems.Ph.D
Coordinating Team Tactics for Swarm-vs.-Swarm Adversarial Games
While swarms of UAVs have received much attention in the last few years, adversarial swarms (i.e., competitive, swarm-vs.-swarm games) have been less well studied. In this dissertation, I investigate the factors influential in team-vs.-team UAV aerial combat scenarios, elucidating the impacts of force concentration and opponent spread in the engagement space. Specifically, this dissertation makes the following contributions:
(1) Tactical Analysis: Identifies conditions under which either explicitly-coordinating tactics or decentralized, greedy tactics are superior in engagements as small as 2-vs.-2 and as large as 10-vs.-10, and examines how these patterns change with the quality of the teams' weapons;
(2) Coordinating Tactics: Introduces and demonstrates a deep-reinforcement-learning framework that equips agents to learn to use their own and their teammates' situational context to decide which pre-scripted tactics to employ in what situations, and which teammates, if any, to coordinate with throughout the engagement; the efficacy of agents using the neural network trained within this framework outperform baseline tactics in engagements against teams of agents employing baseline tactics in N-vs.-N engagements for N as small as two and as large as 64; and
(3) Bio-Inspired Coordination: Discovers through Monte-Carlo agent-based simulations the importance of prioritizing the team's force concentration against the most threatening opponent agents, but also of preserving some resources by deploying a smaller defense force and defending against lower-penalty threats in addition to high-priority threats to maximize the remaining fuel within the defending team's fuel reservoir.Ph.D
Collision Free Navigation of a Multi-Robot Team for Intruder Interception
In this report, we propose a decentralised motion control algorithm for the
mobile robots to intercept an intruder entering (k-intercepting) or escaping
(e-intercepting) a protected region. In continuation, we propose a
decentralized navigation strategy (dynamic-intercepting) for a multi-robot team
known as predators to intercept the intruders or in the other words, preys,
from escaping a siege ring which is created by the predators. A necessary and
sufficient condition for the existence of a solution of this problem is
obtained. Furthermore, we propose an intelligent game-based decision-making
algorithm (IGD) for a fleet of mobile robots to maximize the probability of
detection in a bounded region. We prove that the proposed decentralised
cooperative and non-cooperative game-based decision-making algorithm enables
each robot to make the best decision to choose the shortest path with minimum
local information. Then we propose a leader-follower based collision-free
navigation control method for a fleet of mobile robots to traverse an unknown
cluttered environment where is occupied by multiple obstacles to trap a target.
We prove that each individual team member is able to traverse safely in the
region, which is cluttered by many obstacles with any shapes to trap the target
while using the sensors in some indefinite switching points and not
continuously, which leads to saving energy consumption and increasing the
battery life of the robots consequently. And finally, we propose a novel
navigation strategy for a unicycle mobile robot in a cluttered area with moving
obstacles based on virtual field force algorithm. The mathematical proof of the
navigation laws and the computer simulations are provided to confirm the
validity, robustness, and reliability of the proposed methods