Cooperative Pursuit with Multi-Pursuer and One Faster Free-moving Evader

This paper addresses a multi-pursuer single-evader pursuit-evasion game where the free-moving evader moves faster than the pursuers. Most of the existing works impose constraints on the faster evader such as limited moving area and moving direction. When the faster evader is allowed to move freely without any constraint, the main issues are how to form an encirclement to trap the evader into the capture domain, how to balance between forming an encirclement and approaching the faster evader, and what conditions make the capture possible. In this paper, a distributed pursuit algorithm is proposed to enable pursuers to form an encirclement and approach the faster evader. An algorithm that balances between forming an encirclement and approaching the faster evader is proposed. Moreover, sufficient capture conditions are derived based on the initial spatial distribution and the speed ratios of the pursuers and the evader. Simulation and experimental results on ground robots validate the effectiveness and practicability of the proposed method

Multi-Agent Pursuit of a Faster Evader with Application to Unmanned Aerial Vehicles

Robotic applications like search and rescue missions, surveillance, police missions, patrolling, and warfare can all be modeled as a Pursuit-Evasion Game (PEG). Most of these tasks are multi-agent problems, often including a cooperation between team members and a conflict between adversarial teams. In order to realize such a situation with robots, two major problems have to be solved. Initially, a decomposition of the PEG has to be performed for getting results in reasonable time. Present embedded computers lack the computational power enabling them to process the highly complex solution algorithm of the non-decomposed game fast enough. Secondly, a framework has to be defined, enabling the computation of optimal actions for both the pursuers and the evaders when a cooperation within the teams is possible. It is intended to develop strategies, that allow the team of pursuers to capture a faster evader in a visibility-based PEG setup due to cooperation. For tackling the first problem a game structure is sought, aiming to considerably reduce the time complexity of the solution process. The first step is the decomposition of the action space, and the second is the change of the game structure itself. The latter is reached by defining a two-pursuer one-evader PEG with three different game structures, which are the Non-Decomposed Game, the Multiple Two-Player Game Decomposition (MTPGD) game, and the Team-Subsumption Two-Player Game (TSTPG). Several simulation results demonstrate, that both methods yield close results in respect to the full game. With increasing cardinality of each player’s strategy space, the MTPGD yields a relevant decrease of the run-time. Otherwise, the TSTPG does not minimize the time complexity, but enables the use of more sophisticated algorithms for two-player games, resulting in a decreased runtime. The cooperation within a team is enabled by introducing a hierarchical decomposition of the game. On a superordinate collaboration level, the pursuers choose their optimal behavioral strategy (e.g. pursuit and battue) resulting in the case of a two-pursuer one-evader PEG in a three-player noncooperative dynamic game, which is solved in a subordinate level of the overall game. This structure enables an intelligent behavior change for the pursuers based on game-theoretical solution methods. Depending on the state of the game, which behavioral strategy yields the best results for the pursuers within a predefined time horizon has to be evaluated. It is shown that the pursuer’s outcome can be improved by using a superordinate cooperation. Moreover, conditions are presented under which a capture of a faster evader by a group of two pursuers is possible in a visibility-based PEG with imperfect information. Since Unmanned Aerial Vehicles (UAVs) are increasingly a common platform used in the aforementioned applications, this work focuses only on PEGs with multi-rotor UAVs. Furthermore, the realization of the concepts in this thesis are applied on a real hex rotor. The feasibility of the approach is proven in experiments, while all implementations on the UAV are running in real-time. This framework provides a solution concept for all types of dynamic games with an 1-M or N-1 setup, that have a non-cooperative and cooperative nature. At this stage a N-M dynamic game is not applicable. Nevertheless, an approach to extend this framework to the N-M case is proposed in the last chapter of this work

Games of Pursuit-Evasion with Multiple Agents and Subject to Uncertainties

Over the past decade, there have been constant efforts to induct unmanned aerial vehicles (UAVs) into military engagements, disaster management, weather monitoring, and package delivery, among various other applications. With UAVs starting to come out of controlled environments into real-world scenarios, uncertainties that can be either exogenous or endogenous play an important role in the planning and decision-making aspects of deploying UAVs. At the same time, while the demand for UAVs is steadily increasing, major governments are working on their regulations. There is an urgency to design surveillance and security systems that can efficiently regulate the traffic and usage of these UAVs, especially in secured airspaces. With this motivation, the thesis primarily focuses on airspace security, providing solutions for safe planning under uncertainties while addressing aspects concerning target acquisition and collision avoidance. In this thesis, we first present our work on solutions developed for airspace security that employ multiple agents to capture multiple targets in an efficient manner. Since multi-pursuer multi-evader problems are known to be intractable, heuristics based on the geometry of the game are employed to obtain task-allocation algorithms that are computationally efficient. This is achieved by first analyzing pursuit-evasion problems involving two pursuers and one evader. Using the insights obtained from this analysis, a dynamic allocation algorithm for the pursuers, which is independent of the evader's strategy, is proposed. The algorithm is further extended to solve multi-pursuer multi-evader problems for any number of pursuers and evaders, assuming both sets of agents to be heterogeneous in terms of speed capabilities. Next, we consider stochastic disturbances, analyzing pursuit-evasion problems under stochastic flow fields using forward reachability analysis, and covariance steering. The problem of steering a Gaussian in adversarial scenarios is first analyzed under the framework of general constrained games. The resulting covariance steering problem is solved numerically using iterative techniques. The proposed approach is applied to the missile endgame guidance problem. Subsequently, using the theory of covariance steering, an approach to solve pursuit-evasion problems under external stochastic flow fields is discussed. Assuming a linear feedback control strategy, a chance-constrained covariance game is constructed around the nominal solution of the players. The proposed approach is tested on realistic linear and nonlinear flow fields. Numerical simulations suggest that the pursuer can effectively steer the game towards capture. Finally, the uncertainties are assumed to be parametric in nature. To this end, we first formalize optimal control under parametric uncertainties while introducing sensitivity functions and costates based techniques to address robustness under parametric variations. Utilizing the sensitivity functions, we address the problem of safe path planning in environments containing dynamic obstacles with an uncertain motion model. The sensitivity function based-approach is then extended to address game-theoretic formulations that resemble a "fog of war" situation.Ph.D

Game Theoretic Strategies for Spacecraft Rendezvous and Motion Synchronization

Uno dei possibili sviluppi della guida e del controllo relativo nello spazio è quella di estendere gli algoritmi per operazioni di rendezvous e di docking autonome a più veicoli spaziali che collaborano tra di loro. Il problema del rendezvous tra due veicoli spaziali viene risolto utilizzando la teoria dei giochi differenziali lineari quadratici. La dinamica del gioco viene descritta in un sistema di riferimento cartesiano non inerziale. Per estendere l'utilizzo della teoria dei giochi differenziali lineari quadratici alle equazioni non lineari di moto relativo è stata utilizzata le tecnica di parametrizzazione in funzione dello stato o linearizzazione estesa. Nelle simulazioni è stato valutato il confronto tra le prestazioni e le traiettorie ottenute con l'equilibrio di Pareto e quello di Nash quando entrambi i veicoli spaziali agiscono sotto spinta continua. Una strategia sequenziale è stata utilizzata per estendere il gioco differenziali a più di due giocatori per avere la sincronizzazione del moto relativo durante operazioni di assemblaggio nello spazio. One of the main challenges for autonomous spacecraft relative guidance and control is extending the algorithms for autonomous rendezvous and docking (AR&D) operations to multiple collaborative spacecraft. In this thesis, the autonomous rendezvous problem, between two active spacecraft, is formulated as a two player nonzero-sum differential game. The local-vertical local-horizontal (LVLH) rotating reference frame is used to describe the dynamic of the game. The State-Dependent Riccati equation (SDRE) method is applied to extend the Linear Quadratic differential game theory to obtain a feedback control law for nonlinear equation of relative motion. In the simulations both the spacecraft use continuous thrust engines. A comparison among Pareto and Nash equilibrium has been performed. A multiplayer sequential game strategy is used to extend the control law to many spacecraft for relative motion synchronization in an on-orbit self assembly strategy

Game Theoretic Strategies for Spacecraft Rendezvous and Motion Synchronization

The rendezvous problem between two active spacecraft is formulated as a two player nonzero-sum differential game. The local-vertical local-horizontal (LVLH) rotating reference frame is used to describe the dynamics of the game. Linear quadratic cooperative and noncooperative differential games are applied to obtain a feedback control law. A comparison between Pareto and Nash equilibria was then performed. The state-dependent Riccati equation (SDRE) method is applied to extend the Linear Quadratic differential game theory to obtain a feedback controller in the case of nonlinear relative motion dynamics. Finally, a multiplayer sequential game strategy is synthesized to extend the control law to the relative motion synchronization of multiple vehicles

A Methodology to Enhance Quantitative Technology Evaluation Through Exploration of Employment Concepts in Engagement Analysis

The process of designing a new system has often been treated as a purely technological problem, where the infusion or synthesis of new technologies forms the basis of progress. However, recent trends in design and analysis methodologies have tried to shift away from the narrow scope of technology-centric approaches. One such trend is the increase in analysis scope from the level of an isolated system to that of multiple interacting systems. Analysis under this broader scope allows for the exploration of non-materiel solutions to existing or future problems. Solutions of this type can reduce the cost of closing capability gaps by mitigating the need to procure new systems to achieve desired levels of performance. In particular, innovations in the employment concepts can enhance existing, evolutionary, or revolutionary materiel solutions. The task of experimenting with non-materiel solutions often falls to operators after the system has been designed and produced. This begs the question as to whether the chosen design adequately accounted for the possibility of innovative employment concepts which operators might discover. Attempts can be made to bring the empirical knowledge possessed by skilled operators upstream in the design process. However, care must be taken to ensure such attempts do not introduce unwanted bias, and there can be significant difficulty in translating human intuition into an appropriate modeling paradigm for analysis. Furthermore, the capacity for human operators to capitalize on the potential benefits of a given technology may be limited or otherwise infeasible in design space explorations where the number of alternatives becomes very large. This is especially relevant to revolutionary concepts to which prior knowledge may not be applicable. Each of these complicating factors is exacerbated by interactions between systems, where changes in the decision-making processes of individual entities can greatly influence outcomes. This necessitates exploration and analysis of employment concepts for all relevant entities, not only that or those to which the technology applies. This research sought to address the issues of exploring employment concepts in the early phases of the system design process. A characterization of the problem identified several gaps in existing methodologies, particularly with respect to the representation, generation, and evaluation of alternative employment concepts. Relevant theories, including behavioral psychology, control theory, and game theory were identified to facilitate closure of these gaps. However, these theories also introduced technical challenges which had to be overcome. These challenges stemmed from systematic problems such as the curse of dimensionality, temporal credit assignment, and the complexities of entity interactions. A candidate approach was identified through thorough review of available literature: Multi-agent reinforcement learning. Experiments show the proposed approach can be used to generate highly effective models of behavior which could out-perform existing models on a representative problem. It was further shown that models produced by this new method can achieve consistently high levels of performance in competitive scenarios. Additional experimentation demonstrated how incorporation of design variables into the state space allowed models to learn policies which were effective across a continuous design space and outperformed their respective baselines. All of these results were obtained without reliance on prior knowledge, mitigating risks in and enhancing the capabilities of the analysis process. Lastly, the completed methodology was applied to the design of a fighter aircraft for one-on-one, gun-only air combat engagements to demonstrate its efficacy on and applicability to more complex problems.Ph.D

Coordinated Defense Allocation in Reach-Avoid Scenarios with Efficient Online Optimization

In this paper, we present a dual-layer online optimization strategy for defender robots operating in multiplayer reach-avoid games within general convex environments. Our goal is to intercept as many attacker robots as possible without prior knowledge of their strategies. To balance optimality and efficiency, our approach alternates between coordinating defender coalitions against individual attackers and allocating coalitions to attackers based on predicted single-attack coordination outcomes. We develop an online convex programming technique for single-attack defense coordination, which not only allows adaptability to joint states but also identifies the maximal region of initial joint states that guarantees successful attack interception. Our defense allocation algorithm utilizes a hierarchical iterative method to approximate integer linear programs with a monotonicity constraint, reducing computational burden while ensuring enhanced defense performance over time. Extensive simulations conducted in 2D and 3D environments validate the efficacy of our approach in comparison to state-of-the-art approaches, and show its applicability in wheeled mobile robots and quadcopters

Emergent Behavior Development and Control in Multi-Agent Systems

Emergence in natural systems is the development of complex behaviors that result from the aggregation of simple agent-to-agent and agent-to-environment interactions. Emergence research intersects with many disciplines such as physics, biology, and ecology and provides a theoretical framework for investigating how order appears to spontaneously arise in complex adaptive systems. In biological systems, emergent behaviors allow simple agents to collectively accomplish multiple tasks in highly dynamic environments; ensuring system survival. These systems all display similar properties: self-organized hierarchies, robustness, adaptability, and decentralized task execution. However, current algorithmic approaches merely present theoretical models without showing how these models actually create hierarchical, emergent systems. To fill this research gap, this dissertation presents an algorithm based on entropy and speciation - defined as morphological or physiological differences in a population - that results in hierarchical emergent phenomena in multi-agent systems. Results show that speciation creates system hierarchies composed of goal-aligned entities, i.e. niches. As niche actions aggregate into more complex behaviors, more levels emerge within the system hierarchy, eventually resulting in a system that can meet multiple tasks and is robust to environmental changes. Speciation provides a powerful tool for creating goal-aligned, decentralized systems that are inherently robust and adaptable, meeting the scalability demands of current, multi-agent system design. Results in base defense, k-n assignment, division of labor and resource competition experiments, show that speciated populations create hierarchical self-organized systems, meet multiple tasks and are more robust to environmental change than non-speciated populations

Optimal steering for kinematic vehicles with applications to spatially distributed agents

The recent technological advances in the field of autonomous vehicles have resulted in a growing impetus for researchers to improve the current framework of mission planning and execution within both the military and civilian contexts. Many recent efforts towards this direction emphasize the importance of replacing the so-called monolithic paradigm, where a mission is planned, monitored, and controlled by a unique global decision maker, with a network centric paradigm, where the same mission related tasks are performed by networks of interacting decision makers (autonomous vehicles). The interest in applications involving teams of autonomous vehicles is expected to significantly grow in the near future as new paradigms for their use are constantly being proposed for a diverse spectrum of real world applications. One promising approach to extend available techniques for addressing problems involving a single autonomous vehicle to those involving teams of autonomous vehicles is to use the concept of Voronoi diagram as a means for reducing the complexity of the multi-vehicle problem. In particular, the Voronoi diagram provides a spatial partition of the environment the team of vehicles operate in, where each element of this partition is associated with a unique vehicle from the team. The partition induces, in turn, a graph abstraction of the operating space that is in a one-to-one correspondence with the network abstraction of the team of autonomous vehicles; a fact that can provide both conceptual and analytical advantages during mission planning and execution. In this dissertation, we propose the use of a new class of Voronoi-like partitioning schemes with respect to state-dependent proximity (pseudo-) metrics rather than the Euclidean distance or other generalized distance functions, which are typically used in the literature. An important nuance here is that, in contrast to the Euclidean distance, state-dependent metrics can succinctly capture system theoretic features of each vehicle from the team (e.g., vehicle kinematics), as well as the environment-vehicle interactions, which are induced, for example, by local winds/currents. We subsequently illustrate how the proposed concept of state-dependent Voronoi-like partition can induce local control schemes for problems involving networks of spatially distributed autonomous vehicles by examining different application scenarios.PhDCommittee Chair: Tsiotras Panagiotis; Committee Member: Egerstedt Magnus; Committee Member: Feron Eric; Committee Member: Haddad Wassim; Committee Member: Shamma Jef