Search and Pursuit-Evasion in Mobile Robotics, A survey

This paper surveys recent results in pursuitevasion and autonomous search relevant to applications in mobile robotics. We provide a taxonomy of search problems that highlights the differences resulting from varying assumptions on the searchers, targets, and the environment. We then list a number of fundamental results in the areas of pursuit-evasion and probabilistic search, and we discuss field implementations on mobile robotic systems. In addition, we highlight current open problems in the area and explore avenues for future work

Multi Robot Intruder Search

The aim of this work is the development and analysis of methods and algorithms to allow a multi robot system to cooperatively search a closed, 2-dimensional environment for a human intruder. The underlying problem corresponds to the game-theoretic concept of a classical pursuit evasion game, whereas the focus is set to the generation of plans for the group of pursuers. While the main aspect of of this work lies in the field of probabilistic robotics, concepts and ideas are incorporated from differential game theory, algorithmic geometry and graph theory. The probabilistic basis allows the integration of sensor error as well as nondeterministic robot motion. The main contributions of this work can be divided into three major parts: The first part deals with the development and implementation of probabilistic human models. Depending on the specific behavior of an intruder, ranging from uncooperative to unaware, different classes of intruders are identified. Models are proposed for two of these classes. For the case of a clever and uncooperative intruder who actively evades detection, we propose a model based on the concept of contamination. The second class corresponds to a person who is unaware of the pursuit. We show that simple Markov models, which are often proposed in literature, are not suited for modeling realistic human motion and develop advanced Markov models, which conform to random waypoint motion models. The second part, which is also the most extensive part of this work, deals with the problem of finding an uncooperative and clever intruder. A solution is presented, which projects the problem on a graph structure, which is then searched by a highly optimized A* planner. The solution for the corresponding graph problem is afterwards projected back to the original search space and can be executed by the robotic pursuers. By means of the models proposed in the first part, the performance and correctness of the method is shown. We present experiments in simulation as on real robots to show the practicability and efficiency of the method. The third part deals with the problem of finding an intruder who is unaware of the search. Based on the advanced Markov model previously discussed, a greedy algorithm is proposed, which aims at maximizing the probability to find the intruder in the near future. Experimental results for this method are shown and comparisons to simpler methods are given.Mehrroboter-Eindringlings-Suche Ziel dieser Arbeit ist die Entwicklung und Analyse von Methoden und Algorithmen, die einem kooperativen Mehrrobotersystem erlauben nach einem Eindringling in einer zweidimensionalen, geschlossenen Umgebung zu suchen. Das dem zugrunde liegende Problem entspricht dem spieltheoretischen Konzept eines Suche und Ausweichen Spieles (pursuit evasion game), wobei der Fokus auf der Generierung von Plänen für die Verfolger liegt. Der Hauptaspekt dieser Arbeit liegt dabei im Feld der probabilistischen Robotik, wobei Konzepte und Ideen aus dem Gebiet der differentiellen Spieltheorie, der algorithmischen Geometrie und der Graph Theorie verwendet werden. Die probabilistische Modellierung erlaubt dabei die Integration von Sensorfehlern wie auch nichtdeterministische Roboter-Bewegung. Die Arbeit gliedert sich in drei Hauptteile: Der erste Teil beschäftigt sich mit dem Entwurf und der Implementierung von probabilistischen Modellen für menschliche Bewegung. Abhängig vom angenommenen Verhalten eines Eindringlings, von aktiv ausweichend bis zu ignorant, werden verschiedene Klassen von menschlichem Verhalten unterschieden. Für zwei dieser Klassen stellen wir Modelle auf: Für den Fall einer intelligenten und aktiv ausweichenden Person, generieren wir ein Modell basierend auf dem Konzept von Kontamination. Das zweite Modell entspricht einem Eindringling, der sich der Suche nach ihm nicht bewusst ist. Wir zeigen, dass einfache Markov-Modelle, wie sie in der Vergangenheit oft vorgeschlagen worden sind, ungeeignet sind, um realistische Bewegung zu abzubilden und entwickeln entsprechend erweiterte Markov-Modelle für eine realistischere Modellierung. Der zweite Teil der Arbeit beschäftigt sich mit der Frage, wie man einen intelligente und aktiv ausweichenden Eindringling aufspüren kann. Die vorgestellte Lösung basiert auf der Projektion des Problems auf einen Graphen, der anschließend von einem hoch optimierten A*-Planungsalgorithmus durchsucht werden kann. Diese Lösung kann anschließend auf den ursprünglichen Raum rückprojeziert werden und kann als direkter Plan von den verfolgenden Robotern ausgeführt werden. Mittels der Modelle aus dem ersten Teil wird die Korrektheit und Effizienz der Lösung gezeigt. Der letzte Teil befasst sich mit der Frage, wie ein Eindringling gefunden werden kann, der sich neutral zur Suche verhält. Basierend auf den erweiterten Markov-Modellen aus dem ersten Teil, wird eine Lösung durch gierige Suche präsentiert, die die Wahrscheinlichkeit eine Person im nächsten Zeitschritt aufzuspüren, maximiert. Wie im zweiten Teil werden Experimente diskutiert und diese mit der Proformanz simplerer Methoden verglichen

On the role and opportunities in teamwork design for advanced multi-robot search systems

Intelligent robotic systems are becoming ever more present in our lives across a multitude of domains such as industry, transportation, agriculture, security, healthcare and even education. Such systems enable humans to focus on the interesting and sophisticated tasks while robots accomplish tasks that are either too tedious, routine or potentially dangerous for humans to do. Recent advances in perception technologies and accompanying hardware, mainly attributed to rapid advancements in the deep-learning ecosystem, enable the deployment of robotic systems equipped with onboard sensors as well as the computational power to perform autonomous reasoning and decision making online. While there has been significant progress in expanding the capabilities of single and multi-robot systems during the last decades across a multitude of domains and applications, there are still many promising areas for research that can advance the state of cooperative searching systems that employ multiple robots. In this article, several prospective avenues of research in teamwork cooperation with considerable potential for advancement of multi-robot search systems will be visited and discussed. In previous works we have shown that multi-agent search tasks can greatly benefit from intelligent cooperation between team members and can achieve performance close to the theoretical optimum. The techniques applied can be used in a variety of domains including planning against adversarial opponents, control of forest fires and coordinating search-and-rescue missions. The state-of-the-art on methods of multi-robot search across several selected domains of application is explained, highlighting the pros and cons of each method, providing an up-to-date view on the current state of the domains and their future challenges

Multiagent Cooperative Learning Strategies for Pursuit-Evasion Games

This study examines the pursuit-evasion problem for coordinating multiple robotic pursuers to locate and track a nonadversarial mobile evader in a dynamic environment. Two kinds of pursuit strategies are proposed, one for agents that cooperate with each other and the other for agents that operate independently. This work further employs the probabilistic theory to analyze the uncertain state information about the pursuers and the evaders and uses case-based reasoning to equip agents with memories and learning abilities. According to the concepts of assimilation and accommodation, both positive-angle and bevel-angle strategies are developed to assist agents in adapting to their environment effectively. The case study analysis uses the Recursive Porous Agent Simulation Toolkit (REPAST) to implement a multiagent system and demonstrates superior performance of the proposed approaches to the pursuit-evasion game

Reachable sets analysis in the cooperative control of pursuer vehicles

This thesis is concerned with the Pursuit-and-Evasion (PE) problem where the pursuer aims to minimize the time to capture the evader while the evader tries to prevent capture. In the problem, the evader has two advantages: a higher manoeuvrability and that the pursuer is uncertain about the evader's state. Cooperation among multiple pursuer vehicles can thus be used to overcome the evader’s advantages. The focus here is on the formulation and development of frameworks and algorithms for cooperation amongst pursuers, aiming at feasible implementation on real and autonomous vehicles. The thesis is split into Parts I and II. Part I considers the problem of capturing an evader of higher manoeuvrability in a deterministic PE game. The approach is the employment of Forward Reachable Set (FRS) analysis in the pursuers’ control. The analysis considers the coverage of the evader’s FRS, which is the set of reachable states at a future time, with the pursuer’s FRS and assumes that the chance of capturing the evader is dependent on the degree of the coverage. Using the union of multiple pursuers’ FRSs intuitively leads to more evader FRS coverage and this forms the mechanism of cooperation. A framework for cooperative control based on the FRS coverage, or FRS-based control, is proposed. Two control algorithms were developed within this framework. Part II additionally introduces the problem of evader state uncertainty due to noise and limited field-of-view of the pursuers’ sensors. A search-and-capture (SAC) problem is the result and a hybrid architecture, which includes multi-sensor estimation using the Particle Filter as well as FRS-based control, is proposed to accomplish the SAC task. The two control algorithms in Part I were tested in simulations against an optimal guidance algorithm. The results show that both algorithms yield a better performance in terms of time and miss distance. The results in Part II demonstrate the effectiveness of the hybrid architecture for the SAC task. The proposed frameworks and algorithms provide insights for the development of effective and more efficient control of pursuer vehicles and can be useful in the practical applications such as defence systems and civil law enforcement

Mobile robotic network deployment for intruder detection and tracking

This thesis investigates the problem of intruder detection and tracking using mobile robotic networks. In the first part of the thesis, we consider the problem of seeking an electromagnetic source using a team of robots that measure the local intensity of the emitted signal. We propose a planner for a team of robots based on Particle Swarm Optimization (PSO) which is a population based stochastic optimization technique. An equivalence is established between particles generated in the traditional PSO technique, and the mobile agents in the swarm. Since the positions of the robots are updated using the PSO algorithm, modifications are required to implement the PSO algorithm on real robots to incorporate collision avoidance strategies. The modifications necessary to implement PSO on mobile robots, and strategies to adapt to real environments are presented in this thesis. Our results are also validated on an experimental testbed. In the second part, we present a game theoretic framework for visibility-based target tracking in multi-robot teams. A team of observers (pursuers) and a team of targets (evaders) are present in an environment with obstacles. The objective of the team of observers is to track the team of targets for the maximum possible time. While the objective of the team of targets is to escape (break line-of-sight) in the minimum time. We decompose the problem into two layers. At the upper level, each pursuer is allocated to an evader through a minimum cost allocation strategy based on the risk of each evader, thereby, decomposing the agents into multiple single pursuer-single evader pairs. Two decentralized allocation strategies are proposed and implemented in this thesis. At the lower level, each pursuer computes its strategy based on the results of the single pursuer-single evader target-tracking problem. We initially address this problem in an environment containing a semi-infinite obstacle with one corner. The pursuer\u27s optimal tracking strategy is obtained regardless of the evader\u27s strategy using techniques from optimal control theory and differential games. Next, we extend the result to an environment containing multiple polygonal obstacles. We construct a pursuit field to provide a guiding vector for the pursuer which is a weighted sum of several component vectors. The performance of different combinations of component vectors is investigated. Finally, we extend our work to address the case when the obstacles are not polygonal, and the observers have constraints in motion

Toward multi-target self-organizing pursuit in a partially observable Markov game

The multiple-target self-organizing pursuit (SOP) problem has wide applications and has been considered a challenging self-organization game for distributed systems, in which intelligent agents cooperatively pursue multiple dynamic targets with partial observations. This work proposes a framework for decentralized multi-agent systems to improve intelligent agents' search and pursuit capabilities. We model a self-organizing system as a partially observable Markov game (POMG) with the features of decentralization, partial observation, and noncommunication. The proposed distributed algorithm: fuzzy self-organizing cooperative coevolution (FSC2) is then leveraged to resolve the three challenges in multi-target SOP: distributed self-organizing search (SOS), distributed task allocation, and distributed single-target pursuit. FSC2 includes a coordinated multi-agent deep reinforcement learning method that enables homogeneous agents to learn natural SOS patterns. Additionally, we propose a fuzzy-based distributed task allocation method, which locally decomposes multi-target SOP into several single-target pursuit problems. The cooperative coevolution principle is employed to coordinate distributed pursuers for each single-target pursuit problem. Therefore, the uncertainties of inherent partial observation and distributed decision-making in the POMG can be alleviated. The experimental results demonstrate that distributed noncommunicating multi-agent coordination with partial observations in all three subtasks are effective, and 2048 FSC2 agents can perform efficient multi-target SOP with almost 100% capture rates