Reachable sets analysis in the cooperative control of pursuer vehicles

This thesis is concerned with the Pursuit-and-Evasion (PE) problem where the pursuer aims to minimize the time to capture the evader while the evader tries to prevent capture. In the problem, the evader has two advantages: a higher manoeuvrability and that the pursuer is uncertain about the evader's state. Cooperation among multiple pursuer vehicles can thus be used to overcome the evader’s advantages. The focus here is on the formulation and development of frameworks and algorithms for cooperation amongst pursuers, aiming at feasible implementation on real and autonomous vehicles. The thesis is split into Parts I and II. Part I considers the problem of capturing an evader of higher manoeuvrability in a deterministic PE game. The approach is the employment of Forward Reachable Set (FRS) analysis in the pursuers’ control. The analysis considers the coverage of the evader’s FRS, which is the set of reachable states at a future time, with the pursuer’s FRS and assumes that the chance of capturing the evader is dependent on the degree of the coverage. Using the union of multiple pursuers’ FRSs intuitively leads to more evader FRS coverage and this forms the mechanism of cooperation. A framework for cooperative control based on the FRS coverage, or FRS-based control, is proposed. Two control algorithms were developed within this framework. Part II additionally introduces the problem of evader state uncertainty due to noise and limited field-of-view of the pursuers’ sensors. A search-and-capture (SAC) problem is the result and a hybrid architecture, which includes multi-sensor estimation using the Particle Filter as well as FRS-based control, is proposed to accomplish the SAC task. The two control algorithms in Part I were tested in simulations against an optimal guidance algorithm. The results show that both algorithms yield a better performance in terms of time and miss distance. The results in Part II demonstrate the effectiveness of the hybrid architecture for the SAC task. The proposed frameworks and algorithms provide insights for the development of effective and more efficient control of pursuer vehicles and can be useful in the practical applications such as defence systems and civil law enforcement

Multi Robot Intruder Search

The aim of this work is the development and analysis of methods and algorithms to allow a multi robot system to cooperatively search a closed, 2-dimensional environment for a human intruder. The underlying problem corresponds to the game-theoretic concept of a classical pursuit evasion game, whereas the focus is set to the generation of plans for the group of pursuers. While the main aspect of of this work lies in the field of probabilistic robotics, concepts and ideas are incorporated from differential game theory, algorithmic geometry and graph theory. The probabilistic basis allows the integration of sensor error as well as nondeterministic robot motion. The main contributions of this work can be divided into three major parts: The first part deals with the development and implementation of probabilistic human models. Depending on the specific behavior of an intruder, ranging from uncooperative to unaware, different classes of intruders are identified. Models are proposed for two of these classes. For the case of a clever and uncooperative intruder who actively evades detection, we propose a model based on the concept of contamination. The second class corresponds to a person who is unaware of the pursuit. We show that simple Markov models, which are often proposed in literature, are not suited for modeling realistic human motion and develop advanced Markov models, which conform to random waypoint motion models. The second part, which is also the most extensive part of this work, deals with the problem of finding an uncooperative and clever intruder. A solution is presented, which projects the problem on a graph structure, which is then searched by a highly optimized A* planner. The solution for the corresponding graph problem is afterwards projected back to the original search space and can be executed by the robotic pursuers. By means of the models proposed in the first part, the performance and correctness of the method is shown. We present experiments in simulation as on real robots to show the practicability and efficiency of the method. The third part deals with the problem of finding an intruder who is unaware of the search. Based on the advanced Markov model previously discussed, a greedy algorithm is proposed, which aims at maximizing the probability to find the intruder in the near future. Experimental results for this method are shown and comparisons to simpler methods are given.Mehrroboter-Eindringlings-Suche Ziel dieser Arbeit ist die Entwicklung und Analyse von Methoden und Algorithmen, die einem kooperativen Mehrrobotersystem erlauben nach einem Eindringling in einer zweidimensionalen, geschlossenen Umgebung zu suchen. Das dem zugrunde liegende Problem entspricht dem spieltheoretischen Konzept eines Suche und Ausweichen Spieles (pursuit evasion game), wobei der Fokus auf der Generierung von Plänen für die Verfolger liegt. Der Hauptaspekt dieser Arbeit liegt dabei im Feld der probabilistischen Robotik, wobei Konzepte und Ideen aus dem Gebiet der differentiellen Spieltheorie, der algorithmischen Geometrie und der Graph Theorie verwendet werden. Die probabilistische Modellierung erlaubt dabei die Integration von Sensorfehlern wie auch nichtdeterministische Roboter-Bewegung. Die Arbeit gliedert sich in drei Hauptteile: Der erste Teil beschäftigt sich mit dem Entwurf und der Implementierung von probabilistischen Modellen für menschliche Bewegung. Abhängig vom angenommenen Verhalten eines Eindringlings, von aktiv ausweichend bis zu ignorant, werden verschiedene Klassen von menschlichem Verhalten unterschieden. Für zwei dieser Klassen stellen wir Modelle auf: Für den Fall einer intelligenten und aktiv ausweichenden Person, generieren wir ein Modell basierend auf dem Konzept von Kontamination. Das zweite Modell entspricht einem Eindringling, der sich der Suche nach ihm nicht bewusst ist. Wir zeigen, dass einfache Markov-Modelle, wie sie in der Vergangenheit oft vorgeschlagen worden sind, ungeeignet sind, um realistische Bewegung zu abzubilden und entwickeln entsprechend erweiterte Markov-Modelle für eine realistischere Modellierung. Der zweite Teil der Arbeit beschäftigt sich mit der Frage, wie man einen intelligente und aktiv ausweichenden Eindringling aufspüren kann. Die vorgestellte Lösung basiert auf der Projektion des Problems auf einen Graphen, der anschließend von einem hoch optimierten A*-Planungsalgorithmus durchsucht werden kann. Diese Lösung kann anschließend auf den ursprünglichen Raum rückprojeziert werden und kann als direkter Plan von den verfolgenden Robotern ausgeführt werden. Mittels der Modelle aus dem ersten Teil wird die Korrektheit und Effizienz der Lösung gezeigt. Der letzte Teil befasst sich mit der Frage, wie ein Eindringling gefunden werden kann, der sich neutral zur Suche verhält. Basierend auf den erweiterten Markov-Modellen aus dem ersten Teil, wird eine Lösung durch gierige Suche präsentiert, die die Wahrscheinlichkeit eine Person im nächsten Zeitschritt aufzuspüren, maximiert. Wie im zweiten Teil werden Experimente diskutiert und diese mit der Proformanz simplerer Methoden verglichen

Estimation and stability of nonlinear control systems under intermittent information with applications to multi-agent robotics

This dissertation investigates the role of intermittent information in estimation and control problems and applies the obtained results to multi-agent tasks in robotics. First, we develop a stochastic hybrid model of mobile networks able to capture a large variety of heterogeneous multi-agent problems and phenomena. This model is applied to a case study where a heterogeneous mobile sensor network cooperatively detects and tracks mobile targets based on intermittent observations. When these observations form a satisfactory target trajectory, a mobile sensor is switched to the pursuit mode and deployed to capture the target. The cost of operating the sensors is determined from the geometric properties of the network, environment and probability of target detection. The above case study is motivated by the Marco Polo game played by children in swimming pools. Second, we develop adaptive sampling of targets positions in order to minimize energy consumption, while satisfying performance guarantees such as increased probability of detection over time, and no-escape conditions. A parsimonious predictor-corrector tracking filter, that uses geometrical properties of targets\u27 tracks to estimate their positions using imperfect and intermittent measurements, is presented. It is shown that this filter requires substantially less information and processing power than the Unscented Kalman Filter and Sampling Importance Resampling Particle Filter, while providing comparable estimation performance in the presence of intermittent information. Third, we investigate stability of nonlinear control systems under intermittent information. We replace the traditional periodic paradigm, where the up-to-date information is transmitted and control laws are executed in a periodic fashion, with the event-triggered paradigm. Building on the small gain theorem, we develop input-output triggered control algorithms yielding stable closed-loop systems. In other words, based on the currently available (but outdated) measurements of the outputs and external inputs of a plant, a mechanism triggering when to obtain new measurements and update the control inputs is provided. Depending on the noise environment, the developed algorithm yields stable, asymptotically stable, and Lp-stable (with bias) closed-loop systems. Control loops are modeled as interconnections of hybrid systems for which novel results on Lp-stability are presented. Prediction of a triggering event is achieved by employing Lp-gains over a finite horizon in the small gain theorem. By resorting to convex programming, a method to compute Lp-gains over a finite horizon is devised. Next, we investigate optimal intermittent feedback for nonlinear control systems. Using the currently available measurements from a plant, we develop a methodology that outputs when to update the control law with new measurements such that a given cost function is minimized. Our cost function captures trade-offs between the performance and energy consumption of the control system. The optimization problem is formulated as a Dynamic Programming problem, and Approximate Dynamic Programming is employed to solve it. Instead of advocating a particular approximation architecture for Approximate Dynamic Programming, we formulate properties that successful approximation architectures satisfy. In addition, we consider problems with partially observable states, and propose Particle Filtering to deal with partially observable states and intermittent feedback. Finally, we investigate a decentralized output synchronization problem of heterogeneous linear systems. We develop a self-triggered output broadcasting policy for the interconnected systems. Broadcasting time instants adapt to the current communication topology. For a fixed topology, our broadcasting policy yields global exponential output synchronization, and Lp-stable output synchronization in the presence of disturbances. Employing a converse Lyapunov theorem for impulsive systems, we provide an average dwell time condition that yields disturbance-to-state stable output synchronization in case of switching topology. Our approach is applicable to directed and unbalanced communication topologies.\u2

Model-Predictive Strategy Generation for Multi-Agent Pursuit-Evasion Games

Multi-agent pursuit-evasion games can be used to model a variety of different real world problems including surveillance, search-and-rescue, and defense-related scenarios. However, many pursuit-evasion problems are computationally difficult, which can be problematic for domains with complex geometry or large numbers of agents. To compound matters further, practical applications often require planning methods to operate under high levels of uncertainty or meet strict running-time requirements. These challenges strongly suggest that heuristic methods are needed to address pursuit-evasion problems in the real world. In this dissertation I present heuristic planning techniques for three related problem domains: visibility-based pursuit-evasion, target following with differential motion constraints, and distributed asset guarding with unmanned sea-surface vehicles. For these domains, I demonstrate that heuristic techniques based on problem relaxation and model-predictive simulation can be used to efficiently perform low-level control action selection, motion goal selection, and high-level task allocation. In particular, I introduce a polynomial-time algorithm for control action selection in visibility-based pursuit-evasion games, where a team of pursuers must minimize uncertainty about the location of an evader. The algorithm uses problem relaxation to estimate future states of the game. I also show how to incorporate a probabilistic opponent model learned from interaction traces of prior games into the algorithm. I verify experimentally that by performing Monte Carlo sampling over the learned model to estimate the location of the evader, the algorithm performs better than existing planning approaches based on worst-case analysis. Next, I introduce an algorithm for motion goal selection in pursuit-evasion scenarios with unmanned boats. I show how a probabilistic model accounting for differential motion constraints can be used to project the future positions of the target boat. Motion goals for the pursuer boat can then be selected based on those projections. I verify experimentally that motion goals selected with this technique are better optimized for travel time and proximity to the target boat when compared to motion goals selected based on the current position of the target boat. Finally, I introduce a task-allocation technique for a team of unmanned sea-surface vehicles (USVs) responsible for guarding a high-valued asset. The team of USVs must intercept and block a set of hostile intruder boats before they reach the asset. The algorithm uses model-predictive simulation to estimate the value of high-level task assignments, which are then realized by a set of learned low-level behaviors. I show experimentally that using model-predictive simulations based on Monte-Carlo sampling is more effective than hand-coded evaluation heuristics

Guaranteed Road Network Search with Small Unmanned Aircraft

The use of teams of small unmanned aircraft in real-world rapid-response missions is fast becoming a reality. One such application is search and detection of an evader in urban areas. This paper draws on results in graph-based pursuit-evasion, developing mappings from these abstractions to primitive motions that may be performed by aircraft, to produce search strategies providing guaranteed capture of road-bound targets. The first such strategy is applicable to evaders of arbitrary speed and agility, offering a conservative solution that is insensitive to motion constraints pursuers may possess. This is built upon to generate two strategies for capture of targets having a known speed bound that require searcher teams of much smaller size. The efficacy of these algorithms is demonstrated by evaluation in extensive simulation using realistic vehicle models across a spectrum of environment classes

Mosquito-inspired Swarming and Pursuit for Autonomous Rotorcraft

The long-term goal of this research is to design cooperative-control algorithms for autonomous vehicles inspired by the collective behaviors in animal groups. The specific research objectives of this dissertation are twofold: (1) to analyze and model the swarming and pursuit behaviors observed in the mating swarms of mosquitoes, and (2) to design mosquito-inspired control algorithms to perform swarming and pursuit with autonomous rotorcraft. The first part of this dissertation analyzes the reconstructed flight data of the malarial mosquito Anopheles gambiae to characterize the velocity-alignment interaction between male mosquitoes, who aggregate to form mating swarms and subsequently pursue a female mosquito. Both swarming and pursuit behaviors are represented using self-propelled particle models. The model is used together with tools from control theory to investigate the connection between velocity-alignment behavior and success in pursuit. The results of this research have a potential impact on vector-control methods for malaria, and are also utilized in the second part of this dissertation. The second part of this dissertation studies two types of pursuit problems inspired by the collective behavior in mosquito swarms. The first problem considers the strategy for a single pursuer chasing a single target. This problem has been studied extensively for the application to missile guidance and navigation. Here, we tailor the assumptions on the dynamics of the agents as well as the design criteria for the application to small and agile rotorcraft. The second pursuit problem incorporates the swarming behavior by considering a scenario in which multiple guardian vehicles are deployed to protect an area against fast intruders. We derive necessary and sufficient conditions for capturing the intruder. We also present swarming strategies to maximize the performance of the guardians, inspired by the random-oscillatory motion and the velocity-alignment behavior of male mosquitoes

Sensor-Based Topological Coverage And Mapping Algorithms For Resource-Constrained Robot Swarms

Coverage is widely known in the field of sensor networks as the task of deploying sensors to completely cover an environment with the union of the sensor footprints. Related to coverage is the task of exploration that includes guiding mobile robots, equipped with sensors, to map an unknown environment (mapping) or clear a known environment (searching and pursuit- evasion problem) with their sensors. This is an essential task for robot swarms in many robotic applications including environmental monitoring, sensor deployment, mine clearing, search-and-rescue, and intrusion detection. Utilizing a large team of robots not only improves the completion time of such tasks, but also improve the scalability of the applications while increasing the robustness to systems’ failure. Despite extensive research on coverage, mapping, and exploration problems, many challenges remain to be solved, especially in swarms where robots have limited computational and sensing capabilities. The majority of approaches used to solve the coverage problem rely on metric information, such as the pose of the robots and the position of obstacles. These geometric approaches are not suitable for large scale swarms due to high computational complexity and sensitivity to noise. This dissertation focuses on algorithms that, using tools from algebraic topology and bearing-based control, solve the coverage related problem with a swarm of resource-constrained robots. First, this dissertation presents an algorithm for deploying mobile robots to attain a hole-less sensor coverage of an unknown environment, where each robot is only capable of measuring the bearing angles to the other robots within its sensing region and the obstacles that it touches. Next, using the same sensing model, a topological map of an environment can be obtained using graph-based search techniques even when there is an insufficient number of robots to attain full coverage of the environment. We then introduce the landmark complex representation and present an exploration algorithm that not only is complete when the landmarks are sufficiently dense but also scales well with any swarm size. Finally, we derive a multi-pursuers and multi-evaders planning algorithm, which detects all possible evaders and clears complex environments

On the role and opportunities in teamwork design for advanced multi-robot search systems

Intelligent robotic systems are becoming ever more present in our lives across a multitude of domains such as industry, transportation, agriculture, security, healthcare and even education. Such systems enable humans to focus on the interesting and sophisticated tasks while robots accomplish tasks that are either too tedious, routine or potentially dangerous for humans to do. Recent advances in perception technologies and accompanying hardware, mainly attributed to rapid advancements in the deep-learning ecosystem, enable the deployment of robotic systems equipped with onboard sensors as well as the computational power to perform autonomous reasoning and decision making online. While there has been significant progress in expanding the capabilities of single and multi-robot systems during the last decades across a multitude of domains and applications, there are still many promising areas for research that can advance the state of cooperative searching systems that employ multiple robots. In this article, several prospective avenues of research in teamwork cooperation with considerable potential for advancement of multi-robot search systems will be visited and discussed. In previous works we have shown that multi-agent search tasks can greatly benefit from intelligent cooperation between team members and can achieve performance close to the theoretical optimum. The techniques applied can be used in a variety of domains including planning against adversarial opponents, control of forest fires and coordinating search-and-rescue missions. The state-of-the-art on methods of multi-robot search across several selected domains of application is explained, highlighting the pros and cons of each method, providing an up-to-date view on the current state of the domains and their future challenges

Data-driven Metareasoning for Collaborative Autonomous Systems

When coordinating their actions to accomplish a mission, the agents in a multi-agent system may use a collaboration algorithm to determine which agent performs which task. This paper describes a novel data-driven metareasoning approach that generates a metareasoning policy that the agents can use whenever they must collaborate to assign tasks. This metareasoning approach collects data about the performance of the algorithms at many decision points and uses this data to train a set of surrogate models that can estimate the expected performance of different algorithms. This yields a metareasoning policy that, based on the current state of the system, estimated the algorithms’ expected performance and chose the best one. For a ship protection scenario, computational results show that one version of the metareasoning policy performed as well as the best component algorithm but required less computational effort. The proposed data-driven metareasoning approach could be a promising tool for developing policies to control multi-agent autonomous systems.This work was supported in part by the U.S. Naval Air Warfare Center-Aircraft Division

Reconstruction, Analysis and Synthesis of Collective Motion

As collective motion plays a crucial role in modern day robotics and engineering, it seems appealing to seek inspiration from nature, which abounds with examples of collective motion (starling flocks, fish schools etc.). This approach towards understanding and reverse-engineering a particular aspect of nature forms the foundation of this dissertation, and its main contribution is threefold. First we identify the importance of appropriate algorithms to extract parameters of motion from sampled observations of the trajectory, and then by assuming an appropriate generative model we turn this into a regularized inversion problem with the regularization term imposing smoothness of the reconstructed trajectory. First we assume a linear triple-integrator model, and by penalizing high values of the jerk path integral we reconstruct the trajectory through an analytical approach. Alternatively, the evolution of a trajectory can be governed by natural Frenet frame equations. Inadequacy of integrability theory for nonlinear systems poses the utmost challenge in having an analytic solution, and forces us to adopt a numerical optimization approach. However, by noting the fact that the underlying dynamics defines a left invariant vector field on a Lie group, we develop a framework based on Pontryagin's maximum principle. This approach toward data smoothing yields a semi-analytic solution. Equipped with appropriate algorithms for trajectory reconstruction we analyze flight data for biological motions, and this marks the second contribution of this dissertation. By analyzing the flight data of big brown bats in two different settings (chasing a free-flying praying mantis and competing with a conspecific to catch a tethered mealworm), we provide evidence to show the presence of a context specific switch in flight strategy. Moreover, our approach provides a way to estimate the behavioral latency associated with these foraging behaviors. On the other hand, we have also analyzed the flight data of European starling flocks, and it can be concluded from our analysis that the flock-averaged coherence (the average cosine of the angle between the velocities of a focal bird and its neighborhood center of mass, averaged over the entire flock) gets maximized by considering 5-7 nearest neighbors. The analysis also sheds some light into the underlying feedback mechanism for steering control. The third and final contribution of this dissertation lies in the domain of control law synthesis. Drawing inspiration from coherent movement of starling flocks, we introduce a strategy (Topological Velocity Alignment) for collective motion, wherein each agent aligns its velocity along the direction of motion of its neighborhood center of mass. A feedback law has also been proposed for achieving this strategy, and we have analyzed two special cases (two-body system; and an N-body system with cyclic interaction) to show effectiveness of our proposed feedback law. It has been observed through numerical simulation and robotic implementation that this approach towards collective motion can give rise to a splitting behavior