Developing an online cooperative police patrol routing strategy

A cooperative routing strategy for daily operations is necessary to maintain the effects of hotspot policing and to reduce crime and disorder. Existing robot patrol routing strategies are not suitable, as they omit the peculiarities and challenges of daily police patrol including minimising the average time lag between two consecutive visits to hotspots, as well as coordinating multiple patrollers and imparting unpredictability to patrol routes. In this research, we propose a set of guidelines for patrol routing strategies to meet the challenges of police patrol. Following these guidelines, we develop an innovative heuristic-based and Bayesian-inspired real-time strategy for cooperative routing police patrols. Using two real-world cases and a benchmark patrol strategy, an online agent-based simulation has been implemented to testify the efficiency, flexibility, scalability, unpredictability, and robustness of the proposed strategy and the usability of the proposed guidelines

An Energy-aware, Fault-tolerant, and Robust Deep Reinforcement Learning based approach for Multi-agent Patrolling Problems

Autonomous vehicles are suited for continuous area patrolling problems. However, finding an optimal patrolling strategy can be challenging for many reasons. Firstly, patrolling environments are often complex and can include unknown environmental factors. Secondly, autonomous vehicles can have failures or hardware constraints, such as limited battery life. Importantly, patrolling large areas often requires multiple agents that need to collectively coordinate their actions. In this work, we consider these limitations and propose an approach based on model-free, deep multi-agent reinforcement learning. In this approach, the agents are trained to automatically recharge themselves when required, to support continuous collective patrolling. A distributed homogeneous multi-agent architecture is proposed, where all patrolling agents execute identical policies locally based on their local observations and shared information. This architecture provides a fault-tolerant and robust patrolling system that can tolerate agent failures and allow supplementary agents to be added to replace failed agents or to increase the overall patrol performance. The solution is validated through simulation experiments from multiple perspectives, including the overall patrol performance, the efficiency of battery recharging strategies, and the overall fault tolerance and robustness

A Multi-Vehicle Cooperative Localization Approach for an Autonomy Framework

Offensive techniques produced by technological advancement present opportunities for adversaries to threaten the operational advantages of our joint and allied forces. Combating these new methodologies requires continuous and rapid development towards our own set of \game-changing technologies. Through focused development of unmanned systems and autonomy, the Air Force can strive to maintain its technological superiority. Furthermore, creating a robust framework capable of testing and evaluating the principles that define autonomy allows for the exploration of future capabilities. This research presents development towards a hybrid reactive/deliberative architecture that will allow for the testing of the principles of task, cognitive, and peer flexibility. Specifically, this work explores peer flexibility in multi-robot systems to solve a localization problem using the Hybrid Architecture for Multiple Robots (HAMR) as a basis for the framework. To achieve this task a combination of vehicle perception and navigation tools formulate inferences on an operating environment. These inferences are then used for the construction of Factor Graphs upon which the core algorithm for localization implements iSAM2, a high performing incremental matrix factorization method. A key component for individual vehicle control within the framework is the Unified Behavior Framework (UBF), a behavior-based control architecture which uses modular arbitration techniques to generate actions that enable actuator control. Additionally, compartmentalization of a World Model is explored through the use of containers to minimize communication overhead and streamline state information. The design for this platform takes on a polymorphic approach for modularity and robustness enabling future development

Effective Cooperation and Scalability in Multi-Robot Teams for Automatic Patrolling of Infrastructures

Tese de doutoramento em Engenharia Electrotécnica e de Computadores, apresentada ao Departamento de Engenharia Electrotécnica e de Computadores da Faculdade de Ciências e Tecnologia da Universidade de CoimbraIn the digital era that we live in, advances in technology have proliferated throughout our society, quickening the completion of tasks that were painful in the old days, improving solutions to the everyday problems that we face, and generally assisting human beings both in their professional and personal life. Robotics is a clear example of a broad technological field that evolves every day. In fact, scientists predict that in the upcoming few decades, robots will naturally interact and coexist alongside human beings. While it is true that robots already have a strong presence in industrial environments, e.g., robotic arms for manufacturing, the average person still looks upon robots with suspicion, since they are not acquainted by such type of technology. In this thesis, the author deploys teams of mobile robots in indoor scenarios to cooperatively perform patrolling missions, which represents an effort to bring robots closer to humans and assist them in monotonous or repetitive tasks, such as supervising and monitoring indoor infrastructures or simply cooperatively cleaning floors. In this context, the team of robots should be able to sense the environment, localize and navigate autonomously between way points while avoiding obstacles, incorporate any number of robots, communicate actions in a distributed way and being robust not only to agent failures but also communication failures, so as to effectively coordinate to achieve optimal collective performance. The referred capabilities are an evidence that such systems can only prove their reliability in real-world environments if robots are endowed with intelligence and autonomy. Thus, the author follows a line of research where patrolling units have the necessary tools for intelligent decision-making, according to the information of the mission, the environment and teammates' actions, using distributed coordination architectures. An incremental approach is followed. Firstly, the problem is presented and the literature is deeply studied in order to identify potential weaknesses and research opportunities, backing up the objectives and contributions proposed in this thesis. Then, problem fundamentals are described and benchmarking of multi-robot patrolling algorithms in realistic conditions is conducted. In these earlier stages, the role of different parameters of the problem, like environment connectivity, team size and strategy philosophy, will become evident through extensive empirical results and statistical analysis. In addition, scalability is deeply analyzed and tied with inter-robot interference and coordination, imposed by each patrolling strategy. After gaining sensibility to the problem, preliminary models for multi-robot patrol with special focus on real-world application are presented, using a Bayesian inspired formalism. Based on these, distributed strategies that lead to superior team performance are described. Interference between autonomous agents is explicitly dealt with, and the approaches are shown to scale to large teams of robots. Additionally, the robustness to agent and communication failures is demonstrated, as well as the flexibility of the model proposed. In fact, by later generalizing the model with learning agents and maintaining memory of past events, it is then shown that these capabilities can be inherited, while at the same time increasing team performance even further and fostering adaptability. This is verified in simulation experiments and real-world results in a large indoor scenario. Furthermore, since the issue of team scalability is highly in focus in this thesis, a method for estimating the optimal team size in a patrolling mission, according to the environment topology is proposed. Upper bounds for team performance prior to the mission start are provided, supporting the choice of the number of robots to be used so that temporal constraints can be satisfied. All methods developed in this thesis are tested and corroborated by experimental results, showing the usefulness of employing cooperative teams of robots in real-world environments and the potential for similar systems to emerge in our society.FCT - SFRH/BD/64426/200

Near-Optimal Adversarial Policy Switching for Decentralized Asynchronous Multi-Agent Systems

A key challenge in multi-robot and multi-agent systems is generating solutions that are robust to other self-interested or even adversarial parties who actively try to prevent the agents from achieving their goals. The practicality of existing works addressing this challenge is limited to only small-scale synchronous decision-making scenarios or a single agent planning its best response against a single adversary with fixed, procedurally characterized strategies. In contrast this paper considers a more realistic class of problems where a team of asynchronous agents with limited observation and communication capabilities need to compete against multiple strategic adversaries with changing strategies. This problem necessitates agents that can coordinate to detect changes in adversary strategies and plan the best response accordingly. Our approach first optimizes a set of stratagems that represent these best responses. These optimized stratagems are then integrated into a unified policy that can detect and respond when the adversaries change their strategies. The near-optimality of the proposed framework is established theoretically as well as demonstrated empirically in simulation and hardware

Overview of Key Technologies for Water-based Automatic Security Marking Platform

Water-based automatic security marking platform composed of multifunctional underwater robots and unmanned surface vessel has become the development trend and focus for exploring complex and dangerous waters,and its related technologies have flourished and gradually developed from single control to multi-platform collaborative direction in complex and dangerous waters to reduce casualties. This paper composes and analyzes the key technologies of the water-based automatic security marking platform based on the cable underwater robot and the unmanned surface vessel, describes the research and application status of the key technologies of the water-based automatic security marking platform from the aspects of the unmanned surface vessel, underwater robot and underwater multisensor information fusion, and outlooks the research direction and focus of the water automatic security inspection and marking platform

Semi-Informed Multi-Agent Patrol Strategies

The adversarial multi-agent patrol problem is an active research topic with many real-world applications such as physical robots guarding an area and software agents protecting a computer network. In it, agents patrol a graph looking for so-called critical vertices that are subject to attack by adversaries. The agents are unaware of which vertices are subject to attack by adversaries and when they encounter such a vertex they attempt to protect it from being compromised (an adversary must occupy the vertex it targets a certain amount of time for the attack to succeed). Even though the terms adversary and attack are used, the problem domain extends to patrolling a graph for other interesting noncompetitive contexts such as search and rescue. The problem statement adopted in this work is formulated such that agents obtain knowledge of local graph topology and critical vertices over the course of their travels via an API ; there is no global knowledge of the graph or communication between agents. The challenge is to balance exploration, necessary to discover critical vertices, with exploitation, necessary to protect critical vertices from attack. Four types of adversaries were used for experiments, three from previous research – waiting, random, and statistical - and the fourth, a hybrid of those three. Agent strategies for countering each of these adversaries are designed and evaluated. Benchmark graphs and parameter settings from related research will be employed. The proposed research culminates in the design and evaluation of agents to counter these various types of adversaries under a range of conditions. The results of this work are agent strategies in which each agent becomes solely responsible for protecting those critical vertices it discovers. The agents use emergent behavior to minimize successful attacks and maximize the discovery of new critical vertices. A set of seven edge choosing primitives (ECPs) are defined that are combined in different ways to yield a range of agent strategies using the chain of responsibility OOP design pattern. Every permutation of them were tested and measured in order to identify those strategies that perform well. One strategy performed particularly well against all adversaries, graph topology, and other experimental variables. This particular strategy combines ECPs of: A hard-deadline return to covered vertices to counter the random adversary, efficiently checking vertices to see if they are being attacked by the waiting adversary, and random movement to impede the statistical adversary