Multi-agent persistent monitoring of a finite set of targets

The general problem of multi-agent persistent monitoring finds applications in a variety of domains ranging from meter to kilometer-scale systems, such as surveillance or environmental monitoring, down to nano-scale systems such as tracking biological macromolecules for studying basic biology and disease. The problem can be cast as moving the agents between targets, acquiring information from or in some fashion controlling the states of the targets. Under this formulation, at least two questions need to be addressed. The first is the design of motion trajectories for the agents as they move among the spatially distributed targets and jointly optimize a given cost function that describes some desired application. The second is the design of the controller that an agent will use at a target to steer the target's state as desired. The first question can be viewed in at least two ways: first, as an optimal control problem with the domain of the targets described as a continuous space, and second as a discrete scheduling task. In this work we focus on the second approach, which formulates the target dynamics as a hybrid automaton, and the geometry of the targets as a graph. We show how to find solutions by translating the scheduling problem into a search for the optimal route. With a route specifying the visiting sequence in place, we derive the optimal time the agent spends at each target analytically. The second question, namely that of steering the target's state, can be formulated from the perspective of the target, rather than the agent. The mobile nature of the agents leads to intermittencontrol, such that the controller is assumed to be disconnected when no agent is at the target. The design of the visiting schedule of agents to one target can affect the reachability (controllability) of this target's control system and the design of any specific controller. Existing test techniques for reachability are combined with the idea of lifting to provide conditions on systems such that reachability is maintained in the presence of periodic disconnections from the controller. While considering an intermittently connected control with constraints on the control authority and in the presence of a disturbance, the concept of 'degree of controllability' is introduced. The degree is measured by a region of states that can be brought back to the origin in a given finite time. The size of this region is estimated to evaluate the performance of a given sequence

Multi-Robot Path Planning for Persistent Monitoring in Stochastic and Adversarial Environments

In this thesis, we study multi-robot path planning problems for persistent monitoring tasks. The goal of such persistent monitoring tasks is to deploy a team of cooperating mobile robots in an environment to continually observe locations of interest in the environment. Robots patrol the environment in order to detect events arriving at the locations of the environment. The events stay at those locations for a certain amount of time before leaving and can only be detected if one of the robots visits the location of an event while the event is there. In order to detect all possible events arriving at a vertex, the maximum time spent by the robots between visits to that vertex should be less than the duration of the events arriving at that vertex. We consider the problem of finding the minimum number of robots to satisfy these revisit time constraints, also called latency constraints. The decision version of this problem is PSPACE-complete. We provide an O(log p) approximation algorithm for this problem where p is the ratio of the maximum and minimum latency constraints. We also present heuristic algorithms to solve the problem and show through simulations that a proposed orienteering-based heuristic algorithm gives better solutions than the approximation algorithm. We additionally provide an algorithm for the problem of minimizing the maximum weighted latency given a fixed number of robots. In case the event stay durations are not fixed but are drawn from a known distribution, we consider the problem of maximizing the expected number of detected events. We motivate randomized patrolling paths for such scenarios and use Markov chains to represent those random patrolling paths. We characterize the expected number of detected events as a function of the Markov chains used for patrolling and show that the objective function is submodular for randomly arriving events. We propose an approximation algorithm for the case where the event durations for all the vertices is a constant. We also propose a centralized and an online distributed algorithm to find the random patrolling policies for the robots. We also consider the case where the events are adversarial and can choose where and when to appear in order to maximize their chances of remaining undetected. The last problem we study in this thesis considers events triggered by a learning adversary. The adversary has a limited time to observe the patrolling policy before it decides when and where events should appear. We study the single robot version of this problem and model this problem as a multi-stage two player game. The adversary observes the patroller’s actions for a finite amount of time to learn the patroller’s strategy and then either chooses a location for the event to appear or reneges based on its confidence in the learned strategy. We characterize the expected payoffs for the players and propose a search algorithm to find a patrolling policy in such scenarios. We illustrate the trade off between hard to learn and hard to attack strategies through simulations

Cooperative control for multi-agent persistent monitoring problems

In persistent monitoring tasks, cooperating mobile agents are used to monitor a dynamically changing environment that cannot be fully covered by a stationary team of agents. The exploration process leads to the discovery of various "points of interest" (targets) to be perpetually monitored. Through an optimal control approach, the first part of this dissertation shows that in a one-dimensional mission space the solution can be reduced to a simpler parametric problem. The behavior of agents under optimal control is described by a hybrid system which can be analyzed using Infinitesimal Perturbation Analysis (IPA) to obtain an on-line solution. IPA allows the modeling of virtually arbitrary stochastic effects in target uncertainty and its event-driven nature renders the solution scalable in the number of events rather than the state space. The second part of this work extends the results of the one-dimensional persistent monitoring problem to a two-dimensional space with constrained agent mobility. Under a general graph setting, the properties of the one-dimensional optimal control solution are largely inherited. The solution involves the design of agent trajectories defined by both the sequence of nodes to be visited and the amount of time spent at each node. A class of distributed threshold-based parametric controllers is proposed to reduce the computational complexity. These parameters are optimized through an event-driven IPA gradient-based algorithm and yield optimal controllers within this family of threshold-based policies. The performance of the threshold-based parametric controller is close to that of the optimal controller derived through dynamic programming and its computational complexity is smaller by orders of magnitude. Although effective, the aforementioned optimal controls are established on the assumption that agents are all connected via a centralized controller which is energy-consuming and unreliable in adversarial environments. The third part of this work extends the previous controls by developing decentralized controllers which distribute functionality to the agents so that each one acts upon local information and sparse communication with neighbors. The complexity of decentralization for persistent monitoring problems is significant given agent mobility and the overall time-varying graph topology. Conditions are identified and a decentralized framework is proposed under which the centralized solution can be exactly recovered in a decentralized event-driven manner based on local information -- except for one event requiring communication from a non-neighbor agent