Multi-agent persistent monitoring of a finite set of targets

The general problem of multi-agent persistent monitoring finds applications in a variety of domains ranging from meter to kilometer-scale systems, such as surveillance or environmental monitoring, down to nano-scale systems such as tracking biological macromolecules for studying basic biology and disease. The problem can be cast as moving the agents between targets, acquiring information from or in some fashion controlling the states of the targets. Under this formulation, at least two questions need to be addressed. The first is the design of motion trajectories for the agents as they move among the spatially distributed targets and jointly optimize a given cost function that describes some desired application. The second is the design of the controller that an agent will use at a target to steer the target's state as desired. The first question can be viewed in at least two ways: first, as an optimal control problem with the domain of the targets described as a continuous space, and second as a discrete scheduling task. In this work we focus on the second approach, which formulates the target dynamics as a hybrid automaton, and the geometry of the targets as a graph. We show how to find solutions by translating the scheduling problem into a search for the optimal route. With a route specifying the visiting sequence in place, we derive the optimal time the agent spends at each target analytically. The second question, namely that of steering the target's state, can be formulated from the perspective of the target, rather than the agent. The mobile nature of the agents leads to intermittencontrol, such that the controller is assumed to be disconnected when no agent is at the target. The design of the visiting schedule of agents to one target can affect the reachability (controllability) of this target's control system and the design of any specific controller. Existing test techniques for reachability are combined with the idea of lifting to provide conditions on systems such that reachability is maintained in the presence of periodic disconnections from the controller. While considering an intermittently connected control with constraints on the control authority and in the presence of a disturbance, the concept of 'degree of controllability' is introduced. The degree is measured by a region of states that can be brought back to the origin in a given finite time. The size of this region is estimated to evaluate the performance of a given sequence

Persistent monitoring of targets with uncertain states

In a wide range of domains, such as pipeline inspection, surveillance in smart cities and tracking of multiple microparticles by an optical microscope, a common goal is to use mobile agents to persistently monitor a set of targets. We refer to this as the persistent monitoring problem. In this dissertation, we assume that each of these targets has an internal state that evolves with linear stochastic dynamics. The agents can observe these states when they are close to the targets, and the goal is to plan agent trajectories such that the sensed data can be used to minimize the uncertainty of the estimation process. We study scalable approaches for planning agent trajectories that minimize the long term uncertainty of the target states. We design algorithms that are computationally efficient and simple to implement, but grounded in mathematically proven performance guarantees. First we approach the problem from a continuous time perspective with the goal of finding locally optimal agent trajectories using a gradient descent scheme. We assume that trajectories are fully defined by a finite set of parameters and compute the cost gradients. Considering periodic agent trajectories and an infinite time horizon, we prove that, under some natural assumptions, the uncertainty of each target converges to a limit cycle. We also show that, in 1D environments with bounded controls, an optimal control is parametric. In multidimensional settings, we propose an efficient parameterization using Fourier curves. Simulation results show the efficiency of our approach. Next, we consider a graph-constrained, single-agent version of the problem, where agents can only move in the edges of the graph and observe the target when they are visiting the node corresponding to it. We prove that, in this scenario, an optimal policy is such that all the agent have a common peak uncertainty. Using this property of the optimal solution, we develop lightweight algorithms that, instead of directly solving the optimization problem, balance the dwelling times to fulfill such property of an optimal policy. In some particular situations, global optimality of the proposed algorithm is proven. Using a custom-designed greedy exploration scheme, we develop an efficient method for obtaining efficient target visiting sequences. We extended this approach to multi-agent scenarios by using a divide-and conquer strategy, where targets are divided in clusters and each of these clusters is only visited by one agent. Then, we extend those ideas to a discrete time version of the problem. We show that, for a periodic trajectory with fixed cycle length, the problem can be formulated as set of semidefinite programs. This allowed us to leverage efficient SDP solvers to provide fast solutions to the persistent monitoring problem. We design a scheme that leverages the spatial configuration of the targets to guide the search over this set of optimization problems to provide efficient trajectories. Finally we describe an application of the proposed techniques to the problem of tracking multiple diffusing particles using a feedback-driven confocal microscope. The proposed persistent monitoring algorithm was used as the higher level controller in a hierarchical scheme, defining which particle should be tracked at each instant. Then an extremum seeking controller was used as a lower level controller in order to track the moving particle and provide efficient observations

Optimal Event-Driven Multi-Agent Persistent Monitoring of a Finite Set of Targets

We consider the problem of controlling the movement of multiple cooperating agents so as to minimize an uncertainty metric associated with a finite number of targets. In a one-dimensional mission space, we adopt an optimal control framework and show that the solution is reduced to a simpler parametric optimization problem: determining a sequence of locations where each agent may dwell for a finite amount of time and then switch direction. This amounts to a hybrid system which we analyze using Infinitesimal Perturbation Analysis (IPA) to obtain a complete on-line solution through an event-driven gradient-based algorithm which is also robust with respect to the uncertainty model used. The resulting controller depends on observing the events required to excite the gradient-based algorithm, which cannot be guaranteed. We solve this problem by proposing a new metric for the objective function which creates a potential field guaranteeing that gradient values are non-zero. This approach is compared to an alternative graph-based task scheduling algorithm for determining an optimal sequence of target visits. Simulation examples are included to demonstrate the proposed methods.Comment: 12 pages full version, IEEE Conference on Decision and Control, 201

An Optimal Control Approach for the Data Harvesting Problem

We propose a new method for trajectory planning to solve the data harvesting problem. In a two-dimensional mission space, $N$ mobile agents are tasked with the collection of data generated at $M$ stationary sources and delivery to a base aiming at minimizing expected delays. An optimal control formulation of this problem provides some initial insights regarding its solution, but it is computationally intractable, especially in the case where the data generating processes are stochastic. We propose an agent trajectory parameterization in terms of general function families which can be subsequently optimized on line through the use of Infinitesimal Perturbation Analysis (IPA). Explicit results are provided for the case of elliptical and Fourier series trajectories and some properties of the solution are identified, including robustness with respect to the data generation processes and scalability in the size of an event set characterizing the underlying hybrid dynamic system