Overcoming local optima in control and optimization of cooperative multi-agent systems

A cooperative multi-agent system is a collection of interacting agents deployed in a mission space where each agent is allowed to control its local state so that the fleet of agents collectively optimizes a common global objective. While optimization problems associated with multi-agent systems intend to determine the fixed set of globally optimal agent states, control problems aim to obtain the set of globally optimal agent controls. Associated non-convexities in these problems result in multiple local optima. This dissertation explores systematic techniques that can be deployed to either escape or avoid poor local optima while in search of provably better (still local) optima. First, for multi-agent optimization problems with iterative gradient-based solutions, a distributed approach to escape local optima is proposed based on the concept of boosting functions. These functions temporarily transform gradient components at a local optimum into a set of boosted non-zero gradient components in a systematic manner so that it is more effective compared to the methods where gradient components are randomly perturbed. A novel variable step size adjustment scheme is also proposed to establish the convergence of this distributed boosting process. Developed boosting concepts are successfully applied to the class of coverage problems. Second, as a means of avoiding convergence to poor local optima in multi-agent optimization, the use of greedy algorithms in generating effective initial conditions is explored. Such greedy methods are computationally cheap and can often exploit submodularity properties of the problem to provide performance bound guarantees to the obtained solutions. For the class of submodular maximization problems, two new performance bounds are proposed and their effectiveness is illustrated using the class of coverage problems. Third, a class of multi-agent control problems termed Persistent Monitoring on Networks (PMN) is considered where a team of agents is traversing a set of nodes (targets) interconnected according to a network topology aiming to minimize a measure of overall node state. For this class of problems, a gradient-based parametric control solution developed in a prior work relies heavily on the initial selection of its `parameters' which often leads to poor local optima. To overcome this initialization challenge, the PMN system's asymptotic behavior is analyzed, and an off-line greedy algorithm is proposed to systematically generate an effective set of initial parameters. Finally, for the same class of PMN problems, a computationally efficient distributed on-line Event-Driven Receding Horizon Control (RHC) solution is proposed as an alternative. This RHC solution is parameter-free as it automatically optimizes its planning horizon length and gradient-free as it uses explicitly derived solutions for each RHC problem invoked at each agent upon each event of interest. Hence, unlike the gradient-based parametric control solutions, the proposed RHC solution does not force the agents to converge to one particular behavior that is likely to be a poor local optimum. Instead, it keeps the agents actively searching for the optimum behavior. In each of these four parts of the thesis, an interactive simulation platform is developed (and made available online) to generate extensive numerical examples that highlight the respective contributions made compared to the state of the art

Persistent monitoring of targets with uncertain states

In a wide range of domains, such as pipeline inspection, surveillance in smart cities and tracking of multiple microparticles by an optical microscope, a common goal is to use mobile agents to persistently monitor a set of targets. We refer to this as the persistent monitoring problem. In this dissertation, we assume that each of these targets has an internal state that evolves with linear stochastic dynamics. The agents can observe these states when they are close to the targets, and the goal is to plan agent trajectories such that the sensed data can be used to minimize the uncertainty of the estimation process. We study scalable approaches for planning agent trajectories that minimize the long term uncertainty of the target states. We design algorithms that are computationally efficient and simple to implement, but grounded in mathematically proven performance guarantees. First we approach the problem from a continuous time perspective with the goal of finding locally optimal agent trajectories using a gradient descent scheme. We assume that trajectories are fully defined by a finite set of parameters and compute the cost gradients. Considering periodic agent trajectories and an infinite time horizon, we prove that, under some natural assumptions, the uncertainty of each target converges to a limit cycle. We also show that, in 1D environments with bounded controls, an optimal control is parametric. In multidimensional settings, we propose an efficient parameterization using Fourier curves. Simulation results show the efficiency of our approach. Next, we consider a graph-constrained, single-agent version of the problem, where agents can only move in the edges of the graph and observe the target when they are visiting the node corresponding to it. We prove that, in this scenario, an optimal policy is such that all the agent have a common peak uncertainty. Using this property of the optimal solution, we develop lightweight algorithms that, instead of directly solving the optimization problem, balance the dwelling times to fulfill such property of an optimal policy. In some particular situations, global optimality of the proposed algorithm is proven. Using a custom-designed greedy exploration scheme, we develop an efficient method for obtaining efficient target visiting sequences. We extended this approach to multi-agent scenarios by using a divide-and conquer strategy, where targets are divided in clusters and each of these clusters is only visited by one agent. Then, we extend those ideas to a discrete time version of the problem. We show that, for a periodic trajectory with fixed cycle length, the problem can be formulated as set of semidefinite programs. This allowed us to leverage efficient SDP solvers to provide fast solutions to the persistent monitoring problem. We design a scheme that leverages the spatial configuration of the targets to guide the search over this set of optimization problems to provide efficient trajectories. Finally we describe an application of the proposed techniques to the problem of tracking multiple diffusing particles using a feedback-driven confocal microscope. The proposed persistent monitoring algorithm was used as the higher level controller in a hierarchical scheme, defining which particle should be tracked at each instant. Then an extremum seeking controller was used as a lower level controller in order to track the moving particle and provide efficient observations

Optimal Minimax Mobile Sensor Scheduling Over a Network

We investigate the problem of monitoring multiple targets using a single mobile sensor, with the goal of minimizing the maximum estimation error among all the targets over long time horizons. The sensor can move in a network-constrained structure, where it has to plan which targets to visit and for how long to dwell at each node. We prove that in an optimal observation time allocation, the peak uncertainty is the same among all the targets. By further restricting the agent policy to only visit each target once every cycle, we develop a scheme to optimize the agent's behavior that is significantly simpler computationally when compared to previous approaches for similar problems

Advancing Robot Autonomy for Long-Horizon Tasks

Autonomous robots have real-world applications in diverse fields, such as mobile manipulation and environmental exploration, and many such tasks benefit from a hands-off approach in terms of human user involvement over a long task horizon. However, the level of autonomy achievable by a deployment is limited in part by the problem definition or task specification required by the system. Task specifications often require technical, low-level information that is unintuitive to describe and may result in generic solutions, burdening the user technically both before and after task completion. In this thesis, we aim to advance task specification abstraction toward the goal of increasing robot autonomy in real-world scenarios. We do so by tackling problems that address several different angles of this goal. First, we develop a way for the automatic discovery of optimal transition points between subtasks in the context of constrained mobile manipulation, removing the need for the human to hand-specify these in the task specification. We further propose a way to automatically describe constraints on robot motion by using demonstrated data as opposed to manually-defined constraints. Then, within the context of environmental exploration, we propose a flexible task specification framework, requiring just a set of quantiles of interest from the user that allows the robot to directly suggest locations in the environment for the user to study. We next systematically study the effect of including a robot team in the task specification and show that multirobot teams have the ability to improve performance under certain specification conditions, including enabling inter-robot communication. Finally, we propose methods for a communication protocol that autonomously selects useful but limited information to share with the other robots.Comment: PhD dissertation. 160 page

An Algorithmic View on Sensor Networks - Surveillance, Localization, and Communication

This thesis focuses on scalability issues of diverse problems on sensor networks and presents efficient solutions. First, we show that it is NP-hard to find optimal activation schedules for monitoring areas and provide an EPTAS algorithm. Second, we present a distributed algorithm for the detection of network boundaries that only requires local connectivity information. Finally, we introduce an FPTAS for computing shortest paths and describe an algorithm for determining alternative routes

Representation Learning: A Review and New Perspectives

The success of machine learning algorithms generally depends on data representation, and we hypothesize that this is because different representations can entangle and hide more or less the different explanatory factors of variation behind the data. Although specific domain knowledge can be used to help design representations, learning with generic priors can also be used, and the quest for AI is motivating the design of more powerful representation-learning algorithms implementing such priors. This paper reviews recent work in the area of unsupervised feature learning and deep learning, covering advances in probabilistic models, auto-encoders, manifold learning, and deep networks. This motivates longer-term unanswered questions about the appropriate objectives for learning good representations, for computing representations (i.e., inference), and the geometrical connections between representation learning, density estimation and manifold learning