On Games Arising From Multi-Depot Chinese Postman Problems

This paper introduces cooperative games arising from multi-depot Chinese postman problems and explores the properties of these games. A multi-depot Chinese postman problem (MDCP) is represented by a connected (di)graph G, a set of k depots that is a subset of the vertices of G, and a non-negative weight function on the edges of G. A solution to the MDCP is a minimum weight tour of the (di)graph that visits all edges (arcs) of the graph and that consists of a collection of subtours such that the subtours originate from dierent depots, and each subtour starts and ends at the same depot. A cooperative Chinese postman (CP) game is induced by a MDCP by associating every edge of the graph with a dierent player. This paper characterizes globally and locally k-CP balanced and submodular (di)graphs. A (di)graph G is called globally (locally) k-CP balanced (respectively submodular), if the induced CP game of the corresponding MDCP problem on G is balanced (respectively submodular) for any (some) choice of the locations of the k depots and every non-negative weight function

Monitoring using Heterogeneous Autonomous Agents.

This dissertation studies problems involving different types of autonomous agents observing objects of interests in an area. Three types of agents are considered: mobile agents, stationary agents, and marsupial agents, i.e., agents capable of deploying other agents or being deployed themselves. Objects can be mobile or stationary. The problem of a mobile agent without fuel constraints revisiting stationary objects is formulated. Visits to objects are dictated by revisit deadlines, i.e., the maximum time that can elapse between two visits to the same object. The problem is shown to be NP-complete and heuristics are provided to generate paths for the agent. Almost periodic paths are proven to exist. The efficacy of the heuristics is shown through simulation. A variant of the problem where the agent has a finite fuel capacity and purchases fuel is treated. Almost periodic solutions to this problem are also shown to exist and an algorithm to compute the minimal cost path is provided. A problem where mobile and stationary agents cooperate to track a mobile object is formulated, shown to be NP-hard, and a heuristic is given to compute paths for the mobile agents. Optimal configurations for the stationary agents are then studied. Several methods are provided to optimally place the stationary agents; these methods are the maximization of Fisher information, the minimization of the probability of misclassification, and the minimization of the penalty incurred by the placement. A method to compute optimal revisit deadlines for the stationary agents is given. The placement methods are compared and their effectiveness shown using numerical results. The problem of two marsupial agents, one carrier and one passenger, performing a general monitoring task using a constrained optimization formulation is stated. Necessary conditions for optimal paths are provided for cases accounting for constrained release of the passenger, termination conditions for the task, as well as retrieval and constrained retrieval of the passenger. A problem involving two marsupial agents collecting information about a stationary object while avoiding detection is then formulated. Necessary conditions for optimal paths are provided and rectilinear motion is demonstrated to be optimal for both agents.PhDAerospace EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/111439/1/jfargeas_1.pd

Multi-Robot Path Planning for Persistent Monitoring in Stochastic and Adversarial Environments

In this thesis, we study multi-robot path planning problems for persistent monitoring tasks. The goal of such persistent monitoring tasks is to deploy a team of cooperating mobile robots in an environment to continually observe locations of interest in the environment. Robots patrol the environment in order to detect events arriving at the locations of the environment. The events stay at those locations for a certain amount of time before leaving and can only be detected if one of the robots visits the location of an event while the event is there. In order to detect all possible events arriving at a vertex, the maximum time spent by the robots between visits to that vertex should be less than the duration of the events arriving at that vertex. We consider the problem of finding the minimum number of robots to satisfy these revisit time constraints, also called latency constraints. The decision version of this problem is PSPACE-complete. We provide an O(log p) approximation algorithm for this problem where p is the ratio of the maximum and minimum latency constraints. We also present heuristic algorithms to solve the problem and show through simulations that a proposed orienteering-based heuristic algorithm gives better solutions than the approximation algorithm. We additionally provide an algorithm for the problem of minimizing the maximum weighted latency given a fixed number of robots. In case the event stay durations are not fixed but are drawn from a known distribution, we consider the problem of maximizing the expected number of detected events. We motivate randomized patrolling paths for such scenarios and use Markov chains to represent those random patrolling paths. We characterize the expected number of detected events as a function of the Markov chains used for patrolling and show that the objective function is submodular for randomly arriving events. We propose an approximation algorithm for the case where the event durations for all the vertices is a constant. We also propose a centralized and an online distributed algorithm to find the random patrolling policies for the robots. We also consider the case where the events are adversarial and can choose where and when to appear in order to maximize their chances of remaining undetected. The last problem we study in this thesis considers events triggered by a learning adversary. The adversary has a limited time to observe the patrolling policy before it decides when and where events should appear. We study the single robot version of this problem and model this problem as a multi-stage two player game. The adversary observes the patroller’s actions for a finite amount of time to learn the patroller’s strategy and then either chooses a location for the event to appear or reneges based on its confidence in the learned strategy. We characterize the expected payoffs for the players and propose a search algorithm to find a patrolling policy in such scenarios. We illustrate the trade off between hard to learn and hard to attack strategies through simulations

Graph methods in Multi Agent Systems Coordination and Social Network Analysis

In this thesis several results on two main topics are collected: the coordination of networked multi agents systems and the diffusion of innovation of social networks. The results are organized in two parts, each one related with one of the two main topics. The common aspect of all the presented problems is the following: all the system are represented by graphs. Two are the main contributions of the first part. • A formation control strategy, based on gossip, which leads a set of autonomous vehicles to converge to a desired spatial disposition in absence of a common reference frame. If the vehicles have common direction, we prove that the proposed algorithm is robust against noise on displacement measurement. • \item The formalization of the Heterogeneous Multi Vehicle Routing Problem, which can be described as follows: given an heterogeneous set of mobile robots, and a set of task to be served randomly displaced in a 2D environment, find the optimal task assignment to minimize the service cost. We firstly characterize the optimal centralized solution, and then we propose two distributed algorithms, based on gossip, which lead the system to a sub-optimal solutions and are significantly computationally more efficient than the optimal one. The contributions of the second part are the following. • We study how the innovation spreads in a Social Network according to the so called Linear Threshold Model, in which the innovation is incepted in the network starting from a seed set, and nodes adopt the innovation if the ratio of the neighbours that have already adopted it is greater than or equal a certain threshold value. We focus on the cohesive subset of the network, which can be used to compute the set of final adopters. If a set is cohesive and none of the nodes have adopted the innovation at a certain time $t$, then they are not able to adopt the innovation at any $t'>t$. We propose an algorithm based on linear programming which computes the maximal cohesive subset of the complement of the seed set. • According to the Linear Threshold Model, we define two problem of interest in Social Networks analysis and characterize the optimal solution: the Influence Maximization Problem in Finite Time and the diffusion of innovation over a target set. • We characterize the novel Non Progressive Linear Threshold Model, which extends the classical Linear Threshold Model. We formalize the model and we give a characterization of the network dynamics in terms of cohesive and persistent set

Graph methods in Multi Agent Systems Coordination and Social Network Analysis

In this thesis several results on two main topics are collected: the coordination of networked multi agents systems and the diffusion of innovation of social networks. The results are organized in two parts, each one related with one of the two main topics. The common aspect of all the presented problems is the following: all the system are represented by graphs. Two are the main contributions of the first part. • A formation control strategy, based on gossip, which leads a set of autonomous vehicles to converge to a desired spatial disposition in absence of a common reference frame. If the vehicles have common direction, we prove that the proposed algorithm is robust against noise on displacement measurement. • \item The formalization of the Heterogeneous Multi Vehicle Routing Problem, which can be described as follows: given an heterogeneous set of mobile robots, and a set of task to be served randomly displaced in a 2D environment, find the optimal task assignment to minimize the service cost. We firstly characterize the optimal centralized solution, and then we propose two distributed algorithms, based on gossip, which lead the system to a sub-optimal solutions and are significantly computationally more efficient than the optimal one. The contributions of the second part are the following. • We study how the innovation spreads in a Social Network according to the so called Linear Threshold Model, in which the innovation is incepted in the network starting from a seed set, and nodes adopt the innovation if the ratio of the neighbours that have already adopted it is greater than or equal a certain threshold value. We focus on the cohesive subset of the network, which can be used to compute the set of final adopters. If a set is cohesive and none of the nodes have adopted the innovation at a certain time $t$, then they are not able to adopt the innovation at any $t'>t$. We propose an algorithm based on linear programming which computes the maximal cohesive subset of the complement of the seed set. • According to the Linear Threshold Model, we define two problem of interest in Social Networks analysis and characterize the optimal solution: the Influence Maximization Problem in Finite Time and the diffusion of innovation over a target set. • We characterize the novel Non Progressive Linear Threshold Model, which extends the classical Linear Threshold Model. We formalize the model and we give a characterization of the network dynamics in terms of cohesive and persistent set

Essays in Retail Operations and Humanitarian Logistics

This dissertation introduces and analyzes research problems related to Retail Operations and Humanitarian Logistics. In Retail Operations, the inventory that ends up as unsaleable at primary markets can be significant (up to 20% of the retail product). Thus retailers look for strategies like selling in secondary markets at a discounted price. In such a setting, the decisions of how much to order for a product of limited shelf life and when (if at all) to start selling the product in the secondary market become critical because these decisions not only affect the retailer's cost of procurement and sales revenues obtained from the product but also affect utilization of shelf space, product rollover and assortment decisions of the retailer. Apart from using secondary markets, retailers that sell seasonal products or products with sales horizons shorter than the typical production/procurement lead time also enter into contractual agreements with suppliers. These contracts are in place to share risks associated with unknown or uncertain demand for the product. Presence of such contracts does affect a retailer's order quantity as well as the time to start selling in the secondary market. In our two essays on retail operations, we analyze a retailer's optimal order quantity and when he/she starts selling in the secondary market. We refer to the former as the 'ordering decision' and the latter as the 'timing decision.' These two decisions are studied first without risk sharing contracts in Essay 1, and then in the presence of contracts in Essay 2. In Essay 1, we build a two-stage model with demand uncertainty. The ordering decision is made in the first stage considering cost of procurement and expected sales revenue. The timing decision is made in the second stage and is conditional on the order quantity determined in the first stage. We introduce a new class of aggregate demand model for this model. We study the structural properties of the retailer's timing and ordering problem and identify optimality conditions for the timing decision. Finally, we complement our analytical results with computational experiments and show how retailer's optimal decisions change when problem parameters are varied. In Essay 2, we extend the work in first essay to include the contracts between the retailer and a supplier. In this essay, we introduce a time-based Poisson demand model. We define three di®erent types of contracts and investigate the effect of each of these contracts on the retailer's ordering and timing decisions. We investigate how the analytical structure of the retailer's decision changes in the presence of these contracts. For a given order quantity, we show that the timing decision depends on the type of contract. Our analytical results on the timing decision are complemented with computational experiments where we investigate the impact of contract type on the optimal order quantity of the retailer. In Humanitarian Logistics, non-profit organizations receive several-billion-dollars-worth of donations every year but lack a sophisticated system to handle their complex logistics operations; the absence of expertly-designed systems is one of the significant reasons why there has been a weak link in the distribution of relief aid. The distribution of relief aid is a complex problem as the goal is humanitarian yet at the same time, due to limited resources, the operations have to be efficient. In the two essays on humanitarian logistics, we study the distribution of aid using homogeneous fleet, with and without capacity restrictions. In Essay 3, we discuss routing for relief operations using one vehicle without capacity restrictions. Contrary to the existing vehicle routing models, the key property of our routing models is that the nodes have priorities along with humanitarian needs. We formulate this model with d-Relaxed Priority rule that captures distance and response time. We formulate routing models with strict and relaxed forms of priority restrictions as Mixed Integer Programs (MIP). We derive bounds for this problem and show that this bound is attained in limiting condition for a worst-case example. Finally, we evaluate the optimal solutions on test problems for response time and distance and show that our vehicle routing model with priorities captures the trade-off between distance and response time unlike existing Vehicle Routing Problem (VRP) models without priorities. In Essay 4, we extend the problem dealt in third essay to consider fleet consisting of multiple vehicles (homogeneous) with capacity and route length restrictions. First, we show that the humanitarian aspect imposes additional challenges and develop routing models that capture performance metrics like fill rate, distance traversed, response time and number of victims satisfied. Proposed routing models are formulated as Mixed Integer Programs and are solved to optimality for small test problems. We conduct computational experiment and show that our models perform well on these performance metrics

Efficient Environment Sensing and Learning for Mobile Robots

Data-driven learning is becoming an integral part of many robotic systems. Robots can be used as mobile sensors to learn about the environment in which they operate. Robots can also seek to learn essential skills, such as navigation, within the environment. A critical challenge in both types of learning is sample efficiency. Acquiring samples with physical robots can be prohibitively time-consuming. As a result, when applying learning techniques in robotics that require physical interaction with the environment, minimizing the number of such interactions becomes a key. The key question we seek to answer is: How do we make robots learn efficiently with a minimal amount of physical interaction? We approach this question along two fronts: extrinsic learning and intrinsic learning. In extrinsic learning, we want the robot to learn about the external environment in which it is operating. In intrinsic learning, our focus is on the robot to learn a skill using reinforcement learning (RL) such as navigating in an environment. In this dissertation, we develop algorithms that carefully plan where the robots obtain samples in order to efficiently perform intrinsic and extrinsic learning. In particular, we exploit the structural properties of Gaussian Process (GP) regression to design efficient sampling algorithms. We study two types of problems under extrinsic learning. We start with the problem of learning a spatially varying field modeled by a GP efficiently. Our goal is to ensure that the GP posterior variance, which is also the mean square error between the learned and actual fields, is below a predefined value. By exploiting the underlying properties of GP, we present a series of constant-factor approximation algorithms for minimizing the number of stationary sensors to place, minimizing the total time taken by a single robot, and minimizing the time taken by a team of robots to learn the field. Here, we assume that the GP hyperparameters are known. We then study a variant where our goal is to identify the hotspot in an environment. Here we do not assume that hyperparameters are unknown. For this problem, we present Upper Confidence Bound (UCB) and Monte Carlo Tree Search (MCTS) based algorithms for a single robot and later extend them to decentralized multi-robot teams. We also validate their performance on real-world datasets. For intrinsic learning, our aim is to reduce the number of physical interactions by leveraging simulations often known as Multi-Fidelity Reinforcement Learning (MFRL). In the MFRL framework, an agent uses multiple simulators of the real environment to perform actions. We present two MFRL framework versions, model-based and model-free, that leverage GPs to learn the optimal policy in a real-world environment. By incorporating GPs in the MFRL framework, we empirically observe a significant reduction in the number of samples for model-based and model-free learning

Information-Theoretic Active Perception for Multi-Robot Teams

Multi-robot teams that intelligently gather information have the potential to transform industries as diverse as agriculture, space exploration, mining, environmental monitoring, search and rescue, and construction. Despite large amounts of research effort on active perception problems, there still remain significant challenges. In this thesis, we present a variety of information-theoretic control policies that enable teams of robots to efficiently estimate different quantities of interest. Although these policies are intractable in general, we develop a series of approximations that make them suitable for real time use. We begin by presenting a unified estimation and control scheme based on Shannon\u27s mutual information that lets small teams of robots equipped with range-only sensors track a single static target. By creating approximate representations, we substantially reduce the complexity of this approach, letting the team track a mobile target. We then scale this approach to larger teams that need to localize a large and unknown number of targets. We also examine information-theoretic control policies to autonomously construct 3D maps with ground and aerial robots. By using Cauchy-Schwarz quadratic mutual information, we show substantial computational improvements over similar information-theoretic measures. To map environments faster, we adopt a hierarchical planning approach which incorporates trajectory optimization so that robots can quickly determine feasible and locally optimal trajectories. Finally, we present a high-level planning algorithm that enables heterogeneous robots to cooperatively construct maps