Learning scalable and transferable multi-robot/machine sequential assignment planning via graph embedding

Can the success of reinforcement learning methods for simple combinatorial optimization problems be extended to multi-robot sequential assignment planning? In addition to the challenge of achieving near-optimal performance in large problems, transferability to an unseen number of robots and tasks is another key challenge for real-world applications. In this paper, we suggest a method that achieves the first success in both challenges for robot/machine scheduling problems. Our method comprises of three components. First, we show a robot scheduling problem can be expressed as a random probabilistic graphical model (PGM). We develop a mean-field inference method for random PGM and use it for Q-function inference. Second, we show that transferability can be achieved by carefully designing two-step sequential encoding of problem state. Third, we resolve the computational scalability issue of fitted Q-iteration by suggesting a heuristic auction-based Q-iteration fitting method enabled by transferability we achieved. We apply our method to discrete-time, discrete space problems (Multi-Robot Reward Collection (MRRC)) and scalably achieve 97% optimality with transferability. This optimality is maintained under stochastic contexts. By extending our method to continuous time, continuous space formulation, we claim to be the first learning-based method with scalable performance among multi-machine scheduling problems; our method scalability achieves comparable performance to popular metaheuristics in Identical parallel machine scheduling (IPMS) problems

Stochastic Online Learning with Probabilistic Graph Feedback

We consider a problem of stochastic online learning with general probabilistic graph feedback, where each directed edge in the feedback graph has probability $p_{ij}$. Two cases are covered. (a) The one-step case, where after playing arm $i$ the learner observes a sample reward feedback of arm $j$ with independent probability $p_{ij}$. (b) The cascade case where after playing arm $i$ the learner observes feedback of all arms $j$ in a probabilistic cascade starting from $i$ -- for each $(i,j)$ with probability $p_{ij}$, if arm $i$ is played or observed, then a reward sample of arm $j$ would be observed with independent probability $p_{ij}$. Previous works mainly focus on deterministic graphs which corresponds to one-step case with $p_{ij} \in \{0,1\}$, an adversarial sequence of graphs with certain topology guarantees, or a specific type of random graphs. We analyze the asymptotic lower bounds and design algorithms in both cases. The regret upper bounds of the algorithms match the lower bounds with high probability

A Random Walk Perspective on Hide-and-Seek Games

We investigate hide-and-seek games on complex networks using a random walk framework. Specifically, we investigate the efficiency of various degree-biased random walk search strategies to locate items that are randomly hidden on a subset of vertices of a random graph. Vertices at which items are hidden in the network are chosen at random as well, though with probabilities that may depend on degree. We pitch various hide and seek strategies against each other, and determine the efficiency of search strategies by computing the average number of hidden items that a searcher will uncover in a random walk of $n$ steps. Our analysis is based on the cavity method for finite single instances of the problem, and generalises previous work of De Bacco et al. [1] so as to cover degree-biased random walks. We also extend the analysis to deal with the thermodynamic limit of infinite system size. We study a broad spectrum of functional forms for the degree bias of both the hiding and the search strategy and investigate the efficiency of families of search strategies for cases where their functional form is either matched or unmatched to that of the hiding strategy. Our results are in excellent agreement with those of numerical simulations. We propose two simple approximations for predicting efficient search strategies. One is based on an equilibrium analysis of the random walk search strategy. While not exact, it produces correct orders of magnitude for parameters characterising optimal search strategies. The second exploits the existence of an effective drift in random walks on networks, and is expected to be efficient in systems with low concentration of small degree nodes.Comment: 31 pages, 10 (multi-part) figure

Modeling epidemics on a regular tree graph

We will first provide a brief introduction to models of disease transmission on so-called contact networks, which can be represented by various structures from the mathematical field of graph theory. These models allow for exploration of stochastic effects and incorporation of more biological detail than the classical compartment-based ordinary differential equation models, which usually assume both homogeneity in the population and uniform mixing. In particular, we use an agent-based modelling platform to compare theoretical predictions from mathematical epidemiology to results obtained from simulations of disease transmission on a regular tree graph. We also demonstrate how this graph reveals connections between network structure and the spread of infectious diseases. Specifically, we discuss results for how certain properties of the tree graph, such as network diameter and density, alter the duration of an outbreak