Learning scalable and transferable multi-robot/machine sequential assignment planning via graph embedding

Can the success of reinforcement learning methods for simple combinatorial optimization problems be extended to multi-robot sequential assignment planning? In addition to the challenge of achieving near-optimal performance in large problems, transferability to an unseen number of robots and tasks is another key challenge for real-world applications. In this paper, we suggest a method that achieves the first success in both challenges for robot/machine scheduling problems. Our method comprises of three components. First, we show a robot scheduling problem can be expressed as a random probabilistic graphical model (PGM). We develop a mean-field inference method for random PGM and use it for Q-function inference. Second, we show that transferability can be achieved by carefully designing two-step sequential encoding of problem state. Third, we resolve the computational scalability issue of fitted Q-iteration by suggesting a heuristic auction-based Q-iteration fitting method enabled by transferability we achieved. We apply our method to discrete-time, discrete space problems (Multi-Robot Reward Collection (MRRC)) and scalably achieve 97% optimality with transferability. This optimality is maintained under stochastic contexts. By extending our method to continuous time, continuous space formulation, we claim to be the first learning-based method with scalable performance among multi-machine scheduling problems; our method scalability achieves comparable performance to popular metaheuristics in Identical parallel machine scheduling (IPMS) problems

Decentralized dynamic task allocation for UAVs with limited communication range

We present the Limited-range Online Routing Problem (LORP), which involves a team of Unmanned Aerial Vehicles (UAVs) with limited communication range that must autonomously coordinate to service task requests. We first show a general approach to cast this dynamic problem as a sequence of decentralized task allocation problems. Then we present two solutions both based on modeling the allocation task as a Markov Random Field to subsequently assess decisions by means of the decentralized Max-Sum algorithm. Our first solution assumes independence between requests, whereas our second solution also considers the UAVs' workloads. A thorough empirical evaluation shows that our workload-based solution consistently outperforms current state-of-the-art methods in a wide range of scenarios, lowering the average service time up to 16%. In the best-case scenario there is no gap between our decentralized solution and centralized techniques. In the worst-case scenario we manage to reduce by 25% the gap between current decentralized and centralized techniques. Thus, our solution becomes the method of choice for our problem

Collision-aware Task Assignment for Multi-Robot Systems

We propose a novel formulation of the collision-aware task assignment (CATA) problem and a decentralized auction-based algorithm to solve the problem with optimality bound. Using a collision cone, we predict potential collisions and introduce a binary decision variable into the local reward function for task bidding. We further improve CATA by implementing a receding collision horizon to address the stopping robot scenario, i.e. when robots are confined to their task location and become static obstacles to other moving robots. The auction-based algorithm encourages the robots to bid for tasks with collision mitigation considerations. We validate the improved task assignment solution with both simulation and experimental results, which show significant reduction of overlapping paths as well as deadlocks

Distributed and Centralized Task Allocation: When and Where to Use Them

Self-organisation is frequently advocated as the solution for managing large, dynamic systems. Distributed algorithms are implicitly designed for infinitely large problems, while small systems are regarded as being controllable using traditional, centralised approaches. Many real-world systems, however, do not fit conveniently into these "small" or "large" categories, resulting in a range of cases where the optimal solution is ambiguous. This difficulty is exacerbated by enthusiasts of either approach constructing problems that suit their preferred control architecture. We address this ambiguity by building an abstract model of task allocation in a community of specialised agents. We are inspired by the problem of work distribution in distributed satellite systems, but the model is also relevant to the resource allocation problems in distributed robotics, autonomic computing and wireless sensor networks. We compare the behaviour of a self-organising, market-based task allocation strategy to a classical approach that uses a central controller with global knowledge. The objective is not to prove one mechanism inherently superior to the other; instead we are interested in the regions of problem space where each of them dominates. Simulation is used to explore the trade-off between energy consumption and robustness in a system of intermediate size, with fixed communication costs and varying rates of component failure. We identify boundaries between regions in the parameter space where one or the other architecture will be favoured. This allows us to derive guidelines for system designers, thus contributing to the development of a disciplined approach to controlling distributed systems using self-organising mechanisms

What’s in it for me? Incentive-compatible route coordination of crowdsourced resources

With the recent trend in crowdsourcing, i.e., using the power of crowds to assist in satisfying demand, the pool of resources suitable for GeoPresence-capable systems has expanded to include already roaming devices, such as mobile phones, and moving vehicles. We envision an environment, in which the motion of these crowdsourced mobile resources is coordinated, according to their preexisting schedules to satisfy geo-temporal demand on a mobility field. In this paper, we propose an incentive compatible route coordination mechanism for crowdsourced resources, in which participating mobile agents satisfy geo-temporal requests in return for monetary rewards. We define the Flexible Route Coordination (FRC) problem, in which an agent’s flexibility is exploited to maximize the coverage of a mobility field, with an objective to maximize the revenue collected from satisfied paying requests. Given that the FRC problem is NP-hard, we define an optimal algorithm to plan the route of a single agent on a graph with evolving labels, then we use that algorithm to define a 1/2-approximation algorithm to solve the problem in its general model, with multiple agents. Moreover, we define an incentive compatible, rational, and cash-positive payment mechanism, which guarantees that an agent’s truthfulness about its flexibility is an ex-post Nash equilibrium strategy. Finally, we analyze the proposed mechanisms theoretically, and evaluate their performance experimentally using real mobility traces from urban environments.Supported in part by NSF Grants, #1430145, #1414119, #1347522, #1239021, and #1012798

Incentive-compatible route coordination of crowdsourced resources

Technical ReportWith the recent trend in crowdsourcing, i.e., using the power of crowds to assist in satisfying demand, the pool of resources suitable for GeoPresen-ce-capable systems has expanded to include already roaming devices, such as mobile phones, and moving vehicles. We envision an environment, in which the motion of these crowdsourced mobile resources is coordinated, according to their preexisting schedules to satisfy geo-temporal demand on a mobility field. In this paper, we propose an incentive compatible route coordination mechanism for crowdsourced resources, in which participating mobile agents satisfy geo-temporal requests in return for monetary rewards. We define the Flexible Route Coordination (FRC) problem, in which an agent’s flexibility is exploited to maximize the coverage of a mobility field, with an objective to maximize the revenue collected from satisfied paying requests. Given that the FRC problem is NP-hard, we define an optimal algorithm to plan the route of a single agent on a graph with evolving labels, then we use that algorithm to define a 1-approximation algorithm to solve the 2 problem in its general model, with multiple agents. Moreover, we define an incentive compatible, rational, and cash-positive payment mechanism, which guarantees that an agent’s truthfulness about its flexibility is an ex-post Nash equilibrium strategy. Finally, we analyze the proposed mechanisms theoretically, and evaluate their performance experimentally using real mobility traces from urban environments

Incentive compatible route coordination of crowdsourced resources and its application to GeoPresence-as-a-Service

With the recent trend in crowdsourcing, i.e., using the power of crowds to assist in satisfying demand, the pool of resources suitable for GeoPresen- ce-capable systems has expanded to include already roaming devices, such as mobile phones, and moving vehicles. We envision an environment, in which the motion of these crowdsourced mobile resources is coordinated, according to their preexisting schedules to satisfy geo-temporal demand on a mobility field. In this paper, we propose an incentive compatible route coordination mechanism for crowdsourced resources, in which participating mobile agents satisfy geo-temporal requests in return for monetary rewards. We define the Flexible Route Coordination (FRC) problem, in which an agent's exibility is exploited to maximize the coverage of a mo- bility field, with an objective to maximize the revenue collected from sat- isfied paying requests. Given that the FRC problem is NP-hard, we define an optimal algorithm to plan the route of a single agent on a graph with evolving labels, then we use that algorithm to define a 1 2 -approximation algorithm to solve the problem in its general model, with multiple agents. Moreover, we define an incentive compatible, rational, and cash-positive payment mechanism, which guarantees that an agent's truthfulness about its exibility is an ex-post Nash equilibrium strategy. Finally, we analyze the proposed mechanisms theoretically, and evaluate their performance experimentally using real mobility traces from urban environments.Supported in part by NSF Grants, #1430145, #1414119, #1347522, #1239021, and #1012798

An auction-based approach with closed-loop bid adjustment to dynamic task allocation in robot teams

Dynamic task allocation is among the most difficult issues in multi-robot coordination, although it is imperative for a multitude of applications. Auction-based approaches are popular methods that allocate tasks to robots by assembling team information at a single location to make practicable decisions. However, a main deficiency of auction-based methods is that robots generally do not have sufficient information to estimate reliable bids to perform tasks, particularly in dynamic environments. While some techniques have been developed to improve bidding, they are mostly open-looped without feed-back adjustments to tune the bid prices for subsequent tasks of the same type. Robots' bids, if not assessed and adjusted accordingly, may not be trustworthy and would indeed impede team performance. To address this issue, we propose a closed-loop bid adjustment mechanism for auction-based multi-robot task allocation, with an aim to evaluate and improve robots' bids, and hence enhance the overall team performance. Each robot in a team maintains and uses its own track record as closed-loop feedback information to adjust and improve its bid prices. After a robot has completed a task, it assesses and records its performance to reflect the discrepancy between the bid price and the actual cost of the task. Such performance records, with time-discounting factors, are taken into account to damp out fluctuations of bid prices. Adopting this adjustment mechanism, a task would be more likely allocated to a competent robot that submits a more accurate bid price, and hence improve the overall team performance. Simulation of task allocation of free-range automated guided vehicles serving at a container terminal is presented to demonstrate the effectiveness of the adjustment mechanism.postprintThe World Congress on Engineering (WCE 2011), London, U.K., 6-8 July 2011. In Proceedings of WCE, 2011, v. 2, p. 1061-106