Lifelong Multi-Agent Path Finding in Large-Scale Warehouses

Multi-Agent Path Finding (MAPF) is the problem of moving a team of agents to their goal locations without collisions. In this paper, we study the lifelong variant of MAPF, where agents are constantly engaged with new goal locations, such as in large-scale automated warehouses. We propose a new framework Rolling-Horizon Collision Resolution (RHCR) for solving lifelong MAPF by decomposing the problem into a sequence of Windowed MAPF instances, where a Windowed MAPF solver resolves collisions among the paths of the agents only within a bounded time horizon and ignores collisions beyond it. RHCR is particularly well suited to generating pliable plans that adapt to continually arriving new goal locations. We empirically evaluate RHCR with a variety of MAPF solvers and show that it can produce high-quality solutions for up to 1,000 agents (= 38.9\% of the empty cells on the map) for simulated warehouse instances, significantly outperforming existing work.Comment: Published at AAAI 202

Optimal and Bounded-Suboptimal Multi-Goal Task Assignment and Path Finding

We formalize and study the multi-goal task assignment and path finding (MG-TAPF) problem from theoretical and algorithmic perspectives. The MG-TAPF problem is to compute an assignment of tasks to agents, where each task consists of a sequence of goal locations, and collision-free paths for the agents that visit all goal locations of their assigned tasks in sequence. Theoretically, we prove that the MG-TAPF problem is NP-hard to solve optimally. We present algorithms that build upon algorithmic techniques for the multi-agent path finding problem and solve the MG-TAPF problem optimally and bounded-suboptimally. We experimentally compare these algorithms on a variety of different benchmark domains.Comment: ICRA 202

Multi-Goal Multi-Agent Pickup and Delivery

In this work, we consider the Multi-Agent Pickup-and-Delivery (MAPD) problem, where agents constantly engage with new tasks and need to plan collision-free paths to execute them. To execute a task, an agent needs to visit a pair of goal locations, consisting of a pickup location and a delivery location. We propose two variants of an algorithm that assigns a sequence of tasks to each agent using the anytime algorithm Large Neighborhood Search (LNS) and plans paths using the Multi-Agent Path Finding (MAPF) algorithm Priority-Based Search (PBS). LNS-PBS is complete for well-formed MAPD instances, a realistic subclass of MAPD instances, and empirically more effective than the existing complete MAPD algorithm CENTRAL. LNS-wPBS provides no completeness guarantee but is empirically more efficient and stable than LNS-PBS. It scales to thousands of agents and thousands of tasks in a large warehouse and is empirically more effective than the existing scalable MAPD algorithm HBH+MLA*. LNS-PBS and LNS-wPBS also apply to a more general variant of MAPD, namely the Multi-Goal MAPD (MG-MAPD) problem, where tasks can have different numbers of goal locations.Comment: IROS 202

The Study of Highway for Lifelong Multi-Agent Path Finding

In modern fulfillment warehouses, agents traverse the map to complete endless tasks that arrive on the fly, which is formulated as a lifelong Multi-Agent Path Finding (lifelong MAPF) problem. The goal of tackling this challenging problem is to find the path for each agent in a finite runtime while maximizing the throughput. However, existing methods encounter exponential growth of runtime and undesirable phenomena of deadlocks and rerouting as the map size or agent density grows. To address these challenges in lifelong MAPF, we explore the idea of highways mainly studied for one-shot MAPF (i.e., finding paths at once beforehand), which reduces the complexity of the problem by encouraging agents to move in the same direction. We utilize two methods to incorporate the highway idea into the lifelong MAPF framework and discuss the properties that minimize the existing problems of deadlocks and rerouting. The experimental results demonstrate that the runtime is considerably reduced and the decay of throughput is gradually insignificant as the map size enlarges under the settings of the highway. Furthermore, when the density of agents increases, the phenomena of deadlocks and rerouting are significantly reduced by leveraging the highway

Optimizations of a Multi-Agent System for a Real-World Warehouse Problem

In recent years, many warehouses applied mobile robots to move products from one location to another. We focus on a traditional warehouse where agents are humans, and they are engaged with tasks to navigate to the next destination one after the other. The possible destinations are determined at the beginning of the daily shift. Our real-world warehouse client asked us to minimize the total wage cost, and to minimize the irritation of the workers because of conflicts in their tasks. We define a heuristic for the optimizations for splitting the orders into warehouse carts, defining the sequence of the products within the carts, and the assignment of the carts to workers. We extend Multi-Agent Path Finding (MAPF) solution techniques. Furthermore, we have implemented our proposal in a simulation software, and we have run several experiments. According to the experiments, the make-span and the wage cost cannot be reduced with the heuristic optimization, however the heuristic optimization considerably reduces the irritation of the workers. We conclude our work with a guideline for the warehouse

Multi-Robot Coordination and Layout Design for Automated Warehousing

With the rapid progress in Multi-Agent Path Finding (MAPF), researchers have studied how MAPF algorithms can be deployed to coordinate hundreds of robots in large automated warehouses. While most works try to improve the throughput of such warehouses by developing better MAPF algorithms, we focus on improving the throughput by optimizing the warehouse layout. We show that, even with state-of-the-art MAPF algorithms, commonly used human-designed layouts can lead to congestion for warehouses with large numbers of robots and thus have limited scalability. We extend existing automatic scenario generation methods to optimize warehouse layouts. Results show that our optimized warehouse layouts (1) reduce traffic congestion and thus improve throughput, (2) improve the scalability of the automated warehouses by doubling the number of robots in some cases, and (3) are capable of generating layouts with user-specified diversity measures. We include the source code at: https://github.com/lunjohnzhang/warehouse_env_gen_publicComment: Accepted to International Joint Conference on Artificial Intelligence (IJCAI), 2023. The paper can be found at IJCAI 2023 proceeding at https://www.ijcai.org/proceedings/2023/061

Multi-UAV simultaneous target assignment and path planning based on deep reinforcement learning in dynamic multiple obstacles environments

Target assignment and path planning are crucial for the cooperativity of multiple unmanned aerial vehicles (UAV) systems. However, it is a challenge considering the dynamics of environments and the partial observability of UAVs. In this article, the problem of multi-UAV target assignment and path planning is formulated as a partially observable Markov decision process (POMDP), and a novel deep reinforcement learning (DRL)-based algorithm is proposed to address it. Specifically, a target assignment network is introduced into the twin-delayed deep deterministic policy gradient (TD3) algorithm to solve the target assignment problem and path planning problem simultaneously. The target assignment network executes target assignment for each step of UAVs, while the TD3 guides UAVs to plan paths for this step based on the assignment result and provides training labels for the optimization of the target assignment network. Experimental results demonstrate that the proposed approach can ensure an optimal complete target allocation and achieve a collision-free path for each UAV in three-dimensional (3D) dynamic multiple-obstacle environments, and present a superior performance in target completion and a better adaptability to complex environments compared with existing methods

Búsqueda bidireccional aplicada al 'Another Solution Problem'

[ES] El Shortest Path Problem (SPP) es uno de los problemas de búsqueda más estudiados en la literatura, razón por la que han cobrado interés otros problemas relacionados como el Another Solution Problem (ASP). El ASP se enuncia como el problema en el que, dada una solución óptima, el objetivo es encontrar la siguiente solución óptima. En este proyecto se propone abordar el problema mediante la aplicación de técnicas de búsqueda bidireccional de modo que el punto en el que se encuentren los dos procesos de búsqueda, comenzando uno por el vértice inicial y otro por el vértice final, determinará el segundo camino óptimo.[CA] El Shortest Path Problem és un dels problemes de cerca de camins més estudiats en la literatura, raó per la qual altres problemes com l’Another Solution Problem (ASP) han guanyat interés. L’ASP s’enuncia com el problema en el qual, donada una solució òptima, l’objetiu és trobar la següent solució òptima. En aquest projecte es proposa resoldre el problema mitjançant l’aplicació de tècniques de cerca bidireccional de manera que el punt en el qual s’encontren els dos processos de cerca, començant un pel vèrtex inicial i l’altre pel vèrtex final, determina el segon camí òptim.[EN] The Shortest Path Problem (SPP) is a well-known and studied problem in the literature. Originated from this problem, other interesting and related problems have emerged like the Another Solution Problem (ASP). Given a problem and its optimal solution the ASP lies in finding the subsequent optimal solution. There exist different techniques to address the ASP but most of them present several limitations due to the exploration of unnecessary paths or an unsucessful search. In this project we propose to address the ASP by using techniques of bidirectional search, which consist in starting two search process, from the initial and final nodes, respectively, such that the node in which both searches find will determine the second best solution.Martín Navarro, JL. (2019). Búsqueda bidireccional aplicada al 'Another Solution Problem'. http://hdl.handle.net/10251/126138TFG

Sistemas multiagentes para coleta e entrega combinando algoritmos genéticos e planejamento de caminho

Distribution centers are increasingly automated with the use of autonomous mobile robots. These robots perform the function of moving products, increasing productivity in warehouses. Several works have been studied in the literature that try to capture these characteristics and the Multi-Agent Pickup and Delivery (MAPD) problem is an example of problem from this area. In this problem, tasks appear in the system at dierent times, and each task has two positions, a pickup and a delivery position. Agents attend to a stream of incoming tasks, moving to pickup position and then to delivery position. Commonly, this problem has two parts: (i) task allocation, in which the agent receives a sequence of tasks to be executed, and (ii) path planning, in which it is necessary to find the best way for the agent to perform its task without colliding with other agents. The problem of finding multi-agent paths is also known as Multi-Agent Path Finding (MAPF). In this work, dierent genetic algorithms were proposed to solve the task allocation part of the MAPD. An analysis of the objectives of the problems was also presented and this problem was treated with a multi-objective approach, using the Non-dominated Sorting Genetic Algorithm (NSGA-II) in order to minimize two objective functions, namely, makespan and service time. For the MAPF sub-problem, path planning algorithms from the literature were used: Prioritized Planning (PP) and Conflict Based Search (CBS). Another approach to Prioritized Planning was also proposed, called PP-E. This approach aims to avoid future collisions between agents, allowing the agent to move to another free position, after reaching its objective position. Computational experiments were carried out in two environments, with dierent sizes, number of agents, number of tasks and rate of entry of tasks in the system. The results were compared with algorithms from the literature and showed that the proposed approach achieves better results when compared to other techniques.Centros de distribuição estão cada dia mais automatizados com a utilização de robôs móveis autônomos. Estes robôs desempenham a função de movimentar produtos, aumentando a produtividade nos armazéns. Vários trabalhos vêm sendo estudados na literatura que buscam capturar essas características e o Multi-Agent Pickup and Delivery (MAPD) é um exemplo de problema desta área. Neste problema, tarefas aparecem no sistema em diferentes instantes de tempo, e cada tarefa tem duas posições, uma posição de coleta e uma de entrega. Os agentes devem atender a esse fluxo de tarefas, se deslocando para a posição de coleta e depois de entrega da tarefa. Comumente, esse problema tem duas partes: (i) alocação de tarefas, em que o agente recebe uma sequência de tarefas a serem executadas, e (ii) planejamento de caminho, no qual é necessário encontrar o melhor caminho para o agente realizar sua tarefa sem colidir com outros agentes. O problema de encontrar caminhos para multiagentes também é conhecido como Multi-Agent Path Finding (MAPF). Neste trabalho, foram propostos diferentes algoritmos genéticos para resolver a parte de alocação de tarefas do MAPD. Também foi apresentado uma análise dos objetivos dos problemas e este problema foi tratado com uma abordagem multiobjetivo, utilizando o Non-dominated Sorting Genetic Algorithm (NSGA-II) a fim de minimizar duas funções objetivo, makespan e service time. Para o sub-problema MAPF, foram utilizados algoritmos de planejamento de caminhos já conhecidos na literatura: o Prioritized Planning (PP) e a Conflict Based Search (CBS). Também foi utilizada outra abordagem para o Prioritized Planning, denominado PP-E. Esta abordagem, PPE, tem como fim evitar futuras colisões entre agentes, possibilitando que o agente se desloque para outra posição livre após chegar na sua posição de objetivo. Experimentos computacionais foram realizados em dois ambientes, com diferentes tamanhos, números de agentes, quantidade de tarefas e taxa de entrada de tarefas no sistema. Os resultados foram comparados com algoritmos da literatura e mostraram que a abordagem proposta alcança melhores resultados quando comparada a outras técnicas.CAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível Superio

Pathfinding Algorithm Optimization Via Evolution

Pathfinding is a popular computer science problem in both academic research and industrial development. The objective of pathfinding is to search for a path, often the shortest path, from one location to another on a graph. Many real world applications can be considered as pathfinding problems, including motion planning, video games, logistics, and decision making. Computer scientists have proposed different algorithms to efficiently search for the shortest path. A* search algorithm is the de facto pathfinding algorithm that uses a heuristic function to determine the best action to take based on the given information. It is the most popular pathfinding algorithm due to its simplicity and efficiency. The performance of A* is heavily dependent on the quality of the heuristic function. The heuristic function determines the search speed, accuracy, and memory consumption. Hence, designing good heuristic functions for specific domains becomes the primary research focus on pathfinding algorithm optimization. In this dissertation, we address and solve several commonly known challenges in pathfinding problems and A* algorithm. First, designing new heuristic functions is a difficult and time-consuming task, especially when they are used to solve complex problems. The task requires the user to have expert knowledge of the problem. Moreover, a single heuristic function might not be enough to digest all the provided information and return the best guidance during the search. Previous works suggest that multiple heuristics for complex problems can dramatically speed up the search. However, choosing the appropriate combination of heuristic functions is tricky. Current optimization approaches rely on hand-tuning the parameters via trial and error by engineers over many iterations. There is a need to reduce the difficulty of designing heuristic functions for search performance maximization. Our first contribution is to propose an improved A* with a self-evolving heuristic function named Evolutionary Heuristic A* (EHA*) that reduces engineering effort to design the heuristic function for A* and maximize the search performance. Our experiment results show that EHA* (i) preserves path optimality; (ii) is not limited to a particular application; (iii) speeds up the path searching process; and (iv) most importantly, dramatically reduces the difficulty for software engineers to design heuristic functions for A* search. Moreover, our work can be applied to other existing works on the performance improvement of A* search. Search, A* search suffers from poor performance on large search spaces. Although EHA* improves the quality of heuristic functions, large search space still leads to many unnecessary searches. Our second contribution is Regions Discovery Algorithm (RDA), a map clustering technique to partition a grid based map into different categories to reduce search spaces and increase search speed. Our approach reduces the size of search spaces by partitioning a graph into many segments and identifying the segments by their characteristics. By identifying segments in different categories, we can easily eliminate search space, such as rooms, that are not possible (better use needed?) to be part of the optimal solution. Unlike the existing approaches that might result in non-optimal solutions, our experiment results show that RDA guarantees optimal solutions. Our third contribution, the Hierarchical Evolutionary Heuristic A* (HEHA*), further improves the search ability of handling complex pathfinding problems and boosting the search performance, by reducing search spaces and exploiting parallelism techniques. HEHA* combines the strength of EHA* and RDA to reduce search spaces and improve search speed. HEHA* shows that it provides better search performance with less memory consumption. In the pre-processing phase, first HEHA* partitions a graph into different segments and then applies different optimized heuristic functions for each segment to maximize the search performance. During the online process, HEHA* searches on the abstract level first to reduce search area, and exploits parallelism to speed up the search. Fourth, we improve and apply HEHA* to Multi-Agent Pathfinding (MAPF) problems. MAPF is the fundamental problem of many robotic and logistic applications, where the main constraint is that all agents can find the shortest paths while not colliding with each other. While the current trend favors the central controlled system, our approach is to develop a distributed version of HEHA* that can efficiently plan the optimal path for each agent. Such a system requires data sharing and exchanging among the agents, so that each agent can make its own decision without a supervising system. Our experiment results show that the Multi-Agent version of HEHA* maintains a high success rate when the number of agents increases. While EHA* and HEHA* provide a novel approach for heuristic function design, the pre-processing times are not trivial. To boost the performance of the preprocessing steps in EHA* and HEHA*, we propose a FPGA-based reconfigurable hardware accelerator that is not bound to any specific applications as our fifth contribution. Since GA requires many independent processes, it is suitable to implement it in a hardware accelerator to gain maximum performance. We apply the following techniques to enhance performance: deep pipelining, reconfigurable computing, massive parallel processing, and degree of parallelism maximization. Our results show that the FPGA accelerator for EHA* improves the scalability, throughput, and latency