Lifelong Path Planning with Kinematic Constraints for Multi-Agent Pickup and Delivery

The Multi-Agent Pickup and Delivery (MAPD) problem models applications where a large number of agents attend to a stream of incoming pickup-and-delivery tasks. Token Passing (TP) is a recent MAPD algorithm that is efficient and effective. We make TP even more efficient and effective by using a novel combinatorial search algorithm, called Safe Interval Path Planning with Reservation Table (SIPPwRT), for single-agent path planning. SIPPwRT uses an advanced data structure that allows for fast updates and lookups of the current paths of all agents in an online setting. The resulting MAPD algorithm TP-SIPPwRT takes kinematic constraints of real robots into account directly during planning, computes continuous agent movements with given velocities that work on non-holonomic robots rather than discrete agent movements with uniform velocity, and is complete for well-formed MAPD instances. We demonstrate its benefits for automated warehouses using both an agent simulator and a standard robot simulator. For example, we demonstrate that it can compute paths for hundreds of agents and thousands of tasks in seconds and is more efficient and effective than existing MAPD algorithms that use a post-processing step to adapt their paths to continuous agent movements with given velocities.Comment: AAAI 201

Development of transportation and supply chain problems with the combination of agent-based simulation and network optimization

Demand drives a different range of supply chain and logistics location decisions, and agent-based modeling (ABM) introduces innovative solutions to address supply chain and logistics problems. This dissertation focuses on an agent-based and network optimization approach to resolve those problems and features three research projects that cover prevalent supply chain management and logistics problems. The first case study evaluates demographic densities in Norway, Finland, and Sweden, and covers how distribution center (DC) locations can be established using a minimizing trip distance approach. Furthermore, traveling time maps are developed for each scenario. In addition, the Nordic area consisting of those three countries is analyzed and five DC location optimization results are presented. The second case study introduces transportation cost modelling in the process of collecting tree logs from several districts and transporting them to the nearest collection point. This research project presents agent-based modelling (ABM) that incorporates comprehensively the key elements of the pick-up and delivery supply chain model and designs the components as autonomous agents communicating with each other. The modelling merges various components such as GIS routing, potential facility locations, random tree log pickup locations, fleet sizing, trip distance, and truck and train transportation. The entire pick-up and delivery operation are modeled by ABM and modeling outcomes are provided by time series charts such as the number of trucks in use, facilities inventory and travel distance. In addition, various scenarios of simulation based on potential facility locations and truck numbers are evaluated and the optimal facility location and fleet size are identified. In the third case study, an agent-based modeling strategy is used to address the problem of vehicle scheduling and fleet optimization. The solution method is employed to data from a real-world organization, and a set of key performance indicators are created to assess the resolution's effectiveness. The ABM method, contrary to other modeling approaches, is a fully customized method that can incorporate extensively various processes and elements. ABM applying the autonomous agent concept can integrate various components that exist in the complex supply chain and create a similar system to assess the supply chain efficiency.Tuotteiden kysyntä ohjaa erilaisia toimitusketju- ja logistiikkasijaintipäätöksiä, ja agenttipohjainen mallinnusmenetelmä (ABM) tuo innovatiivisia ratkaisuja toimitusketjun ja logistiikan ongelmien ratkaisemiseen. Tämä väitöskirja keskittyy agenttipohjaiseen mallinnusmenetelmään ja verkon optimointiin tällaisten ongelmien ratkaisemiseksi, ja sisältää kolme tapaustutkimusta, jotka voidaan luokitella kuuluvan yleisiin toimitusketjun hallinta- ja logistiikkaongelmiin. Ensimmäinen tapaustutkimus esittelee kuinka käyttämällä väestötiheyksiä Norjassa, Suomessa ja Ruotsissa voidaan määrittää strategioita jakelukeskusten (DC) sijaintiin käyttämällä matkan etäisyyden minimoimista. Kullekin skenaariolle kehitetään matka-aikakartat. Lisäksi analysoidaan näistä kolmesta maasta koostuvaa pohjoismaista aluetta ja esitetään viisi mahdollista sijaintia optimointituloksena. Toinen tapaustutkimus esittelee kuljetuskustannusmallintamisen prosessissa, jossa puutavaraa kerätään useilta alueilta ja kuljetetaan lähimpään keräyspisteeseen. Tämä tutkimusprojekti esittelee agenttipohjaista mallinnusta (ABM), joka yhdistää kattavasti noudon ja toimituksen toimitusketjumallin keskeiset elementit ja suunnittelee komponentit keskenään kommunikoiviksi autonomisiksi agenteiksi. Mallinnuksessa yhdistetään erilaisia komponentteja, kuten GIS-reititys, mahdolliset tilojen sijainnit, satunnaiset puunhakupaikat, kaluston mitoitus, matkan pituus sekä monimuotokuljetukset. ABM:n avulla mallinnetaan noutojen ja toimituksien koko ketju ja tuloksena saadaan aikasarjoja kuvaamaan käytössä olevat kuorma-autot, sekä varastomäärät ja ajetut matkat. Lisäksi arvioidaan erilaisia simuloinnin skenaarioita mahdollisten laitosten sijainnista ja kuorma-autojen lukumäärästä sekä tunnistetaan optimaalinen toimipisteen sijainti ja tarvittava autojen määrä. Kolmannessa tapaustutkimuksessa agenttipohjaista mallinnusstrategiaa käytetään ratkaisemaan ajoneuvojen aikataulujen ja kaluston optimoinnin ongelma. Ratkaisumenetelmää käytetään dataan, joka on peräisin todellisesta organisaatiosta, ja ratkaisun tehokkuuden arvioimiseksi luodaan lukuisia keskeisiä suorituskykyindikaattoreita. ABM-menetelmä, toisin kuin monet muut mallintamismenetelmät, on täysin räätälöitävissä oleva menetelmä, joka voi sisältää laajasti erilaisia prosesseja ja elementtejä. Autonomisia agentteja soveltava ABM voi integroida erilaisia komponentteja, jotka ovat olemassa monimutkaisessa toimitusketjussa ja luoda vastaavan järjestelmän toimitusketjun tehokkuuden arvioimiseksi yksityiskohtaisesti.fi=vertaisarvioitu|en=peerReviewed

Simulation and optimization of a multi-agent system on physical internet enabled interconnected urban logistics.

An urban logistics system is composed of multiple agents, e.g., shippers, carriers, and distribution centers, etc., and multi-modal networks. The structure of Physical Internet (PI) transportation network is different from current logistics practices, and simulation can effectively model a series of PI-approach scenarios. In addition to the baseline model, three more scenarios are enacted based on different characteristics: shared trucks, shared hubs, and shared flows with other less-than-truckload shipments passing through the urban area. Five performance measures, i.e., truck distance per container, mean truck time per container, lead time, CO2 emissions, and transport mean fill rate, are included in the proposed procedures using real data in an urban logistics case. The results show that PI enables a significant improvement of urban transportation efficiency and sustainability. Specifically, truck time per container reduces 26 percent from that of the Private Direct scenario. A 42 percent reduction of CO2 emissions is made from the current logistics practice. The fill rate of truckload is increased by almost 33 percent, whereas the relevant longer distance per container and the lead time has been increased by an acceptable range. Next, the dissertation applies an auction mechanism in the PI network. Within the auction-based transportation planning approach, a model is developed to match the requests and the transport services in transport marketplaces and maximize the carriers’ revenue. In such transportation planning under the protocol of PI, it is a critical system design problem for decision makers to understand how various parameters through interactions affect this multi-agent system. This study provides a comprehensive three-layer structure model, i.e. agent-based simulation, auction mechanism, and optimization via simulation. In term of simulation, a multi-agent model simulates a complex PI transportation network in the context of sharing economy. Then, an auction mechanism structure is developed to demonstrate a transport selection scheme. With regard of an optimization via simulation approach and sensitivity analysis, it has been provided with insights on effects of combination of decision variables (i.e. truck number and truck capacity) and parameters settings, where results can be drawn by using a case study in an urban freight transportation network. In the end, conclusions and discussions of the studies have been summarized. Additionally, some relevant areas are required for further elaborate research, e.g., operational research on airport gate assignment problems and the simulation modelling of air cargo transportation networks. Due to the complexity of integration with models, I relegate those for future independent research

Graph Attention Multi-Agent Fleet Autonomy for Advanced Air Mobility

Autonomous mobility is emerging as a new mode of urban transportation for moving cargo and passengers. However, such fleet coordination schemes face significant challenges in scaling to accommodate fast-growing fleet sizes that vary in their operational range, capacity, and communication capabilities. We introduce the concept of partially observable advanced air mobility games to coordinate a fleet of aerial vehicle agents accounting for their heterogeneity and self-interest inherent to commercial mobility fleets. We propose a novel heterogeneous graph attention-based encoder-decoder (HetGAT Enc-Dec) neural network to construct a generalizable stochastic policy stemming from the inter- and intra-agent relations within the mobility system. We train our policy by leveraging deep multi-agent reinforcement learning, allowing decentralized decision-making for the agents using their local observations. Through extensive experimentation, we show that the fleets operating under the HetGAT Enc-Dec policy outperform other state-of-the-art graph neural network-based policies by achieving the highest fleet reward and fulfillment ratios in an on-demand mobility network.Comment: 12 pages, 12 figures, 3 table

Fostering collaboration and coordination in urban delivery: a multi-agent microsimulation model

Given the dynamic nature of Urban Freight Transport (UFT) processes, the involved transport and logistics operators face with internal and external issues that should tackle to improve last-mile levels of service and decrease total costs while performing delivery operations. Customers (i.e., freight receivers) perceive the level of service through the acceptance of their requests, while total operational costs are mainly determined by the total travel costs (i.e., distance and/or time) required to accomplish the customers' request. In addition, the vehicle-kilometres travelled are related to the externalities produced. Given that the actors involved in the process operate in a stochastic environment (with changes that can occur both in terms of demand – receivers' requests, and in supply – travel times), collaboration and coordination among the operators could play a key role in meeting the customers' requests as well as in reducing both internal and external delivery costs. Therefore, the paper proposes an UFT modelling framework that integrates collaboration and coordination processes among the different involved actors, and allows the benefits to be assessed. The model has a multi-agent architecture based on microsimulation. In particular, the multi-agent architecture allows us to point out the different actors’ responses to various internal (e.g., delivery requests) and external (e.g., delivery times) changes occurring in the daily delivery operations. It consists of three layers. The first one simulates the interactions among actors operating collaboratively. The second layer microsimulates the collaborative processes of information management. Finally, a third layer integrates the two previous layers, facilitating a decision-making process in such a dynamic context. The whole modelling framework is tested in a real case study in which it is possible to validate pros and cons of working in a collaborative and coordinative environment. The results show significant benefits from actors/operators involved in the process and subsequently can address the policy/measure implementation towards a more sustainable and liveable city

Fostering resilient execution of multi-agent plans through self-organisation

Traditional multi-agent planning addresses the coordination of multiple agents towards common goals, by producing an integrated plan of actions for each of those agents. For systems made of large numbers of cooperating agents, however, the execution and monitoring of a plan should enhance its high-level steps, possibly involving entire sub-teams, with a flexible and adaptable lower-level behaviour of the individual agents. In order to achieve such a goal, we need to integrate the behaviour dictated by a multi-agent plan with self-organizing, swarm-based approaches, capable of automatically adapting their behaviour based on the contingent situation, departing from the predetermined plan whenever needed. Moreover, in order to deal with multiple domains and unpredictable situations, the system should, as far as possible, exhibit such capabilities without hard-coding the agents behaviour and interactions. In this paper, we investigate the relationship between multi-agent planning and self-organisation through the combination of two representative approaches both enjoying declarativity. We consider a functional approach to self-organising systems development, called Aggregate Programming (AP), and propose to exploit collective adaptive behaviour to carry out plan revisions. We describe preliminary results in this direction on a case study of execution monitoring and repair of a Multi-Agent PDDL plan