Simulationsgestützte Lösung von Deadlocks bei fahrerlosen Transportsystemen mit Hilfe von Deep Reinforcement Learning

This paper discusses the use of deep reinforcement learning to resolve deadlocks in material flow systems with automated guided vehicles (AGVs). The paper proposes a strategy for dealing with deadlocks based on a single Agent reinforcement learning approach (SARL). The agent will find the optimal solution strategy in real time. The proposed approach is evaluated using a material flow simulation for a real use case in industry. The effectiveness in reducing the occurrence of deadlocks as well as the number of collisions in the system is demonstrated. This study highlights the potential of deep reinforcement learning for improving the performance and efficiency of material flow systems with AGVs

Reactive mission and motion planning with deadlock resolution avoiding dynamic obstacles

In the near future mobile robots, such as personal robots or mobile manipulators, will share the workspace with other robots and humans. We present a method for mission and motion planning that applies to small teams of robots performing a task in an environment with moving obstacles, such as humans. Given a mission specification written in linear temporal logic, such as patrolling a set of rooms, we synthesize an automaton from which the robots can extract valid strategies. This centralized automaton is executed by the robots in the team at runtime, and in conjunction with a distributed motion planner that guarantees avoidance of moving obstacles. Our contribution is a correct-by-construction synthesis approach to multi-robot mission planning that guarantees collision avoidance with respect to moving obstacles, guarantees satisfaction of the mission specification and resolves encountered deadlocks, where a moving obstacle blocks the robot temporally. Our method provides conditions under which deadlock will be avoided by identifying environment behaviors that, when encountered at runtime, may prevent the robot team from achieving its goals. In particular, (1) it identifies deadlock conditions; (2) it is able to check whether they can be resolved; and (3) the robots implement the deadlock resolution policy locally in a distributed manner. The approach is capable of synthesizing and executing plans even with a high density of dynamic obstacles. In contrast to many existing approaches to mission and motion planning, it is scalable with the number of moving obstacles. We demonstrate the approach in physical experiments with walking humanoids moving in 2D environments and in simulation with aerial vehicles (quadrotors) navigating in 2D and 3D environments.Boeing CompanyUnited States. Office of Naval Research. Multidisciplinary University Research Initiative. SMARTS (N00014-09-1051)United States. Office of Naval Research (N00014-12-1-1000)National Science Foundation (U.S.). Expeditions in Computer Augmented Program Engineerin

Lifelong Multi-Agent Path Finding in Large-Scale Warehouses

Multi-Agent Path Finding (MAPF) is the problem of moving a team of agents to their goal locations without collisions. In this paper, we study the lifelong variant of MAPF, where agents are constantly engaged with new goal locations, such as in large-scale automated warehouses. We propose a new framework Rolling-Horizon Collision Resolution (RHCR) for solving lifelong MAPF by decomposing the problem into a sequence of Windowed MAPF instances, where a Windowed MAPF solver resolves collisions among the paths of the agents only within a bounded time horizon and ignores collisions beyond it. RHCR is particularly well suited to generating pliable plans that adapt to continually arriving new goal locations. We empirically evaluate RHCR with a variety of MAPF solvers and show that it can produce high-quality solutions for up to 1,000 agents (= 38.9\% of the empty cells on the map) for simulated warehouse instances, significantly outperforming existing work.Comment: Published at AAAI 202

A distribute deadlock detection and resolution algorithm using agents

Deadlock is an intrinsic bottleneck in Distributed Real-Time Database Systems (DRTDBS). Deadlock detection and resolution algorithms are important because in DRTDBS, deadlocked transactions are prone to missing deadlines. We propose an Agent Deadlock Detection and Resolution algorithm (ADCombine), a novel framework for distributed deadlock handling using stationary agents, to address the high overhead suffered by current agent-based algorithms. We test a combined deadlock detection and resolution algorithm that enables the Multi Agent System to adjust its execution based on the changing system load, and that selects its victim transactions more judiciously. We demonstrate the advantages of ADCombine over existing algorithms that use agents or traditional edge-chasing through simulation experiments that measure overhead and performance under a widely varying of experimental conditions.deadlockdistribute real-time database systemsdrtdbsalgorithmmulti agent syste

Designing Solvable Graphs for Multiple Moving Agents

Solvable Graphs (also known as Reachable Graphs) are types of graphs that any arrangement of a specified number of agents located on the graph’s vertices can be reached from any initial arrangement through agents’ moves along the graph’s edges, while avoiding deadlocks (interceptions). In this paper, the properties of Solvable Graphs are investigated, and a new concept in multi agent motion planning, called Minimal Solvable Graphs is introduced. Minimal Solvable Graphs are the smallest graphs among Solvable Graphs in terms of the number of vertices. Also, for the first time, the problem of deciding whether a graph is Solvable for m agents is answered, and a new algorithm is presented for making an existing graph solvable and lean for a given number of agents. Finally, through an industrial example, it is demonstrated that how the findings of this paper can be used in designing and reshaping transportation networks (e.g. railways, traffic roads, AGV routs, robotic workspaces, etc.) for multiple moving agents such as trains, vehicles, and robots

A Robust Scheme for Aggregating Quasi-Blind Robots in an Active Environment

International audienceThe question of how to aggregate autonomous agents with limited abilities in the absence of centralized coordination is known as the Decentralized Gathering Problem. The authors present a bio-inspired aggregation scheme that solves this problem and study a first application of this scheme to a small team of robots. The robots (Alice and Khepera III) obey simple rules and have only a rudimentary perception of their environment. The collective behavior is based on stigmergic principles and uses an active environment to relay the communications between robots. This results in an aggregation process that shows good properties of robustness and that can in principle be extended to swarms of robots

Mechanism design for distributed task and resource allocation among self-interested agents in virtual organizations

The aggregate power of all resources on the Internet is enormous. The Internet can be viewed as a massive virtual organization that holds tremendous amounts of information and resources with different ownerships. However, little is known about how to run this organization efficiently. This dissertation studies the problems of distributed task and resource allocation among self-interested agents in virtual organizations. The developed solutions are not allocation mechanisms that can be imposed by a centralized designer, but decentralized interaction mechanisms that provide incentives to self-interested agents to behave cooperatively. These mechanisms also take computational tractability into consideration due to the inherent complexity of distributed task and resource allocation problems. Targeted allocation mechanisms can achieve global task allocation efficiency in a virtual organization and establish stable resource-sharing communities based on agentsâÃÂàown decisions about whether or not to behave cooperatively. This high level goal requires solving the following problems: synthetic task allocation, decentralized coalition formation and automated multiparty negotiation. For synthetic task allocation, in which each task needs to be accomplished by a virtual team composed of self-interested agents from different real organizations, my approach is to formalize the synthetic task allocation problem as an algorithmic mechanism design optimization problem. I have developed two approximation mechanisms that I prove are incentive compatible for a synthetic task allocation problem. This dissertation also develops a decentralized coalition formation mechanism, which is based on explicit negotiation among self-interested agents. Each agent makes its own decisions about whether or not to join a candidate coalition. The resulting coalitions are stable in the core in terms of coalition rationality. I have applied this mechanism to form resource sharing coalitions in computational grids and buyer coalitions in electronic markets. The developed negotiation mechanism in the decentralized coalition formation mechanism realizes automated multilateral negotiation among self-interested agents who have symmetric authority (i.e., no mediator exists and agents are peers). In combination, the decentralized allocation mechanisms presented in this dissertation lay a foundation for realizing automated resource management in open and scalable virtual organizations

Integration of a mobile autonomous robot in a surveillance multi-agent system

This dissertation aims to guarantee the integration of a mobile autonomous robot equipped with many sensors in a multi-agent distributed and georeferenced surveillance system. The integration of a mobile autonomous robot in this system leads to new features that will be available to clients of surveillance system may use. These features may be of two types: using the robot as an agent that will act in the environment or by using the robot as a mobile set of sensors. As an agent in the system, the robot can move to certain locations when alerts are received, in order to acknowledge the underlying events or take to action in order to assist in resolving this event. As a sensor platform in the system, it is possible to access information that is read from the sensors of the robot and access complementary measurements to the ones taken by other sensors in the multi-agent system. To integrate this mobile robot in an effective way it is necessary to extend the current multi-agent system architecture to make the connection between the two systems and to integrate the functionalities provided by the robot into the multi-agent system