Decentralized dynamic task allocation for UAVs with limited communication range

We present the Limited-range Online Routing Problem (LORP), which involves a team of Unmanned Aerial Vehicles (UAVs) with limited communication range that must autonomously coordinate to service task requests. We first show a general approach to cast this dynamic problem as a sequence of decentralized task allocation problems. Then we present two solutions both based on modeling the allocation task as a Markov Random Field to subsequently assess decisions by means of the decentralized Max-Sum algorithm. Our first solution assumes independence between requests, whereas our second solution also considers the UAVs' workloads. A thorough empirical evaluation shows that our workload-based solution consistently outperforms current state-of-the-art methods in a wide range of scenarios, lowering the average service time up to 16%. In the best-case scenario there is no gap between our decentralized solution and centralized techniques. In the worst-case scenario we manage to reduce by 25% the gap between current decentralized and centralized techniques. Thus, our solution becomes the method of choice for our problem

Cost Adaptation for Robust Decentralized Swarm Behaviour

Decentralized receding horizon control (D-RHC) provides a mechanism for coordination in multi-agent settings without a centralized command center. However, combining a set of different goals, costs, and constraints to form an efficient optimization objective for D-RHC can be difficult. To allay this problem, we use a meta-learning process -- cost adaptation -- which generates the optimization objective for D-RHC to solve based on a set of human-generated priors (cost and constraint functions) and an auxiliary heuristic. We use this adaptive D-RHC method for control of mesh-networked swarm agents. This formulation allows a wide range of tasks to be encoded and can account for network delays, heterogeneous capabilities, and increasingly large swarms through the adaptation mechanism. We leverage the Unity3D game engine to build a simulator capable of introducing artificial networking failures and delays in the swarm. Using the simulator we validate our method on an example coordinated exploration task. We demonstrate that cost adaptation allows for more efficient and safer task completion under varying environment conditions and increasingly large swarm sizes. We release our simulator and code to the community for future work.Comment: Accepted to IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 201

Multi-robot Task Allocation using Agglomerative Clustering

The main objective of this thesis is to solve the problem of balancing tasks in the Multi-robot Task Allocation problem domain. When allocating a large number of tasks to a multi-robot system, it is important to balance the load effectively across the robots in the system. In this thesis an algorithm is proposed in which tasks are allocated through clustering, investigating the effectiveness of agglomerative hierarchical clustering as compared to K-means clustering. Once the tasks are clustered, each agent claims a cluster through a greedy self-assignment. This thesis investigates the performance both when all tasks are known ahead of time as well as when new tasks are injected into the system periodically. To account for new tasks, both global re-clustering and greedy clustering methods are considered. Three metrics: 1) total travel cost, 2) maximum distance traveled per robot, and 3) balancing cost index are used to compare the performance of the overall system in environments both with and without obstacles. The results collected from the experiments show that agglomerative hierarchical clustering is deterministic and better at minimizing the total travel cost, especially for large numbers of agents, whereas K-means works better to balance costs. In addition to this, the greedy approach for clustering new tasks works better for frequently appearing tasks than infrequent ones

Cooperative transportation scheduling : an application domain for DAI

A multiagent approach to designing the transportation domain is presented. The MARS system is described which models cooperative order scheduling within a society of shipping companies. We argue why Distributed Artificial Intelligence (DAI) offers suitable tools to deal with the hard problems in this domain. We present three important instances for DAI techniques that proved useful in the transportation application: cooperation among the agents, task decomposition and task allocation, and decentralised planning. An extension of the contract net protocol for task decomposition and task allocation is presented; we show that it can be used to obtain good initial solutions for complex resource allocation problems. By introducing global information based upon auction protocols, this initial solution can be improved significantly. We demonstrate that the auction mechanism used for schedule optimisation can also be used for implementing dynamic replanning. Experimental results are provided evaluating the performance of different scheduling strategies

An Auction-based Coordination Strategy for Task-Constrained Multi-Agent Stochastic Planning with Submodular Rewards

In many domains such as transportation and logistics, search and rescue, or cooperative surveillance, tasks are pending to be allocated with the consideration of possible execution uncertainties. Existing task coordination algorithms either ignore the stochastic process or suffer from the computational intensity. Taking advantage of the weakly coupled feature of the problem and the opportunity for coordination in advance, we propose a decentralized auction-based coordination strategy using a newly formulated score function which is generated by forming the problem into task-constrained Markov decision processes (MDPs). The proposed method guarantees convergence and at least 50% optimality in the premise of a submodular reward function. Furthermore, for the implementation on large-scale applications, an approximate variant of the proposed method, namely Deep Auction, is also suggested with the use of neural networks, which is evasive of the troublesome for constructing MDPs. Inspired by the well-known actor-critic architecture, two Transformers are used to map observations to action probabilities and cumulative rewards respectively. Finally, we demonstrate the performance of the two proposed approaches in the context of drone deliveries, where the stochastic planning for the drone league is cast into a stochastic price-collecting Vehicle Routing Problem (VRP) with time windows. Simulation results are compared with state-of-the-art methods in terms of solution quality, planning efficiency and scalability.Comment: 17 pages, 5 figure

Path planning for first responders in the presence of moving obstacles

Navigation services have gained much importance for all kinds of human activities ranging from tourist navigation to support of rescue teams in disaster management. However, despite the considerable amount of route guidance research that has been performed, many issues that are related to navigation for first responders still need to be addressed. During disasters, emergencies can result in different types of moving obstacles (e.g., fires, plumes, floods), which make some parts of the road network temporarily unavailable. After such incidents occur, responders have to go to different destinations to perform their tasks in the environment affected by the disaster. Therefore they need a path planner that is capable of dealing with such moving obstacles, as well as generating and coordinating their routes quickly and efficiently. During the past decades, more and more hazard simulations, which can modify the models with incorporation of dynamic data from the field, have been developed. These hazard simulations use methods such as data assimilation, stochastic estimation, and adaptive measurement techniques, and are able to generate more reliable results of hazards. This would allow the hazard simulation models to provide valuable information regarding the state of road networks affected by hazards, which supports path planning for first responders among the moving obstacles. The objective of this research is to develop an integrated navigation system for first responders in the presence of moving obstacles. Such system should be able to navigate one or more responders to one or multiple destinations avoiding the moving obstacles, using the predicted information of the moving obstacles generated from by hazard simulations. In this dissertation, the objective we have is expressed as the following research question: How do we safely and efficiently navigate one or more first responders to one or more destinations avoiding moving obstacles? To address the above research questions, this research has been conducted using the following outline: 1). literature review; 2). conceptual design and analysis; 3). implementation of the prototype; and 4). assessment of the prototype and adaption. We investigated previous research related to navigation in disasters, and designed an integrated navigation system architecture, assisting responders in spatial data storage, processing and analysis.Within this architecture, we employ hazard models to provide the predicted information about the obstacles, and select a geo-database to store the data needed for emergency navigation. Throughout the development of the prototype navigation system, we have proposed: a taxonomy of navigation among obstacles, which categorizes navigation cases on basis of type and multiplicity of first responders, destinations, and obstacles; a multi-agent system, which supports information collection from hazard simulations, spatio-temporal data processing and analysis, connection with a geo-database, and route generation in dynamic environments affected by disasters; data models, which structure the information required for finding paths among moving obstacles, capturing both static information, such as the type of the response team, the topology of the road network, and dynamic information, such as changing availabilities of roads during disasters, the uncertainty of the moving obstacles generated from hazard simulations, and the position of the vehicle; path planning algorithms, which generate routes for one or more responders in the presence of moving obstacles. Using the speed of vehicles, departure time, and the predicted information about the state of the road network, etc., three versions (I, II, and III) of Moving Obstacle Avoiding A* (MOAAStar) algorithms are developed: 1). MOAAstar– I/Non-waiting, which supports path planning in the case of forest fires; 2). MOAAstar–II/Waiting, which introduces waiting options to avoid moving obstacles like plumes; 3). MOAAstar–III/Uncertainty, which can handle the uncertainty in predictions of moving obstacles and incorporate the profile of responders into the routing. We have applied the developed prototype navigation system to different navigation cases with moving obstacles. The main conclusions drawn from our applications are summarized as follows: In the proposed taxonomy, we have identified 16 navigation cases that could occur in disaster response and need to be investigated. In addressing these navigation problems, it would be quite useful to employ computer simulations and models, which can make reliable predicted information about responders, the targets, and obstacles, in finding safe routes for the responders. The approach we provide is general and not limited to the cases of plumes and fires. In our data model, the data about the movement of hazards is represented as moving polygons. This allows the data model to be easily adjusted to merge and organize information from models of different types of disasters. For example, the areas that are affected by floods can also be represented as moving polygons. To facilitate the route calculation, not only the data of obstacles but also the information about the state of road networks affected by obstacles need to be structured and stored in the database. In planning routes for responders, the routing algorithms should incorporate the dynamic data of obstacles to be able to avoid the hazards. Besides, other factors, such as the operation time of tasks, the required arrival time, and departure time, also need to be considered to achieve the objectives in a rescue process, e.g., to minimize the delays caused by the moving obstacles. The profile of responders is quite important for generation of feasible routes for a specific disaster situation. The responders may have different protective equipment that allows them to pass through different types of moving obstacles, and thus can have different classification of risk levels to define the state of the road network. By taking into account the profile of the responders, the navigation system can propose customized and safe routes to them, which would facilitate their disaster response processes. On the basis of our findings, we suggest the following topics for future work: As presented Wang and Zlatanova (2013c), there are still a couple of navigation cases that need to be addressed, especially the ones that involve dynamic destinations. More algorithms would be needed to solve these navigation problems. Besides, some extreme cases (e.g., the obstacle covers the target point during the course of an incident) also need to be investigated. Using standard Web services, an Android navigation application, which can provide navigation services in the environment affected by hazards, needs to be developed and tested in both the daily practice and real disasters. In this application, a user interface with various styling options should also be designed for different situations, e.g., waiting and moving, day and night, and urgent and non-urgent. Because the communication infrastructure may not be available or work properly during a disaster response, a decentralized method is needed to allow different users to negotiate with each other and to make local agreements on the distribution of tasks in case there is no support from the central planning system. Another type of multi-agent system would be needed to handle this situation. Introduce variable traveling speed into the re-routing process. The vehicle speed plays an important role in generation of routes avoiding moving obstacle, and can be influenced by many factors, such as the obstacles, the type of vehicles, traffic conditions, and the type of roads. Therefore, it would be needed to investigate how to derive the current and future speed from trajectories of vehicles. Apply the system to aid navigation in various types of natural disasters, using different hazard simulation models (e.g., flood model). More types of agents would be needed and integrated into the system to handle heterogeneous data from these models. Extensions of the data model are also required to meet a wider range of informational needs when multiple disasters occur simultaneously

Design and Simulation Analysis of Deep Learning Based Approaches and Multi-Attribute Algorithms for Warehouse Task Selection

With the growth and adoption of global supply chains and internet technologies, warehouse operations have become more demanding. Particularly, the number of orders being processed over a given time frame is drastically increasing, leading to more work content. This makes operational tasks, such as material retrieval and storage, done manually more inefficient. To improve system-level warehouse efficiency, collaborating Autonomous Vehicles (AVs) are needed. Several design challenges encompass an AV, some critical aspects are navigation, path planning, obstacle avoidance, task selection decisions, communication, and control systems. The current study addresses the warehouse task selection problem given a dynamic pending task list and considering multiple attributes: distance, traffic, collaboration, and due date, using situational decision-making approaches. The study includes the design and analysis of two situational decision-making approaches for multi-attribute dynamic warehouse task selection: Deep Learning Approach for Multi-Attribute Task Selection (DLT) and Situation based Greedy (SGY) algorithm that uses a traditional algorithmic approach. The two approaches are designed and analyzed in the current work. Further, they are evaluated using a simulation-based experiment. The results show that both the DLT and SGY have potential and are effective in comparison to the earliest due date first and shortest travel distance-based rules in addressing the multi-attribute task selection needs of a warehouse operation under the given experimental conditions and trade-offs

TOKEN-BASED APPROACH FOR SCALABLE TEAMCOORDINATION

To form a cooperative multiagent team, autonomous agents are required to harmonize activities and make the best use of exclusive resources to achieve their common goal. In addition, to handle uncertainty and quickly respond to external environmental events, they should share knowledge and sensor in formation. Unlike small team coordination, agents in scalable team must limit the amount of their communications while maximizing team performance. Communication decisions are critical to scalable-team coordination because agents should target their communications, but these decisions cannot be supported by a precise model or by complete team knowledge.The hypothesis of my thesis is: local routing of tokens encapsulating discrete elements of control, based only on decentralized local probability decision models, will lead to efficient scalable coordination with several hundreds of agents. In my research, coordination controls including all domain knowledge, tasks and exclusive resources are encapsulated into tokens. By passing tokens around, agents transfer team controls encapsulated in the tokens. The team benefits when a token is passed to an agent who can make use of it, but communications incur costs. Hence, no single agent has sole responsible over any shared decision. The key problem lies in how agents make the correct decisions to target communications and pass tokens so that they will potentially benefit the team most when considering communication costs.My research on token-based coordination algorithm starts from the investigation of random walk of token movement. I found a little increase of the probabilities that agents make the right decision to pass a token, the overall efficiency of the token movement could be greatly enhanced. Moreover, if token movements are modeled as a Markov chain, I found that the efficiency of passing tokens could be significantly varied based on different network topologies.My token-based algorithm starts at the investigation of each single decision theoretic agents. Although under the uncertainties that exist in large multiagent teams, agents cannot act optimal, it is still feasible to build a probability model for each agents to rationally pass tokens. Specifically, this decision only allow agent to pass tokens over an associate network where only a few of team members are considered as token receiver.My proposed algorithm will build each agent's individual decision model based on all of its previously received tokens. This model will not require the complete knowledge of the team. The key idea is that I will make use of the domain relationships between pairs of coordination controls. Previously received tokens will help the receiver to infer whether the sender could benefit the team if a related token is received. Therefore, each token is used to improve the routing of other tokens, leading to a dramatic performance improvement when more tokens are added. By exploring the relationships between different types of coordination controls, an integrated coordination algorithm will be built, and an improvement of one aspect of coordination will enhance the performance of the others