Decentralized multi-robot cooperation with auctioned pomdps

ABSTRACT Planning under uncertainty faces a scalability problem when considering multi-robot teams, as the information space scales exponentially with the number of robots. To address this issue, this paper proposes to decentralize multi-agent Partially Observable Markov Decision Process (POMDPs) while maintaining cooperation between robots by using POMDP policy auctions. Auctions provide a flexible way of coordinating individual policies modeled by POMDPs and have low communication requirements. Additionally, communication models in the multi-agent POMDP literature severely mismatch with real inter-robot communication. We address this issue by applying a decentralized data fusion method in order to efficiently maintain a joint belief state among the robots. The paper focuses on a cooperative tracking application, in which several robots have to jointly track a moving target of interest. The proposed ideas are illustrated in real multi-robot experiments, showcasing the flexible and robust coordination that our techniques can provide

Formal Modelling for Multi-Robot Systems Under Uncertainty

Purpose of Review: To effectively synthesise and analyse multi-robot behaviour, we require formal task-level models which accurately capture multi-robot execution. In this paper, we review modelling formalisms for multi-robot systems under uncertainty, and discuss how they can be used for planning, reinforcement learning, model checking, and simulation. Recent Findings: Recent work has investigated models which more accurately capture multi-robot execution by considering different forms of uncertainty, such as temporal uncertainty and partial observability, and modelling the effects of robot interactions on action execution. Other strands of work have presented approaches for reducing the size of multi-robot models to admit more efficient solution methods. This can be achieved by decoupling the robots under independence assumptions, or reasoning over higher level macro actions. Summary: Existing multi-robot models demonstrate a trade off between accurately capturing robot dependencies and uncertainty, and being small enough to tractably solve real world problems. Therefore, future research should exploit realistic assumptions over multi-robot behaviour to develop smaller models which retain accurate representations of uncertainty and robot interactions; and exploit the structure of multi-robot problems, such as factored state spaces, to develop scalable solution methods.Comment: 23 pages, 0 figures, 2 tables. Current Robotics Reports (2023). This version of the article has been accepted for publication, after peer review (when applicable) but is not the Version of Record and does not reflect post-acceptance improvements, or any corrections. The Version of Record is available online at: https://dx.doi.org/10.1007/s43154-023-00104-

Simultaneous Auctions for "Rendez-Vous" Coordination Phases in Multi-robot Multi-task Mission

International audienceThis paper presents a protocol that permits to automatically allocate tasks, in a distributed way, among a fleet of agents when communication is not permanently available. In cooperation settings when communication is available only during short periods, it is difficult to build joint policies of agents to collectively accomplish a mission defined by a set of tasks. The proposed approach aims to punctually coordinate the agents during "Rendezvous'' phases defined by the short periods when communication is available. This approach consists of a series of simultaneous auctions to coordinate individual policies computed in a distributed way from Markov decision processes oriented by several goals. These policies allow the agents to evaluate their own relevance in each task achievement and to communicate bids when possible. This approach is illustrated on multi-mobile-robot missions similar to distributed traveling salesmen problem. Experimental results (through simulation and on real robots) demonstrate that high-quality allocations are quickly computed

Hybrid control of a multi-agent UAV fleet for formation flight with Dec-POMDP

Voo em formação e controle cooperativo de múltplos VANTs têm sido áreas de estudo de grande interesse das pesquisas mais recentes. Enquanto diversos métodos estão sendo criados para rastreamento fino de referência e formação, muitos empecilhos ainda precisam ser superados tais como descentralização, comunicação confiável, divisão de tarefas, evitamento de colisões e autonomia. Neste cenário, este trabalho propõe um sistema de controle híbrido para ser usado no voo em formação de múltiplos VANTs de asa-fixa, aumentando a performance e eficiência do grupo por permitir que este planeje e controle a frota através de comandos discretos e contínuos. Para contornar o problema da centralização, o método de planejamento Dec-POMDP foi utilizado, de modo a evitar a confiabilidade em um nó central de tomada de decisão, como um líder ou uma estação em terra. Através do uso deste algoritmo, este método também considera transições e observações estocásticas para permitir uma tomada de decisão eficiente mesmo em ambientes ruidosos e incertos. Além disso, a implementação deste sistema em uma malha externa permite reduzir o tempo computacional. Através de simulações, o sistema proposto como uma topologia chaveada entre a política Dec-POMDP e controles PID foi comparada com outros métodos da literatura e apresentou uma performance satisfatória para o voo em formação.CAPESFormation flight and cooperative control of multiple UAVs has been areas of studies of great interest by the most recent researches. As many methods are being created to make fine reference and formation tracking, collision avoidance and disturbance rejection, many trammels are still necessary to be overcome such as decentralization, reliable communications, task division, obstacle avoidance and autonomy. In such scenario, this work proposes an hybrid control system to be used in formation flight of multiple fixed-wing UAVs, increasing the group performance and efficiency by allowing it to plan and control the fleet by using both discrete and continuous commands. To overcome the centralization problem, the Dec-POMDP planning method is used, in order to avoid the reliability on a central decision node, such as a leader or a ground station. By using such algorithm, this approach also considers stochastic transitions and observations to allow an effective decision making in noisy and uncertain environments. Also, the implementation of such system in an outer loop allows to reduce the computational time. Through simulations, the system proposed as a switching topology between the Dec-POMDP policy and PID controls was compared to other methods in the literature and has presented satisfactory performance for formation flight