Algorithms for multi-robot systems on the cooperative exploration & last-mile delivery problems

La aparición de los vehículos aéreos no tripulados (UAVs) y de los vehículos terrestres no tripulados (UGVs) ha llevado a la comunidad científica a enfrentarse a problemas ideando paradigmas de cooperación con UGVs y UAVs. Sin embargo, no suele ser trivial determinar si la cooperación entre UGVs y UAVs es adecuada para un determinado problema. Por esta razón, en esta tesis, investigamos un paradigma particular de cooperación UGV-UAV en dos problemas de la literatura, y proponemos un controlador autónomo para probarlo en escenarios simulados. Primero, formulamos un problema particular de exploración cooperativa que consiste en alcanzar un conjunto de puntos de destino en un área de exploración a gran escala. Este problema define al UGV como una estación de carga móvil para transportar el UAV a través de diferentes lugares desde donde el UAV puede alcanzar los puntos de destino. Por consiguiente, proponemos el algoritmo TERRA para resolverlo. Este algoritmo se destaca por dividir el problema de exploración en cinco subproblemas, en los que cada subproblema se resuelve en una etapa particular del algoritmo. Debido a la explosión de la entrega de paquetes en las empresas de comercio electrónico, formulamos también una generalización del conocido problema de la entrega en la última milla. En este caso, el UGV actúa como una estación de carga móvil que transporta a los paquetes y a los UAVs, y estos se encargan de entregarlos. De esta manera, seguimos la estrategia de división descrita por TERRA, y proponemos el algoritmo COURIER. Este algoritmo replica las cuatro primeras etapas de TERRA, pero construye una nueva quinta etapa para producir un plan de tareas que resuelva el problema. Para evaluar el paradigma de cooperación UGV-UAV en escenarios simulados, proponemos el controlador autónomo ARIES. Este controlador sigue un enfoque jerárquico descentralizado de líder-seguidor para integrar cualquier paradigma de cooperación de manera distribuida. Ambos algoritmos han sido caracterizados para identificar los aspectos relevantes del paradigma de cooperación en los problemas relacionados. Además, ambos demuestran un gran rendimiento del paradigma de cooperación en tales problemas, y al igual que el controlador autónomo, revelan un gran potencial para futuras aplicaciones reales.The emergence of Unmanned Aerial Vehicles (UAVs) and Unmanned Ground Vehicles (UGVs) has conducted the research community to face historical complex problems by devising UGV-UAV cooperation paradigms. However, it is usually not a trivial task to determine whether or not a UGV-UAV cooperation is suitable for a particular problem. For this reason, in this thesis, we investigate a particular UGV-UAV cooperation paradigm over two problems in the literature, and we propose an autonomous controller to test it on simulated scenarios. Driven by the planetary exploration, we formulate a particular cooperative exploration problem consisting of reaching a set of target points in a large-scale exploration area. This problem defines the UGV as a moving charging station to carry the UAV through different locations from where the UAV can reach the target points. Consequently, we propose the cooperaTive ExploRation Routing Algorithm (TERRA) to solve it. This algorithm stands out for splitting up the exploration problem into five sub-problems, in which each sub-problem is solved in a particular stage of the algorithm. In the same way, driven by the explosion of parcels delivery in e-commerce companies, we formulate a generalization of the well-known last-mile delivery problem. This generalization defines the same UGV’s and UAV’s rol as the exploration problem. That is, the UGV acts as a moving charging station which carries the parcels along several UAVs to deliver them. In this way, we follow the split strategy depicted by TERRA to propose the COoperative Unmanned deliveRIEs planning algoRithm (COURIER). This algorithm replicates the first four TERRA’s stages, but it builds a new fifth stage to produce a task plan solving the problem. In order to evaluate the UGV-UAV cooperation paradigm on simulated scenarios, we propose the Autonomous coopeRatIve Execution System (ARIES). This controller follows a hierarchical decentralized leader-follower approach to integrate any cooperation paradigm in a distributed manner. Both algorithms have been characterized to identify the relevant aspects of the cooperation paradigm in the related problems. Also, both of them demonstrate a great performance of the cooperation paradigm in such problems, and as well as the autonomous controller, reveal a great potential for future real applications

Fault-tolerant control policies for multi-robot systems

Throughout the past decade, we have witnessed an active interest in distributed motion coordination algorithms for networked mobile autonomous robots. Often, in multi-robot systems, each robot executing a coordination task is a little cost, a disposable autonomous agent that has ad-hoc sensing or communication capability, and limited mobility. Coordination tasks that a group of multiple mobile robots might perform include formation control, rendezvous, distributed estimation, deployment, flocking, etc. Also, there are challenging tasks that are more suitable for a group of mobile robots than an individual robot, such as surveillance, exploration, or hazardous environmental monitoring. The field has been collectively investigated by many researchers in robotics, control, artificial intelligence, and distributed computing. However, relatively little work has been done on developing algorithms to provide resilience to failures that can occur. The problem is extremely difficult to handle in that any partial failure of a robot is not readily detectable. Some failures in robot resources can have an adverse effect on not only the performance of the robot itself, but also other robots, and the collective task performance as well. This study presents the development of fault-tolerant distributed control policies for multi-robot systems. We consider two problems: rendezvous and coverage. For the former, the goal is to bring all robots to a common location, while for the latter the goal is to deploy robots to achieve optimal coverage of an environment. We consider the case in which each robot is an autonomous decision maker that is anonymous (i.e., robots are indistinguishable to one another), memoryless (i.e., each robot makes decisions based upon only its current information), and dimensionless (i.e., collision checking is not considered). Each robot has a limited sensing range and can directly estimate the state of only those robots within that sensing range, which induces a network topology for the multi-robot system. We assume that it is not possible for the fault-free robots to identify the faulty robots (e.g., due to the anonymous property of the robots). For each problem, we provide an efficient computational framework and analysis of algorithms, all of which converge in the face of faulty robots under a few assumptions on the network topology and sensing abilities. A suite of experiments and simulations confirm our theoretical analysis and demonstrate that our proposed algorithms are useful in fault-prone multi-robot systems

Self-management Framework for Mobile Autonomous Systems

The advent of mobile and ubiquitous systems has enabled the development of autonomous systems such as wireless-sensors for environmental data collection and teams of collaborating Unmanned Autonomous Vehicles (UAVs) used in missions unsuitable for humans. However, with these range of new application domains comes a new challenge – enabling self-management in mobile autonomous systems. The primary challenge in using autonomous systems for real-life missions is shifting the burden of management from humans to these systems themselves without loss of the ability to adapt to failures, changes in context, and changing user requirements. Autonomous systems have to be able to manage themselves individually as well as to form self-managing teams that are able to recover or adapt to failures, protect themselves from attacks and optimise performance. This thesis proposes a novel distributed policy-based framework that enables autonomous systems to perform self management individually and as a team. The framework allows missions to be specified in terms of roles in an adaptable and reusable way, enables dynamic and secure team formation with a utility-based approach for optimal role assignment, caters for communication link maintenance among team members and recovery from failure. Adaptive management is achieved by employing an architecture that uses policy-based techniques to allow dynamic modification of the management strategy relating to resources, role behaviour, team and communications management, without reloading the basic software within the system. Evaluation of the framework shows that it is scalable with respect to the number of roles, and consequently the number of autonomous systems participating in the mission. It is also shown to be optimal with respect to role assignments, and robust to intermittent communication link disconnections and permanent team-member failures. The prototype implementation was tested on mobile robots as a proof-ofconcept demonstration

MARLAS: Multi Agent Reinforcement Learning for cooperated Adaptive Sampling

The multi-robot adaptive sampling problem aims at finding trajectories for a team of robots to efficiently sample the phenomenon of interest within a given endurance budget of the robots. In this paper, we propose a robust and scalable approach using decentralized Multi-Agent Reinforcement Learning for cooperated Adaptive Sampling (MARLAS) of quasi-static environmental processes. Given a prior on the field being sampled, the proposed method learns decentralized policies for a team of robots to sample high-utility regions within a fixed budget. The multi-robot adaptive sampling problem requires the robots to coordinate with each other to avoid overlapping sampling trajectories. Therefore, we encode the estimates of neighbor positions and intermittent communication between robots into the learning process. We evaluated MARLAS over multiple performance metrics and found it to outperform other baseline multi-robot sampling techniques. We further demonstrate robustness to communication failures and scalability with both the size of the robot team and the size of the region being sampled. The experimental evaluations are conducted both in simulations on real data and in real robot experiments on demo environmental setup

Active strategies for coordination of solitary robots

Thesis (PhD)--Stellenbosch University, 2020.ENGLISH ABSTRACT: This thesis considers the problem of search of an unknown environment by multiple solitary robots: self-interested robots without prior knowledge about each other, and with restricted perception and communication capacity. When solitary robots accidentally interact with each other, they can leverage each other’s information to work more effectively. In this thesis, we consider three problems related to the treatment of solitary robots: coordination, construction of a view of the network formed when robots interact, and classifier fusion. Coordination is the key focus for search and rescue. The other two problems are related areas inspired by the problems we encountered while developing our coordination method. We propose a coordination strategy based on cellular decomposition of the search environment, which provides sustainable performance when a known available search time (bound) is insufficient to cover the entire search environment. A sustainable performance is achieved when robots that know about each other explore non-overlapping regions. For network construction, we propose modifications to a scalable decentralised method for constructing a model of network topology which reduces the number of messages exchanged between interacting nodes. The method has wider potential application than mobile robotics. For classifier fusion, we propose an iterative method where outputs of classifiers are combined without using any further information about the behaviour of the individual classifiers. Our approaches for each of these problems are compared to state-of-the-art methods.AFRIKAANSE OPSOMMING: Hierdie tesis beskou die probleem van soektog in ’n onbekende omgewing deur ’n aantal alleenstaande robotte: selfbelangstellende robotte sonder voorafgaande kennis van mekaar, en met beperkte persepsie- en kommunikasievermoëns. Wanneer alleenstaande robotte toevallig mekaar raakloop, kan hulle met mekaar inligting uitruil om meer effektief te werk. Hierdie tesis beskou drie probleme wat verband hou met die hantering van alleenstaande robotte: konstruksie van ’n blik van die netwerk gevorm deur interaksie tussen robotte, koördinasie en klassifiseerdersamesmelting. Koördinasie is die hoof fokuspunt vir soek en redding. Die ander twee probleme is uit verwante areas, gemotiveer deur uitdagings wat ons ervaar het tydens die ontwikkeling van ons koördineringsmetode. Ons stel ’n skaleerbare desentraliseerde metode voor om ’n model van netwerktopologie te bou wat minder boodskappe tussen wisselwerkende nodusse hoet te verruil. Die metode het wyer potensiële toepassings as mobiele robotika. Vir koördinasie, stel ons ’n strategie voor gebaseer op sellulêre ontbinding van die soekomgewing, wat volhoubare prestasie toon wanneer ’n bekende soektyd onvoldoende is om die hele soekomgewing te dek. Vir klassifiseerdersamesmelting, stel ons ’n iteratiewe metode voor, waar klassifiseerders se voorspellings gekombineer word sonder om enige verdere inligting oor die gedrag van die individuele klassifiseerders te gebruik. Ons benaderings vir elkeen van hierdie probleme word vergelyk met stand-van-die-kuns metodes.The financial assistance of the African Institute for Mathematical Sciences (AIMS) and CSIR-SU Centre for Artificial Intelligence Research Group (CSIR-SU CAIR) towards this research is hereby acknowledged. Opinions expressed and conclusions arrived at, are those of the author and are not necessarily to be attributed to the AIMS and CSIR-SU CAIR.Doctora

An Overview of Recent Progress in the Study of Distributed Multi-agent Coordination

This article reviews some main results and progress in distributed multi-agent coordination, focusing on papers published in major control systems and robotics journals since 2006. Distributed coordination of multiple vehicles, including unmanned aerial vehicles, unmanned ground vehicles and unmanned underwater vehicles, has been a very active research subject studied extensively by the systems and control community. The recent results in this area are categorized into several directions, such as consensus, formation control, optimization, task assignment, and estimation. After the review, a short discussion section is included to summarize the existing research and to propose several promising research directions along with some open problems that are deemed important for further investigations

Robotic Wireless Sensor Networks

In this chapter, we present a literature survey of an emerging, cutting-edge, and multi-disciplinary field of research at the intersection of Robotics and Wireless Sensor Networks (WSN) which we refer to as Robotic Wireless Sensor Networks (RWSN). We define a RWSN as an autonomous networked multi-robot system that aims to achieve certain sensing goals while meeting and maintaining certain communication performance requirements, through cooperative control, learning and adaptation. While both of the component areas, i.e., Robotics and WSN, are very well-known and well-explored, there exist a whole set of new opportunities and research directions at the intersection of these two fields which are relatively or even completely unexplored. One such example would be the use of a set of robotic routers to set up a temporary communication path between a sender and a receiver that uses the controlled mobility to the advantage of packet routing. We find that there exist only a limited number of articles to be directly categorized as RWSN related works whereas there exist a range of articles in the robotics and the WSN literature that are also relevant to this new field of research. To connect the dots, we first identify the core problems and research trends related to RWSN such as connectivity, localization, routing, and robust flow of information. Next, we classify the existing research on RWSN as well as the relevant state-of-the-arts from robotics and WSN community according to the problems and trends identified in the first step. Lastly, we analyze what is missing in the existing literature, and identify topics that require more research attention in the future