Mission programming for flying ensembles: combining planning with self-organization

The application of autonomous mobile robots can improve many situations of our daily lives. Robots can enhance working conditions, provide innovative techniques for different research disciplines, and support rescue forces in an emergency. In particular, flying robots have already shown their potential in many use-cases when cooperating in ensembles. Exploiting this potential requires sophisticated measures for the goal-oriented, application-specific programming of flying ensembles and the coordinated execution of so defined programs. Because different goals require different robots providing different capabilities, several software approaches emerged recently that focus on specifically designed robots. These approaches often incorporate autonomous planning, scheduling, optimization, and reasoning attributable to classic artificial intelligence. This allows for the goal-oriented instruction of ensembles, but also leads to inefficiencies if ensembles grow large or face uncertainty in the environment. By leaving the detailed planning of executions to individuals and foregoing optimality and goal-orientation, the selforganization paradigm can compensate for these drawbacks by scalability and robustness. In this thesis, we combine the advantageous properties of autonomous planning with that of self-organization in an approach to Mission Programming for Flying Ensembles. Furthermore, we overcome the current way of thinking about how mobile robots should be designed. Rather than assuming fixed-design robots, we assume that robots are modifiable in terms of their hardware at run-time. While using such robots enables their application in many different use cases, it also requires new software approaches for dealing with this flexible design. The contributions of this thesis thus are threefold. First, we provide a layered reference architecture for physically reconfigurable robot ensembles. Second, we provide a solution for programming missions for ensembles consisting of such robots in a goal-oriented fashion that provides measures for instructing individual robots or entire ensembles as desired in the specific use case. Third, we provide multiple self-organization mechanisms to deal with the system’s flexible design while executing such missions. Combining different self-organization mechanisms ensures that ensembles satisfy the static requirements of missions. We provide additional self-organization mechanisms for coordinating the execution in ensembles ensuring they meet the dynamic requirements of a mission. Furthermore, we provide a solution for integrating goal-oriented swarm behavior into missions using a general pattern we have identified for trajectory-modification-based swarm behavior. Using that pattern, we can modify, quantify, and further process the emergent effect of varying swarm behavior in a mission by changing only the parameters of its implementation. We evaluate results theoretically and practically in different case studies by deploying our techniques to simulated and real hardware.Der Einsatz von autonomen mobilen Robotern kann viele Abläufe unseres täglichen Lebens erleichtern. Ihr Einsatz kann Arbeitsbedingungen verbessern, als innovative Technik für verschiedene Forschungsdisziplinen dienen oder Rettungskräfte im Einsatz unterstützen. Insbesondere Flugroboter haben ihr Potenzial bereits in vielerlei Anwendungsfällen gezeigt, gerade wenn mehrere in Ensembles eingesetzt werden. Das Potenzial fliegender Ensembles zielgerichtet und anwendungsspezifisch auszuschöpfen erfordert ausgefeilte Programmiermethoden und Koordinierungsverfahren. Zu diesem Zweck sind zuletzt viele unterschiedliche und auf speziell entwickelte Roboter zugeschnittene Softwareansätze entstanden. Diese verwenden oft klassische Planungs-, Scheduling-, Optimierungs- und Reasoningverfahren. Während dies vor allem den zielgerichteten Einsatz von Ensembles ermöglicht, ist es jedoch auch oft ineffizient, wenn die Ensembles größer oder deren Einsatzumgebungen unsicher werden. Die genannten Nachteile können durch das Paradigma der Selbstorganisation kompensiert werden: Falls Anwendungen nicht zwangsläufig auf Optimalität und strikte Zielorientierung ausgelegt sind, kann so Skalierbarkeit und Robustheit im System erreicht werden. In dieser Arbeit werden die vorteilhaften Eigenschaften klassischer Planungstechniken mit denen der Selbstorganisation in einem Ansatz zur Missionsprogrammierung für fliegende Ensembles kombiniert. In der dafür entwickelten Lösung wird von der aktuell etablierten Ansicht einer unveränderlichen Roboterkonstruktion abgewichen. Stattdessen wird die Hardwarezusammenstellung der Roboter als zur Laufzeit modifizierbar angesehen. Der Einsatz solcher Roboter erfordert neue Softwareansätze um mit genannter Flexibilität umgehen zu können. Die hier vorgestellten Beiträge zu diesem Thema lassen sich in drei Punkten zusammenfassen: Erstens wird eine Schichtenarchitektur als Referenz für physikalisch konfigurierbare Roboterensembles vorgestellt. Zweitens wird eine Lösung zur zielorientierten Missions-Programmierung für derartige Ensembles präsentiert, mit der sowohl einzelne Roboter als auch ganze Ensembles instruiert werden können. Drittens werden mehrere Selbstorganisationsmechanismen vorgestellt, die die autonome Ausführung so erstellter Missionen ermöglichen. Durch die Kombination verschiedener Selbstorganisationsmechanismen wird sichergestellt, dass Ensembles die missionsspezifischen Anforderungen erfüllen. Zusätzliche Selbstorganisationsmechanismen ermöglichen die koordinierte Ausführung der Missionen durch die Ensembles. Darüber hinaus bietet diese Lösung die Möglichkeit der Integration zielorientierten Schwarmverhaltens. Durch ein allgemeines algorithmisches Verfahren für auf Trajektorien-Modifikation basierendes Schwarmverhalten können allein durch die Änderung des Parametersatzes unterschiedliche emergente Effekte in einer Mission erzielt, quantifiziert und weiterverarbeitet werden. Zur theoretischen und praktischen Evaluierung der Ergebnisse dieser Arbeit wurden die vorgestellten Techniken in verschiedenen Fallstudien auf simulierter sowie realer Hardware zum Einsatz gebracht

Maple-Swarm: programming collective behavior for ensembles by extending HTN-planning

Programming goal-oriented behavior in collective adaptive systems is complex, requires high effort, and is failure-prone. If the system's user wants to deploy it in a real-world environment, hurdles get even higher: Programs urgently require to be situation-aware. With our framework Maple, we previously presented an approach for easing the act of programming such systems on the level of particular robot capabilities. In this paper, we extend our approach for ensemble programming with the possibility to address virtual swarm capabilities encapsulating collective behavior to whole groups of agents. By using the respective concepts in an extended version of hierarchical task networks and by adapting our self-organization mechanisms for executing plans resulting thereof, we can achieve that all agents, any agent, any other set of agents, or a swarm of agents execute (swarm) capabilities. Moreover, we extend the possibilities of expressing situation awareness during planning by introducing planning variables that can get modified at design-time or run-time as needed. We illustrate the possibilities with examples each. Further, we provide a graphical front-end offering the possibility to generate mission-specific problem domain descriptions for ensembles including a lightweight simulation for validating plans

Decentralized Vision-Based Byzantine Agent Detection in Multi-Robot Systems with IOTA Smart Contracts

Multiple opportunities lie at the intersection of multi-robot systems and distributed ledger technologies (DLTs). In this work, we investigate the potential of new DLT solutions such as IOTA, for detecting anomalies and byzantine agents in multi-robot systems in a decentralized manner. Traditional blockchain approaches are not applicable to real-world networked and decentralized robotic systems where connectivity conditions are not ideal. To address this, we leverage recent advances in partition-tolerant and byzantine-tolerant collaborative decision-making processes with IOTA smart contracts. We show how our work in vision-based anomaly and change detection can be applied to detecting byzantine agents within multiple robots operating in the same environment. We show that IOTA smart contracts add a low computational overhead while allowing to build trust within the multi-robot system. The proposed approach effectively enables byzantine robot detection based on the comparison of images submitted by the different robots and detection of anomalies and changes between them

Fast Marching based Rendezvous Path Planning for a Team of Heterogeneous Vehicle

A formulation is developed for deterministically calculating the optimized paths for a multi-agent system consisting of heterogeneous vehicles. The essence of this formulation is the calculation of the shortest time for each agent to reach every grid point from its known initial position. Such arrival time map can be readily assessed using the Fast Marching Method (FMM), a computational algorithm originally designed for solving boundary value problems of the Eikonal equation. Leveraging the FMM method, we demonstrate that the minimal time rendezvous point and paths for all member vehicles can be uniquely determined with minimal computational concerns. To showcase the potential of our method, we use an example of a virtual rendezvous scenario that entails the coordination of a ship, an underwater vehicle, an aerial vehicle, and a ground vehicle to converge at the optimal location within the Tampa Bay area in minimal time. It illustrates the value of the developed framework in efficiently constructing continuous path planning, while accommodating different operational constraints of heterogeneous member vehicles

Multi-Robot Systems: Challenges, Trends and Applications

This book is a printed edition of the Special Issue entitled “Multi-Robot Systems: Challenges, Trends, and Applications” that was published in Applied Sciences. This Special Issue collected seventeen high-quality papers that discuss the main challenges of multi-robot systems, present the trends to address these issues, and report various relevant applications. Some of the topics addressed by these papers are robot swarms, mission planning, robot teaming, machine learning, immersive technologies, search and rescue, and social robotics