Adaptive and learning-based formation control of swarm robots

Autonomous aerial and wheeled mobile robots play a major role in tasks such as search and rescue, transportation, monitoring, and inspection. However, these operations are faced with a few open challenges including robust autonomy, and adaptive coordination based on the environment and operating conditions, particularly in swarm robots with limited communication and perception capabilities. Furthermore, the computational complexity increases exponentially with the number of robots in the swarm. This thesis examines two different aspects of the formation control problem. On the one hand, we investigate how formation could be performed by swarm robots with limited communication and perception (e.g., Crazyflie nano quadrotor). On the other hand, we explore human-swarm interaction (HSI) and different shared-control mechanisms between human and swarm robots (e.g., BristleBot) for artistic creation. In particular, we combine bio-inspired (i.e., flocking, foraging) techniques with learning-based control strategies (using artificial neural networks) for adaptive control of multi- robots. We first review how learning-based control and networked dynamical systems can be used to assign distributed and decentralized policies to individual robots such that the desired formation emerges from their collective behavior. We proceed by presenting a novel flocking control for UAV swarm using deep reinforcement learning. We formulate the flocking formation problem as a partially observable Markov decision process (POMDP), and consider a leader-follower configuration, where consensus among all UAVs is used to train a shared control policy, and each UAV performs actions based on the local information it collects. In addition, to avoid collision among UAVs and guarantee flocking and navigation, a reward function is added with the global flocking maintenance, mutual reward, and a collision penalty. We adapt deep deterministic policy gradient (DDPG) with centralized training and decentralized execution to obtain the flocking control policy using actor-critic networks and a global state space matrix. In the context of swarm robotics in arts, we investigate how the formation paradigm can serve as an interaction modality for artists to aesthetically utilize swarms. In particular, we explore particle swarm optimization (PSO) and random walk to control the communication between a team of robots with swarming behavior for musical creation

Mission-based mobility models for UAV networks

Las redes UAV han atraído la atención de los investigadores durante la última década. Las numerosas posibilidades que ofrecen los sistemas single-UAV aumentan considerablemente al usar múltiples UAV. Sin embargo, el gran potencial del sistema multi-UAV viene con un precio: la complejidad de controlar todos los aspectos necesarios para garantizar que los UAVs cumplen la misión que se les ha asignado. Ha habido numerosas investigaciones dedicadas a los sistemas multi-UAV en el campo de la robótica en las cuales se han utilizado grupos de UAVs para diferentes aplicaciones. Sin embargo, los aspectos relacionados con la red que forman estos sistemas han comenzado a reclamar un lugar entre la comunidad de investigación y han hecho que las redes de UAVs se consideren como un nuevo paradigma entre las redes multi-salto. La investigación de redes de UAVs, de manera similar a otras redes multi-salto, se divide principalmente en dos categorías: i) modelos de movilidad que capturan la movilidad de la red, y ii) algoritmos de enrutamiento. Ambas categorías han heredado muchos algoritmos que pertenecían a las redes MANET, que fueron el primer paradigma de redes multi-salto que atrajo la atención de los investigadores. Aunque hay esfuerzos de investigación en curso que proponen soluciones para ambas categorías, el número de modelos de movilidad y algoritmos de enrutamiento específicos para redes UAV es limitado. Además, en el caso de los modelos de movilidad, las soluciones existentes propuestas son simplistas y apenas representan la movilidad real de un equipo de UAVs, los cuales se utilizan principalmente en operaciones orientadas a misiones, en la que cada UAV tiene asignados movimientos específicos. Esta tesis propone dos modelos de movilidad basados en misiones para una red de UAVs que realiza dos operaciones diferentes. El escenario elegido en el que se desarrollan las misiones corresponde con una región en la que ha ocurrido, por ejemplo, un desastre natural. La elección de este tipo de escenario se debe a que en zonas de desastre, la infraestructura de comunicaciones comúnmente está dañada o totalmente destruida. En este tipo de situaciones, una red de UAVs ofrece la posibilidad de desplegar rápidamente una red de comunicaciones. El primer modelo de movilidad, llamado dPSO-U, ha sido diseñado para capturar la movilidad de una red UAV en una misión con dos objetivos principales: i) explorar el área del escenario para descubrir las ubicaciones de los nodos terrestres, y ii) hacer que los UAVs converjan de manera autónoma a los grupos en los que se organizan los nodos terrestres (también conocidos como clusters). El modelo de movilidad dPSO-U se basa en el conocido algoritmo particle swarm optimization (PSO), considerando los UAV como las partículas del algoritmo, y también utilizando el concepto de valores dinámicos para la inercia, el local best y el neighbour best de manera que el modelo de movilidad tenga ambas capacidades: la de exploración y la de convergencia. El segundo modelo, denominado modelo de movilidad Jaccard-based, captura la movilidad de una red UAV que tiene asignada la misión de proporcionar servicios de comunicación inalámbrica en un escenario de mediano tamaño. En este modelo de movilidad se ha utilizado una combinación del virtual forces algorithm (VFA), de la distancia Jaccard entre cada par de UAVs y metaheurísticas como hill climbing y simulated annealing, para cumplir los dos objetivos de la misión: i) maximizar el número de nodos terrestres (víctimas) que se encuentran bajo el área de cobertura inalámbrica de la red UAV, y ii) mantener la red UAV como una red conectada, es decir, evitando las desconexiones entre UAV. Se han realizado simulaciones exhaustivas con herramientas software específicamente desarrolladas para los modelos de movilidad propuestos. También se ha definido un conjunto de métricas para cada modelo de movilidad. Estas métricas se han utilizado para validar la capacidad de los modelos de movilidad propuestos de emular los movimientos de una red UAV en cada misión.UAV networks have attracted the attention of the research community in the last decade. The numerous capabilities of single-UAV systems increase considerably by using multiple UAVs. The great potential of a multi-UAV system comes with a price though: the complexity of controlling all the aspects required to guarantee that the UAV team accomplish the mission that it has been assigned. There have been numerous research works devoted to multi-UAV systems in the field of robotics using UAV teams for different applications. However, the networking aspects of multi-UAV systems started to claim a place among the research community and have made UAV networks to be considered as a new paradigm among the multihop ad hoc networks. UAV networks research, in a similar manner to other multihop ad hoc networks, is mainly divided into two categories: i) mobility models that capture the network mobility, and ii) routing algorithms. Both categories have inherited previous algorithms mechanisms that originally belong to MANETs, being these the first multihop networking paradigm attracting the attention of researchers. Although there are ongoing research efforts proposing solutions for the aforementioned categories, the number of UAV networks-specific mobility models and routing algorithms is limited. In addition, in the case of the mobility models, the existing solutions proposed are simplistic and barely represent the real mobility of a UAV team, which are mainly used in missions-oriented operations. This thesis proposes two mission-based mobility models for a UAV network carrying out two different operations over a disaster-like scenario. The reason for selecting a disaster scenario is because, usually, the common communication infrastructure is malfunctioning or completely destroyed. In these cases, a UAV network allows building a support communication network which is rapidly deployed. The first mobility model, called dPSO-U, has been designed for capturing the mobility of a UAV network in a mission with two main objectives: i) exploring the scenario area for discovering the location of ground nodes, and ii) making the UAVs to autonomously converge to the groups in which the nodes are organized (also referred to as clusters). The dPSO-U mobility model is based on the well-known particle swarm optimization algorithm (PSO), considering the UAVs as the particles of the algorithm, and also using the concept of dynamic inertia, local best and neighbour best weights so the mobility model can have both abilities: exploration and convergence. The second one, called Jaccard-based mobility model, captures the mobility of a UAV network that has been assigned with the mission of providing wireless communication services in a medium-scale scenario. A combination of the virtual forces algorithm (VFA), the Jaccard distance between each pair of UAVs and metaheuristics such as hill climbing or simulated annealing have been used in this mobility model in order to meet the two mission objectives: i) to maximize the number of ground nodes (i.e. victims) under the UAV network wireless coverage area, and ii) to maintain the UAV network as a connected network, i.e. avoiding UAV disconnections. Extensive simulations have been performed with software tools that have been specifically developed for the proposed mobility models. Also, a set of metrics have been defined and measured for each mobility model. These metrics have been used for validating the ability of the proposed mobility models to emulate the movements of a UAV network in each mission

Machine Learning for Unmanned Aerial System (UAS) Networking

Fueled by the advancement of 5G new radio (5G NR), rapid development has occurred in many fields. Compared with the conventional approaches, beamforming and network slicing enable 5G NR to have ten times decrease in latency, connection density, and experienced throughput than 4G long term evolution (4G LTE). These advantages pave the way for the evolution of Cyber-physical Systems (CPS) on a large scale. The reduction of consumption, the advancement of control engineering, and the simplification of Unmanned Aircraft System (UAS) enable the UAS networking deployment on a large scale to become feasible. The UAS networking can finish multiple complex missions simultaneously. However, the limitations of the conventional approaches are still a big challenge to make a trade-off between the massive management and efficient networking on a large scale. With 5G NR and machine learning, in this dissertation, my contributions can be summarized as the following: I proposed a novel Optimized Ad-hoc On-demand Distance Vector (OAODV) routing protocol to improve the throughput of Intra UAS networking. The novel routing protocol can reduce the system overhead and be efficient. To improve the security, I proposed a blockchain scheme to mitigate the malicious basestations for cellular connected UAS networking and a proof-of-traffic (PoT) to improve the efficiency of blockchain for UAS networking on a large scale. Inspired by the biological cell paradigm, I proposed the cell wall routing protocols for heterogeneous UAS networking. With 5G NR, the inter connections between UAS networking can strengthen the throughput and elasticity of UAS networking. With machine learning, the routing schedulings for intra- and inter- UAS networking can enhance the throughput of UAS networking on a large scale. The inter UAS networking can achieve the max-min throughput globally edge coloring. I leveraged the upper and lower bound to accelerate the optimization of edge coloring. This dissertation paves a way regarding UAS networking in the integration of CPS and machine learning. The UAS networking can achieve outstanding performance in a decentralized architecture. Concurrently, this dissertation gives insights into UAS networking on a large scale. These are fundamental to integrating UAS and National Aerial System (NAS), critical to aviation in the operated and unmanned fields. The dissertation provides novel approaches for the promotion of UAS networking on a large scale. The proposed approaches extend the state-of-the-art of UAS networking in a decentralized architecture. All the alterations can contribute to the establishment of UAS networking with CPS

Multi-Robot Systems: Challenges, Trends and Applications

This book is a printed edition of the Special Issue entitled “Multi-Robot Systems: Challenges, Trends, and Applications” that was published in Applied Sciences. This Special Issue collected seventeen high-quality papers that discuss the main challenges of multi-robot systems, present the trends to address these issues, and report various relevant applications. Some of the topics addressed by these papers are robot swarms, mission planning, robot teaming, machine learning, immersive technologies, search and rescue, and social robotics

Human–machine network through bio‑inspired decentralized swarm intelligence and heterogeneous teaming in SAR operations

Disaster management has always been a struggle due to unpredictable changing conditions and chaotic occurrences that require real-time adaption. Highly optimized missions and robust systems mitigate uncertainty effects and improve notoriously success rates. This paper brings a niching hybrid human–machine system that combines UAVs fast responsiveness with two robust, decentralized, and scalable bio-inspired techniques. Cloud-Sharing Network (CSN) and Pseudo-Central Network (PCN), based on Bacterial and Honeybee behaviors, are presented, and applied to Safe and Rescue (SAR) operations. A post-earthquake scenario is proposed, where a heterogeneous fleet of UAVs cooperates with human rescue teams to detect and locate victims distributed along the map. Monte Carlo simulations are carried out to test both approaches through state-of-the-art metrics. This paper introduces two hybrid and bio-inspired schemes to deal with critical scouting stages, poor communications environments and high uncertainly levels in disaster release operations. Role heterogeneity, path optimization and hive data-sharing structure give PCN an efficient performance as far as task allocation and communications are concerned. Cloud-sharing network gains strength when the allocated agents per victim and square meter is high, allowing fast data transmission. Potential applications of these algorithms are not only comprehended in SAR field, but also in surveillance, geophysical mapping, security and planetary exploration

Swarm Robotics

Collectively working robot teams can solve a problem more efficiently than a single robot, while also providing robustness and flexibility to the group. Swarm robotics model is a key component of a cooperative algorithm that controls the behaviors and interactions of all individuals. The robots in the swarm should have some basic functions, such as sensing, communicating, and monitoring, and satisfy the following properties

Artificial Intelligence Applications for Drones Navigation in GPS-denied or degraded Environments

L'abstract è presente nell'allegato / the abstract is in the attachmen

Unmanned Vehicle Systems & Operations on Air, Sea, Land

Unmanned Vehicle Systems & Operations On Air, Sea, Land is our fourth textbook in a series covering the world of Unmanned Aircraft Systems (UAS) and Counter Unmanned Aircraft Systems (CUAS). (Nichols R. K., 2018) (Nichols R. K., et al., 2019) (Nichols R. , et al., 2020)The authors have expanded their purview beyond UAS / CUAS systems. Our title shows our concern for growth and unique cyber security unmanned vehicle technology and operations for unmanned vehicles in all theaters: Air, Sea and Land – especially maritime cybersecurity and China proliferation issues. Topics include: Information Advances, Remote ID, and Extreme Persistence ISR; Unmanned Aerial Vehicles & How They Can Augment Mesonet Weather Tower Data Collection; Tour de Drones for the Discerning Palate; Underwater Autonomous Navigation & other UUV Advances; Autonomous Maritime Asymmetric Systems; UUV Integrated Autonomous Missions & Drone Management; Principles of Naval Architecture Applied to UUV’s; Unmanned Logistics Operating Safely and Efficiently Across Multiple Domains; Chinese Advances in Stealth UAV Penetration Path Planning in Combat Environment; UAS, the Fourth Amendment and Privacy; UV & Disinformation / Misinformation Channels; Chinese UAS Proliferation along New Silk Road Sea / Land Routes; Automaton, AI, Law, Ethics, Crossing the Machine – Human Barrier and Maritime Cybersecurity.Unmanned Vehicle Systems are an integral part of the US national critical infrastructure The authors have endeavored to bring a breadth and quality of information to the reader that is unparalleled in the unclassified sphere. Unmanned Vehicle (UV) Systems & Operations On Air, Sea, Land discusses state-of-the-art technology / issues facing U.S. UV system researchers / designers / manufacturers / testers. We trust our newest look at Unmanned Vehicles in Air, Sea, and Land will enrich our students and readers understanding of the purview of this wonderful technology we call UV.https://newprairiepress.org/ebooks/1035/thumbnail.jp

A Framework for Automatic Behavior Generation in Multi-Function Swarms

Multi-function swarms are swarms that solve multiple tasks at once. For example, a quadcopter swarm could be tasked with exploring an area of interest while simultaneously functioning as ad-hoc relays. With this type of multi-function comes the challenge of handling potentially conflicting requirements simultaneously. Using the Quality-Diversity algorithm MAP-elites in combination with a suitable controller structure, a framework for automatic behavior generation in multi-function swarms is proposed. The framework is tested on a scenario with three simultaneous tasks: exploration, communication network creation and geolocation of RF emitters. A repertoire is evolved, consisting of a wide range of controllers, or behavior primitives, with different characteristics and trade-offs in the different tasks. This repertoire would enable the swarm to transition between behavior trade-offs online, according to the situational requirements. Furthermore, the effect of noise on the behavior characteristics in MAP-elites is investigated. A moderate number of re-evaluations is found to increase the robustness while keeping the computational requirements relatively low. A few selected controllers are examined, and the dynamics of transitioning between these controllers are explored. Finally, the study develops a methodology for analyzing the makeup of the resulting controllers. This is done through a parameter variation study where the importance of individual inputs to the swarm controllers is assessed and analyzed