Adaptive and learning-based formation control of swarm robots

Autonomous aerial and wheeled mobile robots play a major role in tasks such as search and rescue, transportation, monitoring, and inspection. However, these operations are faced with a few open challenges including robust autonomy, and adaptive coordination based on the environment and operating conditions, particularly in swarm robots with limited communication and perception capabilities. Furthermore, the computational complexity increases exponentially with the number of robots in the swarm. This thesis examines two different aspects of the formation control problem. On the one hand, we investigate how formation could be performed by swarm robots with limited communication and perception (e.g., Crazyflie nano quadrotor). On the other hand, we explore human-swarm interaction (HSI) and different shared-control mechanisms between human and swarm robots (e.g., BristleBot) for artistic creation. In particular, we combine bio-inspired (i.e., flocking, foraging) techniques with learning-based control strategies (using artificial neural networks) for adaptive control of multi- robots. We first review how learning-based control and networked dynamical systems can be used to assign distributed and decentralized policies to individual robots such that the desired formation emerges from their collective behavior. We proceed by presenting a novel flocking control for UAV swarm using deep reinforcement learning. We formulate the flocking formation problem as a partially observable Markov decision process (POMDP), and consider a leader-follower configuration, where consensus among all UAVs is used to train a shared control policy, and each UAV performs actions based on the local information it collects. In addition, to avoid collision among UAVs and guarantee flocking and navigation, a reward function is added with the global flocking maintenance, mutual reward, and a collision penalty. We adapt deep deterministic policy gradient (DDPG) with centralized training and decentralized execution to obtain the flocking control policy using actor-critic networks and a global state space matrix. In the context of swarm robotics in arts, we investigate how the formation paradigm can serve as an interaction modality for artists to aesthetically utilize swarms. In particular, we explore particle swarm optimization (PSO) and random walk to control the communication between a team of robots with swarming behavior for musical creation

A Survey on Aerial Swarm Robotics

The use of aerial swarms to solve real-world problems has been increasing steadily, accompanied by falling prices and improving performance of communication, sensing, and processing hardware. The commoditization of hardware has reduced unit costs, thereby lowering the barriers to entry to the field of aerial swarm robotics. A key enabling technology for swarms is the family of algorithms that allow the individual members of the swarm to communicate and allocate tasks amongst themselves, plan their trajectories, and coordinate their flight in such a way that the overall objectives of the swarm are achieved efficiently. These algorithms, often organized in a hierarchical fashion, endow the swarm with autonomy at every level, and the role of a human operator can be reduced, in principle, to interactions at a higher level without direct intervention. This technology depends on the clever and innovative application of theoretical tools from control and estimation. This paper reviews the state of the art of these theoretical tools, specifically focusing on how they have been developed for, and applied to, aerial swarms. Aerial swarms differ from swarms of ground-based vehicles in two respects: they operate in a three-dimensional space and the dynamics of individual vehicles adds an extra layer of complexity. We review dynamic modeling and conditions for stability and controllability that are essential in order to achieve cooperative flight and distributed sensing. The main sections of this paper focus on major results covering trajectory generation, task allocation, adversarial control, distributed sensing, monitoring, and mapping. Wherever possible, we indicate how the physics and subsystem technologies of aerial robots are brought to bear on these individual areas

Design of an UAV swarm

This master thesis tries to give an overview on the general aspects involved in the design of an UAV swarm. UAV swarms are continuoulsy gaining popularity amongst researchers and UAV manufacturers, since they allow greater success rates in task accomplishing with reduced times. Appart from this, multiple UAVs cooperating between them opens a new field of missions that can only be carried in this way. All the topics explained within this master thesis will explain all the agents involved in the design of an UAV swarm, from the communication protocols between them, navigation and trajectory analysis and task allocation

3D Formation Control in Multi-Robot Teams Using Artificial Potential Fields

Multi-robot teams find applications in emergency response, search and rescue operations, convoy support and many more. Teams of autonomous aerial vehicles can also be used to protect a cargo of airplanes by surrounding them in some geometric shape. This research develops a control algorithm to attract UAVs to one or a set of bounded geometric shapes while avoiding collisions, re-configuring in the event of departure or addition of UAVs and maneuvering in mission space while retaining the configuration. Using potential field theory, weighted vector fields are described to attract UAVs to a desired formation. In order to achieve this, three vector fields are defined: one attracts UAVs located outside the formation towards bounded geometric shape; one pushes them away from the center towards the desired region and the third controls collision avoidance and dispersion of UAVs within the formation. The result is a control algorithm that is theoretically justified and verified using MATLAB which generates velocity vectors to attract UAVs to a loose formation and maneuver in the mission space while remaining in formation. This approach efficiently scales to different team sizes

Multiple UAV systems: a survey

Nowadays, Unmanned Aerial Vehicles (UAVs) are used in many different applications. Using systems of multiple UAVs is the next obvious step in the process of applying this technology for variety of tasks. There are few research works that cover the applications of these systems and they are all highly specialized. The goal of this survey is to fill this gap by providing a generic review on different applications of multiple UAV systems that have been developed in recent years. We also present a nomenclature and architecture taxonomy for these systems. In the end, a discussion on current trends and challenges is provided.This work was funded by the Ministry of Economy, Industryand Competitiveness of Spain under Grant Nos. TRA2016-77012-R and BES-2017-079798Peer ReviewedPostprint (published version

Sistem Kontrol Swarm untuk Flocking Wahana NR-Awak Quadrotor dengan Optimasi Algoritma Genetik

Quadrotor merupakan wahana udara nir-awak jenis lepas landas atau pendaratan vertikal berbentuk silang dan memiliki sebuah rotor pada setiap ujung lengannya dengan kemampuan manuver yang tinggi. Swarm quadrotor yang terdiri dari sekumpulan quadrotor akan menjadi suatu swarm yang baik, sesuai dengan kriteria swarm oleh Reynold yaitu dapat menghindari tumbukan, menyamakan kecepatan, dan pemusatan swarm. Pengontrolan swarm quadrotor memiliki tingkat kerumitan yang tinggi karena melibatkan banyak agen. Riset pengembangan swarm quadrotor masih belum banyak dilakukan dan masih membuka peluang untuk meneliti dengan metoda lain yang lebih baik dalam menghasilkan swarm. Makalah ini mengusulkan pengontrolan swarm quadrotor yang terdiri dari dua tingkat lup kontrol. Lup pertama adalah pengontrol sistem model swarm untuk membangkitkan lintasan swarm dan lup kedua merupakan pengontrol pada quadrotor untuk melakukan penjejakan lintasan swarm. Pengontrol pertama menggunakan pengontrol proporsional derivatif (PD), sedangkan pengontrol kedua menggunakan regulator linier kuadratik (RLK). Pengontrol yang dirancang memiliki parameter yang banyak, sehingga pemilihan parameter yang optimal sangat sulit. Pencarian parameter optimal pada pengontrol model swarm quadrotor membutuhkan teknik optimasi seperti algoritma genetik (AG) untuk mengarahkan pencarian menuju solusi yang menghasilkan kinerja terbaik. Pada makalah ini, penalaan dengan optimasi AG hanya dilakukan pada pengontrol PD untuk menghasilkan lintasan swarm terbaik, sedangkan matrik bobot RLK dilakukan secara uji coba. Hasil simulasi swarm pada model quadrotor menunjukkan parameter , . , dan  yang diperoleh menggunakan AG menghasilkan pergerakan swarm yang baik dengan kesalahan RMS pelacakan 0,0094 m terhadap fungsi obyektif. Sedangkan ketika parameter ,  dan  dicari menggunakan AG, tidak berpengaruh banyak dalam memperbaiki hasil simulasi swarm quadrotor. AbstractThe quadrotor is a type of take-off or vertical landing unmanned aerial vehicles with a cross shape and has one rotor at each end of its arm with high maneuverability. A quadrotor swarm consisting of a group of quadrotors leads to a good swarm, according to Reynold's swarm criteria, which accomplishes collision avoidance, velocity matching, and flock centering. Quadrotor swarm control has a high level of complexity because it involves many agents. Research on the development of quadrotor swarm has received insignificant attention and it still opens opportunities to research other methods that are better at producing swarm. The paper proposes the control of a quadrotor swarm consisted of two levels of control loops. The first loop controls the swarm model system to generate the swarm trajectory and the second loop is the controller on the quadrotor to track the swarm path. The first controller uses a proportional derivative controller (PD), while the second controller uses the linear quadratic regulator (LQR). The controller that is designed has many parameters, so the optimal parameter selection is very difficult. The search for optimal parameters in the swarm model controller requires optimization techniques such as the genetic algorithm (GA) to direct the search for solutions that produce the best performance. In this paper, tuning with the optimization of GA is only done for the PD controller in order to produce the best swarm trajectory, while the weight matrices of the LQR are done on a trial error basis. Swarm simulation results of a quadrotor model system show the parameters , . , and  obtained using GA produce a good swarm movement with RMS error 0.0094 m of the objective function. Whereas when parameters ,  and  are searched using GA, it does not have much effect in improving the quadrotor swarm simulation results

Energy-Efficient Swarm Behavior for Indoor UAV Ad-Hoc Network Deployment

Building an ad-hoc network in emergency situations can be crucial as a primary tool or even when used prior to subsequent operations. The use of mini and micro Unmanned Aerial Vehicles (UAVs) is increasing because of the wide range of possibilities they offer. Moreover, they have been proven to bring sustainability to many applications, such as agriculture, deforestation and wildlife conservation, among others. Therefore, creating a UAV network for an unknown environment is an important task and an active research field. In this article, a mobility model for the creation of ad-hoc networks using UAVs will be presented. This model will be based on pheromones for robust navigation. We will focus mainly on developing energy-efficient behavior, which is essential for this type of vehicle. Although there are in the literature several models of mobility for ad-hoc network creation, we find that either they are not adapted to the specific energy requirements of UAVs or the proposed motion models are unrealistic or not sufficiently robust for final implantation. We will present and analyze the operation of a distributed swarm behavior able to create an ad-hoc network. Then, an analytical model of the swarm energy consumption will be proposed. This model will provide a mechanism to effectively predict the energy consumption needed for the deployment of the network prior to its implementation. Determining the use of the mobility behavior is a requirement to establish and maintain a communication channel for the required time. Finally, this analytical model will be experimentally validated and compared to the Random Waypoint (RWP) mobility strategy.This work was partially supported by the Ministerio de Economía y Competitividad (Spain), project TIN2013-40982-R, the FEDER funds and the “Red de Investigación en el uso del aprendizaje colaborativo para la adquisición de competencias básicas. El caso Erasmus+ EUROBOTIQUE”, Red ICE3701, curso 2016–2017