Autonomous Unmanned Aerial Vehicle Navigation using Reinforcement Learning: A Systematic Review

There is an increasing demand for using Unmanned Aerial Vehicle (UAV), known as drones, in different applications such as packages delivery, traffic monitoring, search and rescue operations, and military combat engagements. In all of these applications, the UAV is used to navigate the environment autonomously --- without human interaction, perform specific tasks and avoid obstacles. Autonomous UAV navigation is commonly accomplished using Reinforcement Learning (RL), where agents act as experts in a domain to navigate the environment while avoiding obstacles. Understanding the navigation environment and algorithmic limitations plays an essential role in choosing the appropriate RL algorithm to solve the navigation problem effectively. Consequently, this study first identifies the main UAV navigation tasks and discusses navigation frameworks and simulation software. Next, RL algorithms are classified and discussed based on the environment, algorithm characteristics, abilities, and applications in different UAV navigation problems, which will help the practitioners and researchers select the appropriate RL algorithms for their UAV navigation use cases. Moreover, identified gaps and opportunities will drive UAV navigation research

A Survey on Aerial Swarm Robotics

The use of aerial swarms to solve real-world problems has been increasing steadily, accompanied by falling prices and improving performance of communication, sensing, and processing hardware. The commoditization of hardware has reduced unit costs, thereby lowering the barriers to entry to the field of aerial swarm robotics. A key enabling technology for swarms is the family of algorithms that allow the individual members of the swarm to communicate and allocate tasks amongst themselves, plan their trajectories, and coordinate their flight in such a way that the overall objectives of the swarm are achieved efficiently. These algorithms, often organized in a hierarchical fashion, endow the swarm with autonomy at every level, and the role of a human operator can be reduced, in principle, to interactions at a higher level without direct intervention. This technology depends on the clever and innovative application of theoretical tools from control and estimation. This paper reviews the state of the art of these theoretical tools, specifically focusing on how they have been developed for, and applied to, aerial swarms. Aerial swarms differ from swarms of ground-based vehicles in two respects: they operate in a three-dimensional space and the dynamics of individual vehicles adds an extra layer of complexity. We review dynamic modeling and conditions for stability and controllability that are essential in order to achieve cooperative flight and distributed sensing. The main sections of this paper focus on major results covering trajectory generation, task allocation, adversarial control, distributed sensing, monitoring, and mapping. Wherever possible, we indicate how the physics and subsystem technologies of aerial robots are brought to bear on these individual areas

Development and Validation of a LQR-Based Quadcopter Control Dynamics Simulation Model

5The growing applications involving unmanned aerial vehicles (UAVs) are requiring more advanced control algorithms to improve rotary-wing UAVs’ performance. To preliminarily tune such advanced controllers, an experimental approach could take a long time and also be dangerous for the vehicle and the onboard hardware components. In this paper, a simulation model of a quadcopter is developed and validated by the comparison of simulation results and experimental data collected during flight tests. For this purpose, an open-source flight controller for quadcopter UAVs is developed and a linear quadratic regulator (LQR) controller is implemented as the control strategy. The input physical quantities are experimentally measured; hence, the LQR controller parameters are tuned on the simulation model. The same tuning is proposed on the developed flight controller with satisfactory results. Finally, flight data and simulation results are compared showing a reliable approximation of the experimental data by the model. Because numerous state-of-the-art simulation models are available, but accurately validated ones are not easy to find, the main purpose of this work is to provide a reliable tool to evaluate the performance for this UAV configuration. DOI: 10.1061/(ASCE)AS.1943-5525.0001336. © 2021 American Society of Civil Engineers.partially_openopenAlessandro Minervini; Simone Godio; Giorgio Guglieri; Fabio Dovis; Alfredo BiciMinervini, Alessandro; Godio, Simone; Guglieri, Giorgio; Dovis, Fabio; Bici, Alfred

Swarming of Unmanned Aerial Vehicles Using Indirect Information Exchange by Observation of the Workspace

Tato práce se soustředí na návrh, implementaci a ověření řídícího systému a systému pro relativní lokalizaci roje bezpilotních autonomních helikoptér v lesním prostředí. Základem lokalizačního systému je ICP algoritmus. Rojový řídící systém je inspirován Boidy a modifikován pro lepší interakci s reálným prostředím. Implementace byla ověřena v realistickém simulátoru Gazebo a pomocí Matlabu. Přístup, který je uveden v této práci, byl následně porovnán se současným systémem pro relativní lokalizaci a navigaci v lese, které používá skupina Multirobotických systémů na ČVUT v Praze.This thesis focuses on the design, implementation, and verification of a control system and relative localization approach for a swarm consisting of unmanned aerial vehicles in a forest environment. The core of the localization system is the ICP algorithm. The control system is based on Boids with modifications to adapt to the forest environment better. Implementation was verified in the realistic Gazebo simulator as well as in Matlab. The approach introduced in this thesis was also compared with the existing system for relative localization and navigation used in the Multi-Robot Systems group at Czech Technical University in Prague

A novel extended potential field controller for use on aerial robots

© 2016 IEEE. Unmanned Aerial Vehicles (UAV), commonly known as drones, have many potential uses in real world applications. Drones require advanced planning and navigation algorithms to enable them to safely move through and interact with the world around them. This paper presents an extended potential field controller (ePFC) which enables an aerial robot, or drone, to safely track a dynamic target location while simultaneously avoiding any obstacles in its path. The ePFC outperforms a traditional potential field controller (PFC) with smoother tracking paths and shorter settling times. The proposed ePFC's stability is evaluated by Lyapunov approach, and its performance is simulated in a Matlab environment. Finally, the controller is implemented on an experimental platform in a laboratory environment which demonstrates the effectiveness of the controller

Comparative Study of Indoor Navigation Systems for Autonomous Flight

Recently, Unmanned Aerial Vehicles (UAVs) have attracted the society and researchers due to the capability to perform in economic, scientific and emergency scenarios, and are being employed in large number of applications especially during the hostile environments. They can operate autonomously for both indoor and outdoor applications mainly including search and rescue, manufacturing, forest fire tracking, remote sensing etc. For both environments, precise localization plays a critical role in order to achieve high performance flight and interacting with the surrounding objects. However, for indoor areas with degraded or denied Global Navigation Satellite System (GNSS) situation, it becomes challenging to control UAV autonomously especially where obstacles are unidentified. A large number of techniques by using various technologies are proposed to get rid of these limits. This paper provides a comparison of such existing solutions and technologies available for this purpose with their strengths and limitations. Further, a summary of current research status with unresolved issues and opportunities is provided that would provide research directions to the researchers of the similar interests

A Survey on Passing-through Control of Multi-Robot Systems in Cluttered Environments

This survey presents a comprehensive review of various methods and algorithms related to passing-through control of multi-robot systems in cluttered environments. Numerous studies have investigated this area, and we identify several avenues for enhancing existing methods. This survey describes some models of robots and commonly considered control objectives, followed by an in-depth analysis of four types of algorithms that can be employed for passing-through control: leader-follower formation control, multi-robot trajectory planning, control-based methods, and virtual tube planning and control. Furthermore, we conduct a comparative analysis of these techniques and provide some subjective and general evaluations.Comment: 18 pages, 19 figure