Drone deep reinforcement learning: A review

Unmanned Aerial Vehicles (UAVs) are increasingly being used in many challenging and diversified applications. These applications belong to the civilian and the military fields. To name a few; infrastructure inspection, traffic patrolling, remote sensing, mapping, surveillance, rescuing humans and animals, environment monitoring, and Intelligence, Surveillance, Target Acquisition, and Reconnaissance (ISTAR) operations. However, the use of UAVs in these applications needs a substantial level of autonomy. In other words, UAVs should have the ability to accomplish planned missions in unexpected situations without requiring human intervention. To ensure this level of autonomy, many artificial intelligence algorithms were designed. These algorithms targeted the guidance, navigation, and control (GNC) of UAVs. In this paper, we described the state of the art of one subset of these algorithms: the deep reinforcement learning (DRL) techniques. We made a detailed description of them, and we deduced the current limitations in this area. We noted that most of these DRL methods were designed to ensure stable and smooth UAV navigation by training computer-simulated environments. We realized that further research efforts are needed to address the challenges that restrain their deployment in real-life scenarios

Rapid prototyping flight test environment for autonomous unmanned aerial vehicles

Test facility is essential for most engineering research activities, from modelling and identification to verification of algorithms/methods and final demonstration. It is well known that flight tests for aerospace vehicles are expensive and quite risky. To overcome this, this paper describes a rapid prototyping platform for autonomous unmanned aerial vehicles (UAV) developed at Loughborough University, where a number of unmanned aerial and ground vehicles can perform various flight and other missions under computer control. Flexibility, maintainability and low expenses are assured by a proper choice of vehicles, sensors and system architecture. Among many other technical challenges, precision navigation of the unmanned vehicles and system integrations of commercial-off-the-shelf components from different vendors with different operational environments are discussed in detail. Matlab/Simulink based software development environment provides a seamless rapid prototyping platform from concept and theoretic developments to numerical simulation and finally flight tests. Finally, two scenarios performed by this test facility are presented to illustrate its capability

Design of miniaturized sensors for a mission-oriented uav application: A new pathway for early warning

In recent decades, the increasing threats associated with Chemical and Radiological (CR) agents prompted the development of new tools to detect and collect samples without putting in danger first responders inside contaminated areas. A particularly promising branch of these technological developments relates to the integration of different detectors and sampling systems with Unmanned Aerial Vehicles (UAV). The adoption of this equipment may bring significant benefits for both military and civilian implementations. For instance, instrumented UAVs could be used in support of specialist military teams such as Sampling and Identification of Biological, Chemical and Radiological Agents (SIBCRA) team, tasked to perform sampling in contaminated areas, detecting the presence of CR substances in field and then confirming, collecting and evaluating the effective threats. Furthermore, instrumented UAVs may find dual-use application in the civil world in support of emergency teams during industrial accidents and in the monitoring activities of critical infrastructures. Small size drones equipped with different instruments for detection and collection of samples may enable, indeed, several applications, becoming a tool versatile and easy to use in different fields, and even featuring equipment normally utilized in manual operation. The authors hereby present the design of miniaturized sensors for a mission-oriented UAV application and the preliminary results from an experimental campaign performed in 2020

Design of Miniaturized Sensors for a Mission-Oriented UAV Application: A New Pathway for Early Warning

In recent decades, the increasing threats associated with Chemical and Radiological (CR) agents prompted the development of new tools to detect and collect samples without putting in danger first responders inside contaminated areas. A particularly promising branch of these technological developments relates to the integration of different detectors and sampling systems with Unmanned Aerial Vehicles (UAV). The adoption of this equipment may bring significant benefits for both military and civilian implementations. For instance, instrumented UAVs could be used in support of specialist military teams such as Sampling and Identification of Biological, Chemical and Radiological Agents (SIBCRA) team, tasked to perform sampling in contaminated areas, detecting the presence of CR substances in field and then confirming, collecting and evaluating the effective threats. Furthermore, instrumented UAVs may find dual-use application in the civil world in support of emergency teams during industrial accidents and in the monitoring activities of critical infrastructures. Small size drones equipped with different instruments for detection and collection of samples may enable, indeed, several applications, becoming a tool versatile and easy to use in different fields, and even featuring equipment normally utilized in manual operation. The authors hereby present the design of miniaturized sensors for a mission-oriented UAV application and the preliminary results from an experimental campaign performed in 2020

System Architectures for Cooperative Teams of Unmanned Aerial Vehicles Interacting Physically with the Environment

Unmanned Aerial Vehicles (UAVs) have become quite a useful tool for a wide range of applications, from inspection & maintenance to search & rescue, among others. The capabilities of a single UAV can be extended or complemented by the deployment of more UAVs, so multi-UAV cooperative teams are becoming a trend. In that case, as di erent autopilots, heterogeneous platforms, and application-dependent software components have to be integrated, multi-UAV system architectures that are fexible and can adapt to the team's needs are required. In this thesis, we develop system architectures for cooperative teams of UAVs, paying special attention to applications that require physical interaction with the environment, which is typically unstructured. First, we implement some layers to abstract the high-level components from the hardware speci cs. Then we propose increasingly advanced architectures, from a single-UAV hierarchical navigation architecture to an architecture for a cooperative team of heterogeneous UAVs. All this work has been thoroughly tested in both simulation and eld experiments in di erent challenging scenarios through research projects and robotics competitions. Most of the applications required physical interaction with the environment, mainly in unstructured outdoors scenarios. All the know-how and lessons learned throughout the process are shared in this thesis, and all relevant code is publicly available.Los vehículos aéreos no tripulados (UAVs, del inglés Unmanned Aerial Vehicles) se han convertido en herramientas muy valiosas para un amplio espectro de aplicaciones, como inspección y mantenimiento, u operaciones de rescate, entre otras. Las capacidades de un único UAV pueden verse extendidas o complementadas al utilizar varios de estos vehículos simultáneamente, por lo que la tendencia actual es el uso de equipos cooperativos con múltiples UAVs. Para ello, es fundamental la integración de diferentes autopilotos, plataformas heterogéneas, y componentes software -que dependen de la aplicación-, por lo que se requieren arquitecturas multi-UAV que sean flexibles y adaptables a las necesidades del equipo. En esta tesis, se desarrollan arquitecturas para equipos cooperativos de UAVs, prestando una especial atención a aplicaciones que requieran de interacción física con el entorno, cuya naturaleza es típicamente no estructurada. Primero se proponen capas para abstraer a los componentes de alto nivel de las particularidades del hardware. Luego se desarrollan arquitecturas cada vez más avanzadas, desde una arquitectura de navegación para un único UAV, hasta una para un equipo cooperativo de UAVs heterogéneos. Todo el trabajo ha sido minuciosamente probado, tanto en simulación como en experimentos reales, en diferentes y complejos escenarios motivados por proyectos de investigación y competiciones de robótica. En la mayoría de las aplicaciones se requería de interacción física con el entorno, que es normalmente un escenario en exteriores no estructurado. A lo largo de la tesis, se comparten todo el conocimiento adquirido y las lecciones aprendidas en el proceso, y el código relevante está publicado como open-source

MRS Drone: A Modular Platform for Real-World Deployment of Aerial Multi-Robot Systems

This paper presents a modular autonomous Unmanned Aerial Vehicle (UAV) platform called the Multi-robot Systems (MRS) Drone that can be used in a large range of indoor and outdoor applications. The MRS Drone features unique modularity with respect to changes in actuators, frames, and sensory configuration. As the name suggests, the platform is specially tailored for deployment within a MRS group. The MRS Drone contributes to the state-of-the-art of UAV platforms by allowing smooth real-world deployment of multiple aerial robots, as well as by outperforming other platforms with its modularity. For real-world multi-robot deployment in various applications, the platform is easy to both assemble and modify. Moreover, it is accompanied by a realistic simulator to enable safe pre-flight testing and a smooth transition to complex real-world experiments. In this manuscript, we present mechanical and electrical designs, software architecture, and technical specifications to build a fully autonomous multi UAV system. Finally, we demonstrate the full capabilities and the unique modularity of the MRS Drone in various real-world applications that required a diverse range of platform configurations.Comment: 49 pages, 39 figures, accepted for publication to the Journal of Intelligent & Robotic System