Saliency-based cooperative landing of a multirotor aerial vehicle on an autonomous surface vehicle

This paper presents a method for vision-based landing of a multirotor unmanned aerial vehicle (UAV) on an autonomous surface vehicle (ASV) equipped with a helipad. The method includes a mechanism for helipad behavioural search when outside the UAV’s field of view, a learning saliency-based mechanism for visual tracking the helipad, and a cooperative strategy for the final vision-based landing phase. Learning how to track the helipad from above occurs during takeoff and cooperation results from having the ASV tracking the UAV for assisting its landing. A set of experimental results with both simulated and physical robots show the feasibility of the presented method.info:eu-repo/semantics/acceptedVersio

A cooperative multi-robot team for the surveillance of shipwreck survivors at sea

The sea as a very extensive area, renders difficult a pre-emptive and long-lasting search for shipwreck survivors. The operational cost for deploying manned teams with such proactive strategy is high and, thus, these teams are only reactively deployed when a disaster like a shipwreck has been communicated. To reduce the involved financial costs, unmanned robotic systems could be used instead as background surveillance teams patrolling the seas. In this sense, a robotic team for search and rescue (SAR) operations at sea is presented in this work. Composed of an Unmanned Surface Vehicle (USV) piggybacking a watertight Unmanned Aerial Vehicle (UAV) with vertical take-off and landing capabilities, the proposed cooperative system is capable of search, track and provide basic life support while reporting the position of human survivors to better prepared manned rescue teams. The USV provides long-range transportation of the UAV and basic survival kits for victims. The UAV assures an augmented perception of the environment due to its high vantage point.info:eu-repo/semantics/acceptedVersio

A Cooperative Approach for Autonomous Landing of UAVs

This dissertation presents a cooperative approach for the autonomous landing of MRVTOL UAVs (Multi Rotor-Vertical Take-off and Landing Unmanned Aerial Vehicles). Most standard UAV autonomous landing systems take an approach, where the UAV detects a pre-set pattern on the landing zone, establishes relative positions and uses them to perform the landing. These methods present some drawbacks such as all of the processing being performed by the UAV itself, requiring high computational power from it. An additional problem arises from the fact most of these methods are only reliable when the UAV is already at relatively low altitudes since the pattern’s features have to be clearly visible from the UAV’s camera. The method presented throughout this dissertation relies on an RGB camera, placed in the landing zone pointing upwards towards the sky. Due to the fact, the sky is a fairly stagnant and uniform environment the unique motion patterns the UAV displays can be singled out and analysed using Background Subtraction and Optical Flow techniques. A terrestrial or surface robotic system can then analyse the images in real-time and relay commands to the UAV. The result is a model-free method, i.e independent of the UAV’s morphological aspect or pre-determined patterns, capable of aiding the UAV during the landing manoeuvre. The approach is reliable enough to be used as a stand-alone method, or be used along traditional methods achieving a more robust system. Experimental results obtained from a dataset encompassing 23 diverse videos showed the ability of the computer vision algorithm to perform the detection of the UAV in 93,44% of the 44557 evaluated frames with a tracking error of 6.6%. A high-level control system that employs the concept of an approach zone to the helipad was also developed. Within the zone every possible three-dimensional position corresponds to a velocity command for the UAV, with a given orientation and magnitude. The control system was tested in a simulated environment and it proved to be effective in performing the landing of the UAV within 13 cm from the goal

A Survey of Offline and Online Learning-Based Algorithms for Multirotor UAVs

Multirotor UAVs are used for a wide spectrum of civilian and public domain applications. Navigation controllers endowed with different attributes and onboard sensor suites enable multirotor autonomous or semi-autonomous, safe flight, operation, and functionality under nominal and detrimental conditions and external disturbances, even when flying in uncertain and dynamically changing environments. During the last decade, given the faster-than-exponential increase of available computational power, different learning-based algorithms have been derived, implemented, and tested to navigate and control, among other systems, multirotor UAVs. Learning algorithms have been, and are used to derive data-driven based models, to identify parameters, to track objects, to develop navigation controllers, and to learn the environment in which multirotors operate. Learning algorithms combined with model-based control techniques have been proven beneficial when applied to multirotors. This survey summarizes published research since 2015, dividing algorithms, techniques, and methodologies into offline and online learning categories, and then, further classifying them into machine learning, deep learning, and reinforcement learning sub-categories. An integral part and focus of this survey are on online learning algorithms as applied to multirotors with the aim to register the type of learning techniques that are either hard or almost hard real-time implementable, as well as to understand what information is learned, why, and how, and how fast. The outcome of the survey offers a clear understanding of the recent state-of-the-art and of the type and kind of learning-based algorithms that may be implemented, tested, and executed in real-time.Comment: 26 pages, 6 figures, 4 tables, Survey Pape

confined spaces industrial inspection with micro aerial vehicles and laser range finder localization

This work addresses the problem of semi-automatic inspection and navigation in confined environments. A system that overcomes many challenges at the state of the art is presented. It comprises a mu..

Guidance, navigation and control of multirotors

Aplicat embargament des de la data de defensa fins el dia 31 de desembre de 2021This thesis presents contributions to the Guidance, Navigation and Control (GNC) systems for multirotor vehicles by applying and developing diverse control techniques and machine learning theory with innovative results. The aim of the thesis is to obtain a GNC system able to make the vehicle follow predefined paths while avoiding obstacles in the vehicle's route. The system must be adaptable to different paths, situations and missions, reducing the tuning effort and parametrisation of the proposed approaches. The multirotor platform, formed by the Asctec Hummingbird quadrotor vehicle, is studied and described in detail. A complete mathematical model is obtained and a freely available and open simulation platform is built. Furthermore, an autopilot controller is designed and implemented in the real platform. The control part is focused on the path following problem. That is, following a predefined path in space without any time constraint. Diverse control-oriented and geometrical algorithms are studied, implemented and compared. Then, the geometrical algorithms are improved by obtaining adaptive approaches that do not need any parameter tuning. The adaptive geometrical approaches are developed by means of Neural Networks. To end up, a deep reinforcement learning approach is developed to solve the path following problem. This approach implements the Deep Deterministic Policy Gradient algorithm. The resulting approach is trained in a realistic multirotor simulator and tested in real experiments with success. The proposed approach is able to accurately follow a path while adapting the vehicle's velocity depending on the path's shape. In the navigation part, an obstacle detection system based on the use of a LIDAR sensor is implemented. A model of the sensor is derived and included in the simulator. Moreover, an approach for treating the sensor data to eliminate the possible ground detections is developed. The guidance part is focused on the reactive path planning problem. That is, a path planning algorithm that is able to re-plan the trajectory online if an unexpected event, such as detecting an obstacle in the vehicle's route, occurs. A deep reinforcement learning approach for the reactive obstacle avoidance problem is developed. This approach implements the Deep Deterministic Policy Gradient algorithm. The developed deep reinforcement learning agent is trained and tested in the realistic simulation platform. This agent is combined with the path following agent and the rest of the elements developed in the thesis obtaining a GNC system that is able to follow different types of paths while avoiding obstacle in the vehicle's route.Aquesta tesi doctoral presenta diverses contribucions relaciones amb els sistemes de Guiat, Navegació i Control (GNC) per a vehicles multirrotor, aplicant i desenvolupant diverses tècniques de control i de machine learning amb resultats innovadors. L'objectiu principal de la tesi és obtenir un sistema de GNC capaç de dirigir el vehicle perquè segueixi una trajectòria predefinida mentre evita els obstacles que puguin aparèixer en el recorregut del vehicle. El sistema ha de ser adaptable a diferents trajectòries, situacions i missions, reduint l'esforç realitzat en l'ajust i la parametrització dels mètodes proposats. La plataforma experimental, formada pel cuadricòpter Asctec Hummingbird, s'estudia i es descriu en detall. S'obté un model matemàtic complet de la plataforma i es desenvolupa una eina de simulació, la qual és de codi lliure. A més, es dissenya un controlador autopilot i s'implementa en la plataforma real. La part de control està enfocada al problema de path following. En aquest problema, el vehicle ha de seguir una trajectòria predefinida en l'espai sense cap tipus de restricció temporal. S'estudien, s'implementen i es comparen diversos algoritmes de control i geomètrics de path following. Després, es milloren els algoritmes geomètrics usant xarxes neuronals per convertirlos en algoritmes adaptatius. Per finalitzar, es desenvolupa un mètode de path following basat en tècniques d'aprenentatge per reforç profund (deep Reinforcement learning). Aquest mètode implementa l'algoritme Deep Deterministic Policy Gradient. L'agent intel. ligent resultant és entrenat en un simulador realista de multirotors i validat en la plataforma experimental real amb èxit. Els resultats mostren que l'agent és capaç de seguir de forma precisa la trajectòria de referència adaptant la velocitat del vehicle segons la curvatura del recorregut. A la part de navegació, s'implementa un sistema de detecció d'obstacles basat en l'ús d'un sensor LIDAR. Es deriva un model del sensor i aquest s'inclou en el simulador. A més, es desenvolupa un mètode per tractar les mesures del sensor per eliminar les possibles deteccions del terra. Pel que fa a la part de guiatge, aquesta està focalitzada en el problema de reactive path planning. És a dir, un algoritme de planificació de trajectòria que és capaç de re-planejar el recorregut del vehicle a l'instant si algun esdeveniment inesperat ocorre, com ho és la detecció d'un obstacle en el recorregut del vehicle. Es desenvolupa un mètode basat en aprenentatge per reforç profund per l'evasió d'obstacles. Aquest mètode implementa l'algoritme Deep Deterministic Policy Gradient. L'agent d'aprenentatge per reforç s'entrena i valida en un simulador de multirotors realista. Aquest agent es combina amb l'agent de path following i la resta d'elements desenvolupats en la tesi per obtenir un sistema GNC capaç de seguir diferents tipus de trajectòries, evadint els obstacles que estiguin en el recorregut del vehicle.Esta tesis doctoral presenta varias contribuciones relacionas con los sistemas de Guiado, Navegación y Control (GNC) para vehículos multirotor, aplicando y desarrollando diversas técnicas de control y de machine learning con resultados innovadores. El objetivo principal de la tesis es obtener un sistema de GNC capaz de dirigir el vehículo para que siga una trayectoria predefinida mientras evita los obstáculos que puedan aparecer en el recorrido del vehículo. El sistema debe ser adaptable a diferentes trayectorias, situaciones y misiones, reduciendo el esfuerzo realizado en el ajuste y la parametrización de los métodos propuestos. La plataforma experimental, formada por el cuadricoptero Asctec Hummingbird, se estudia y describe en detalle. Se obtiene un modelo matemático completo de la plataforma y se desarrolla una herramienta de simulación, la cual es de código libre. Además, se diseña un controlador autopilot, el cual es implementado en la plataforma real. La parte de control está enfocada en el problema de path following. En este problema, el vehículo debe seguir una trayectoria predefinida en el espacio tridimensional sin ninguna restricción temporal Se estudian, implementan y comparan varios algoritmos de control y geométricos de path following. Luego, se mejoran los algoritmos geométricos usando redes neuronales para convertirlos en algoritmos adaptativos. Para finalizar, se desarrolla un método de path following basado en técnicas de aprendizaje por refuerzo profundo (deep reinforcement learning). Este método implementa el algoritmo Deep Deterministic Policy Gradient. El agente inteligente resultante es entrenado en un simulador realista de multirotores y validado en la plataforma experimental real con éxito. Los resultados muestran que el agente es capaz de seguir de forma precisa la trayectoria de referencia adaptando la velocidad del vehículo según la curvatura del recorrido. En la parte de navegación se implementa un sistema de detección de obstáculos basado en el uso de un sensor LIDAR. Se deriva un modelo del sensor y este se incluye en el simulador. Además, se desarrolla un método para tratar las medidas del sensor para eliminar las posibles detecciones del suelo. En cuanto a la parte de guiado, está focalizada en el problema de reactive path planning. Es decir, un algoritmo de planificación de trayectoria que es capaz de re-planear el recorrido del vehículo al instante si ocurre algún evento inesperado, como lo es la detección de un obstáculo en el recorrido del vehículo. Se desarrolla un método basado en aprendizaje por refuerzo profundo para la evasión de obstáculos. Este implementa el algoritmo Deep Deterministic Policy Gradient. El agente de aprendizaje por refuerzo se entrena y valida en un simulador de multirotors realista. Este agente se combina con el agente de path following y el resto de elementos desarrollados en la tesis para obtener un sistema GNC capaz de seguir diferentes tipos de trayectorias evadiendo los obstáculos que estén en el recorrido del vehículo.Postprint (published version

A Review on IoT Deep Learning UAV Systems for Autonomous Obstacle Detection and Collision Avoidance

[Abstract] Advances in Unmanned Aerial Vehicles (UAVs), also known as drones, offer unprecedented opportunities to boost a wide array of large-scale Internet of Things (IoT) applications. Nevertheless, UAV platforms still face important limitations mainly related to autonomy and weight that impact their remote sensing capabilities when capturing and processing the data required for developing autonomous and robust real-time obstacle detection and avoidance systems. In this regard, Deep Learning (DL) techniques have arisen as a promising alternative for improving real-time obstacle detection and collision avoidance for highly autonomous UAVs. This article reviews the most recent developments on DL Unmanned Aerial Systems (UASs) and provides a detailed explanation on the main DL techniques. Moreover, the latest DL-UAV communication architectures are studied and their most common hardware is analyzed. Furthermore, this article enumerates the most relevant open challenges for current DL-UAV solutions, thus allowing future researchers to define a roadmap for devising the new generation affordable autonomous DL-UAV IoT solutions.Xunta de Galicia; ED431C 2016-045Xunta de Galicia; ED431C 2016-047Xunta de Galicia; , ED431G/01Centro Singular de Investigación de Galicia; PC18/01Agencia Estatal de Investigación de España; TEC2016-75067-C4-1-

Aterragem de robôs aéreos multi-rotor em plataformas móveis: uma solução baseada em comportamentos e em saliência visual

Dissertação para obtenção do Grau de Mestre em Engenharia Eletrotécnica e de ComputadoresEsta dissertação apresenta um sistema para aterragem de veículos aéreos multi-rotor em plataformas móveis utilizando um sensor de visão monocular e GPS. O sistema aqui apresentado visa colmatar algumas das limitações dos sistemas atuais, tais como a dependência na existência de uma marca conhecida a identificar a plataforma de aterragem, cuja deteção é robusta apenas a baixas altitudes. Através da utilização de um modelo computacional de saliência visual aliado a um detetor de uma marca específica, esta dissertação apresenta uma solução robusta a um envelope mais alargado de altitudes. Por forma a permitir estender a pesquisa para além do campo visual instantâneo do veículo aéreo, o processo de aterragem é gerido por uma arquitetura comportamental capaz de controlar o posicionamento do veículo de acordo com estimativas da posição mais provável da plataforma, no caso da existência de comunicações diretas entre o veículo e a plataforma de aterragem. O caso de estudo utilizado para validar o sistema é composto por um veículo aéreo com quatro rotores e por uma plataforma de aterragem presente num veículo marítimo. Testes nos veículos físicos e simulados demonstram a capacidade do sistema em gerir a pesquisa e aterragem do veículo aéreo

HoverSea: Inspecção de estruturas marítimas utilizando um drone e um catamarã autónomo

Este projecto tem como objectivo a realização de missões autónomas utilizando um drone e um catamarã. O drone deverá levantar voo de uma plataforma integrante do catamarã e posteriormente deverá deslocar-se até coordenadas GPS determinadas previamente. Por fim, deverá regressar até ao catamarã reconhecer a plataforma e aterrar em segurança

Design of a Specialized UAV Platform for the Discharge of a Fire Extinguishing Capsule

Tato práce se zabývá návrhem systému specializovaného pro autonomní detekci a lokalizaci požárů z palubních senzorů bezpilotních helikoptér. Hašení požárů je zajištěno automatickým vystřelením ampule s hasící kapalinou do zdroje požáru z palubního vystřelovače. Hlavní část této práce se soustředí na detekci požárů v datech termální kamery a jejich následnou lokalizaci ve světě za pomoci palubní hloubkové kamery. Bezpilotní helikoptéra je poté optimálně navigována na pozici pro zajištění průletu ampule s hasící kapalinou do zdroje požáru. Vyvinuté metody jsou detailně analyzovány a jejich chování je testováno jak v simulaci, tak současně i při reálných experimentech. Kvalitativní a kvantitativní analýza ukazuje na použitelnost a robustnost celého systému.This thesis deals with the design of an unmanned multirotor aircraft system specialized for autonomous detection and localization of fires from onboard sensors, and the task of fast and effective fire extinguishment. The main part of this thesis focuses on the detection of fires in thermal images and their localization in the world using an onboard depth camera. The localized fires are used to optimally position the unmanned aircraft in order to effectively discharge an ampoule filled with a fire extinguishant from an onboard launcher. The developed methods are analyzed in detail and their performance is evaluated in simulation scenarios as well as in real-world experiments. The included quantitative and qualitative analysis verifies the feasibility and robustness of the system