Automatic landing on aircraft carrier by visual servoing

International audienceThe landing on carrier is a very difficult task even for trained pilots. This paper presents a method to land automatically using aircraft sensors and three visual features inspired by visual cues used by pilots. These features whose link with the aircraft state is established, are introduced in a control scheme using a linearized aircraft model. The control law demonstrates a large convergence domain using simulated visual features and a 3D tracker applied on synthetic images

Design and integration of vision based sensors for unmanned aerial vehicles navigation and guidance

In this paper we present a novel Navigation and Guidance System (NGS) for Unmanned Aerial Vehicles (UAVs) based on Vision Based Navigation (VBN) and other avionics sensors. The main objective of our research is to design a lowcost and low-weight/volume NGS capable of providing the required level of performance in all flight phases of modern small- to medium-size UAVs, with a special focus on automated precision approach and landing, where VBN techniques can be fully exploited in a multisensory integrated architecture. Various existing techniques for VBN are compared and the Appearance-based Navigation (ABN) approach is selected for implementation

Survey of computer vision algorithms and applications for unmanned aerial vehicles

This paper presents a complete review of computer vision algorithms and vision-based intelligent applications, that are developed in the field of the Unmanned Aerial Vehicles (UAVs) in the latest decade. During this time, the evolution of relevant technologies for UAVs; such as component miniaturization, the increase of computational capabilities, and the evolution of computer vision techniques have allowed an important advance in the development of UAVs technologies and applications. Particularly, computer vision technologies integrated in UAVs allow to develop cutting-edge technologies to cope with aerial perception difficulties; such as visual navigation algorithms, obstacle detection and avoidance and aerial decision-making. All these expert technologies have developed a wide spectrum of application for UAVs, beyond the classic military and defense purposes. Unmanned Aerial Vehicles and Computer Vision are common topics in expert systems, so thanks to the recent advances in perception technologies, modern intelligent applications are developed to enhance autonomous UAV positioning, or automatic algorithms to avoid aerial collisions, among others. Then, the presented survey is based on artificial perception applications that represent important advances in the latest years in the expert system field related to the Unmanned Aerial Vehicles. In this paper, the most significant advances in this field are presented, able to solve fundamental technical limitations; such as visual odometry, obstacle detection, mapping and localization, et cetera. Besides, they have been analyzed based on their capabilities and potential utility. Moreover, the applications and UAVs are divided and categorized according to different criteria.This research is supported by the Spanish Government through the CICYT projects (TRA2015-63708-R and TRA2013-48314-C3-1-R)

Robust hovering controller for uncertain multirotor micro aerial vehicles (MAVS) in gps-denied environments: IMAGE-BASED

This paper proposes an image-based robust hovering controller for multirotor micro aerial vehicles (MAVs) in GPS-denied environments. The proposed controller is robust against the effects of multiple uncertainties in angular dynamics of vehicle which contain external disturbances, nonlinear dynamics, coupling, and parametric uncertainties. Based on visual features extracted from the image, the proposed controller is capable of controlling the pose (position and orientation) of the multirotor relative to the fixed-target. The proposed controller scheme consists of two parts: a spherical image-based visual servoing (IBVS) and a robust flight controller for velocity and attitude control loops. A robust compensator based on a second order robust filter is utilized in the robust flight control design to improve the robustness of the multirotor when subject to multiple uncertainties. Compared to other methods, the proposed method is robust against multiple uncertainties and does not need to keep the features in the field of view. The simulation results prove the effectiveness and robustness of the proposed controller

Low-cost navigation and guidance systems for unmanned aerial vehicles - part 1: Vision-based and integrated sensors

In this paper we present a new low-cost navigation system designed for small size Unmanned Aerial Vehicles (UAVs) based on Vision-Based Navigation (VBN) and other avionics sensors. The main objective of our research was to design a compact, light and relatively inexpensive system capable of providing the Required Navigation Performance (RNP) in all phases of flight of a small UAV, with a special focus on precision approach and landing, where Vision Based Navigation (VBN) techniques can be fully exploited in a multisensor integrated architecture. Various existing techniques for VBN were compared and the Appearance-Based Approach (ABA) was selected for implementation. Feature extraction and optical flow techniques were employed to estimate flight parameters such as roll angle, pitch angle, deviation from the runway and body rates. Additionally, we addressed the possible synergies between VBN, Global Navigation Satellite System (GNSS) and MEMS-IMU (Micro-Electromechanical System Inertial Measurement Unit) sensors, as well as the aiding from Aircraft Dynamics Models (ADMs)

Visual Servoing Approach for Autonomous UAV Landing on a Moving Vehicle

We present a method to autonomously land an Unmanned Aerial Vehicle on a moving vehicle with a circular (or elliptical) pattern on the top. A visual servoing controller approaches the ground vehicle using velocity commands calculated directly in image space. The control laws generate velocity commands in all three dimensions, eliminating the need for a separate height controller. The method has shown the ability to approach and land on the moving deck in simulation, indoor and outdoor environments, and compared to the other available methods, it has provided the fastest landing approach. It does not rely on additional external setup, such as RTK, motion capture system, ground station, offboard processing, or communication with the vehicle, and it requires only a minimal set of hardware and localization sensors. The videos and source codes can be accessed from http://theairlab.org/landing-on-vehicle.Comment: 24 page

Low-cost vision sensors and integrated systems for unmanned aerial vehicle navigation

A novel low cost navigation system based on Vision Based Navigation (VBN) and other avionics sensors is presented, which is designed for small size Unmanned Aerial Vehicle (UAV) applications. The main objective of our research is to design a compact, light and relatively inexpensive system capable of providing the required navigation performance in all phases of flight of a small UAV, with a special focus on precision approach and landing, where Vision Based Navigation (VBN) techniques can be fully exploited in a multisensory integrated architecture. Various existing techniques for VBN are compared and the Appearance-based Navigation (ABN) approach is selected for implementation. Feature extraction and optical flow techniques are employed to estimate flight parameters such as roll angle, pitch angle, deviation from the runway and body rates. Additionally, we address the possible synergies between VBN, Global Navigation Satellite System (GNSS) and MEMS-IMU (Micro-Electromechanical System Inertial Measurement Unit) sensors and also the use of Aircraft Dynamics Models (ADMs) to provide additional information suitable to compensate for the shortcomings of VBN and MEMS-IMU sensors in high-dynamics attitude determination tasks. An Extended Kalman Filter (EKF) is developed to fuse the information provided by the different sensors and to provide estimates of position, velocity and attitude of the UAV platform in real-time. Two different integrated navigation system architectures are implemented. The first uses VBN at 20 Hz and GPS at 1 Hz to augment the MEMS-IMU running at 100 Hz. The second mode also includes the ADM (computations performed at 100 Hz) to provide augmentation of the attitude channel. Simulation of these two modes is performed in a significant portion of the AEROSONDE UAV operational flight envelope and performing a variety of representative manoeuvres (i.e., straight climb, level turning, turning descent and climb, straight descent, etc.). Simulation of the first integrated navigation system architecture (VBN/IMU/GPS) shows that the integrated system can reach position, velocity and attitude accuracies compatible with CAT-II precision approach requirements. Simulation of the second system architecture (VBN/IMU/GPS/ADM) also shows promising results since the achieved attitude accuracy is higher using the ADM/VBS/IMU than using VBS/IMU only. However, due to rapid divergence of the ADM virtual sensor, there is a need for frequent re-initialisation of the ADM data module, which is strongly dependent on the UAV flight dynamics and the specific manoeuvring transitions performed

A low-cost vision based navigation system for small size unmanned aerial vehicle applications

Not availabl

Vision-Based navigation system for unmanned aerial vehicles

Mención Internacional en el título de doctorThe main objective of this dissertation is to provide Unmanned Aerial Vehicles (UAVs) with a robust navigation system; in order to allow the UAVs to perform complex tasks autonomously and in real-time. The proposed algorithms deal with solving the navigation problem for outdoor as well as indoor environments, mainly based on visual information that is captured by monocular cameras. In addition, this dissertation presents the advantages of using the visual sensors as the main source of data, or complementing other sensors in providing useful information; in order to improve the accuracy and the robustness of the sensing purposes. The dissertation mainly covers several research topics based on computer vision techniques: (I) Pose Estimation, to provide a solution for estimating the 6D pose of the UAV. This algorithm is based on the combination of SIFT detector and FREAK descriptor; which maintains the performance of the feature points matching and decreases the computational time. Thereafter, the pose estimation problem is solved based on the decomposition of the world-to-frame and frame-to-frame homographies. (II) Obstacle Detection and Collision Avoidance, in which, the UAV is able to sense and detect the frontal obstacles that are situated in its path. The detection algorithm mimics the human behaviors for detecting the approaching obstacles; by analyzing the size changes of the detected feature points, combined with the expansion ratios of the convex hull constructed around the detected feature points from consecutive frames. Then, by comparing the area ratio of the obstacle and the position of the UAV, the method decides if the detected obstacle may cause a collision. Finally, the algorithm extracts the collision-free zones around the obstacle, and combining with the tracked waypoints, the UAV performs the avoidance maneuver. (III) Navigation Guidance, which generates the waypoints to determine the flight path based on environment and the situated obstacles. Then provide a strategy to follow the path segments and in an efficient way and perform the flight maneuver smoothly. (IV) Visual Servoing, to offer different control solutions (Fuzzy Logic Control (FLC) and PID), based on the obtained visual information; in order to achieve the flight stability as well as to perform the correct maneuver; to avoid the possible collisions and track the waypoints. All the proposed algorithms have been verified with real flights in both indoor and outdoor environments, taking into consideration the visual conditions; such as illumination and textures. The obtained results have been validated against other systems; such as VICON motion capture system, DGPS in the case of pose estimate algorithm. In addition, the proposed algorithms have been compared with several previous works in the state of the art, and are results proves the improvement in the accuracy and the robustness of the proposed algorithms. Finally, this dissertation concludes that the visual sensors have the advantages of lightweight and low consumption and provide reliable information, which is considered as a powerful tool in the navigation systems to increase the autonomy of the UAVs for real-world applications.El objetivo principal de esta tesis es proporcionar Vehiculos Aereos no Tripulados (UAVs) con un sistema de navegacion robusto, para permitir a los UAVs realizar tareas complejas de forma autonoma y en tiempo real. Los algoritmos propuestos tratan de resolver problemas de la navegacion tanto en ambientes interiores como al aire libre basandose principalmente en la informacion visual captada por las camaras monoculares. Ademas, esta tesis doctoral presenta la ventaja de usar sensores visuales bien como fuente principal de datos o complementando a otros sensores en el suministro de informacion util, con el fin de mejorar la precision y la robustez de los procesos de deteccion. La tesis cubre, principalmente, varios temas de investigacion basados en tecnicas de vision por computador: (I) Estimacion de la Posicion y la Orientacion (Pose), para proporcionar una solucion a la estimacion de la posicion y orientacion en 6D del UAV. Este algoritmo se basa en la combinacion del detector SIFT y el descriptor FREAK, que mantiene el desempeno del a funcion de puntos de coincidencia y disminuye el tiempo computacional. De esta manera, se soluciona el problema de la estimacion de la posicion basandose en la descomposicion de las homografias mundo a imagen e imagen a imagen. (II) Deteccion obstaculos y elusion colisiones, donde el UAV es capaz de percibir y detectar los obstaculos frontales que se encuentran en su camino. El algoritmo de deteccion imita comportamientos humanos para detectar los obstaculos que se acercan, mediante el analisis de la magnitud del cambio de los puntos caracteristicos detectados de referencia, combinado con los ratios de expansion de los contornos convexos construidos alrededor de los puntos caracteristicos detectados en frames consecutivos. A continuacion, comparando la proporcion del area del obstaculo y la posicion del UAV, el metodo decide si el obstaculo detectado puede provocar una colision. Por ultimo, el algoritmo extrae las zonas libres de colision alrededor del obstaculo y combinandolo con los puntos de referencia, elUAV realiza la maniobra de evasion. (III) Guiado de navegacion, que genera los puntos de referencia para determinar la trayectoria de vuelo basada en el entorno y en los obstaculos detectados que encuentra. Proporciona una estrategia para seguir los segmentos del trazado de una manera eficiente y realizar la maniobra de vuelo con suavidad. (IV) Guiado por Vision, para ofrecer soluciones de control diferentes (Control de Logica Fuzzy (FLC) y PID), basados en la informacion visual obtenida con el fin de lograr la estabilidad de vuelo, asi como realizar la maniobra correcta para evitar posibles colisiones y seguir los puntos de referencia. Todos los algoritmos propuestos han sido verificados con vuelos reales en ambientes exteriores e interiores, tomando en consideracion condiciones visuales como la iluminacion y las texturas. Los resultados obtenidos han sido validados con otros sistemas: como el sistema de captura de movimiento VICON y DGPS en el caso del algoritmo de estimacion de la posicion y orientacion. Ademas, los algoritmos propuestos han sido comparados con trabajos anteriores recogidos en el estado del arte con resultados que demuestran una mejora de la precision y la robustez de los algoritmos propuestos. Esta tesis doctoral concluye que los sensores visuales tienen las ventajes de tener un peso ligero y un bajo consumo y, proporcionar informacion fiable, lo cual lo hace una poderosa herramienta en los sistemas de navegacion para aumentar la autonomia de los UAVs en aplicaciones del mundo real.Programa Oficial de Doctorado en Ingeniería Eléctrica, Electrónica y AutomáticaPresidente: Carlo Regazzoni.- Secretario: Fernando García Fernández.- Vocal: Pascual Campoy Cerver

Study of Future On-board GNSS/INS Hybridization Architectures

Un développement rapide et une densification du trafic aérien ont conduit à l'introduction de nouvelles opérations d'approches et d'atterrissage utilisant des trajectoires plus flexibles et des minimas plus exigeants. La plupart des opérations de navigation aérienne sont actuellement réalisées grâce au GNSS, augmenté par les systèmes GBAS, SBAS et ABAS qui permettent d'atteindre des opérations d'approches de précision (pour GBAS et SBAS). Cependant ces systèmes nécessitent la mise en place d'un réseau de station de référence relativement couteux et des diffusions constantes de messages aux utilisateurs de l'espace aérien. Afin de surmonter ces contraintes, le système ABAS intègre à bord des informations fournies par les systèmes de navigation inertielle (INS) ainsi améliorant les performances de navigation. Dans cette logique, les avions commerciaux actuels utilisent une solution de couplage des deux systèmes appelée hybridation GPS/baro-INS. Cette solution permet d'atteindre des niveaux de performance en termes de précision, intégrité, disponibilité et continuité supérieurs aux deux systèmes pris séparément. Malheureusement, les niveaux d'exigences requis par les opérations de précision ou les atterrissages automatiques ne peuvent pas encore être totalement couverts par les solutions d'hybridation actuelles. L'idée principale de cette thèse a été d'étendre le processus d'hybridation en incluant d'autres capteurs ou systèmes actuellement disponibles ou non à bord et d'évaluer les niveaux de performance atteints par cette solution de filtre d'hybridation global. L'objectif ciblé est de pouvoir fournir la plupart des paramètres de navigations pour les opérations les plus critiques avec le niveau de performance requis par les exigences OACI. Les opérations ciblées pendant l'étude étaient les approches de précision (en particulier les approches CAT III) et le roulage sur la piste. L'étude des systèmes vidéo a fait l'objet d'une attention particulière pendant la thèse. La navigation basée sur la vidéo est une solution autonome de navigation de plus en plus utilisée de nos jours axée sur des capteurs qui mesurent le mouvement du véhicule et observent l'environnement. Que cela soit pour compenser la perte ou la dégradation d'un des systèmes de navigation ou pour améliorer la solution existante, les intérêts de l'utilisation de la vidéo sont nombreux. ABSTRACT : The quick development of air traffic has led to the improvement of approach and landing operations by using flexible flight paths and by decreasing the minima required to perform these operations. Most of the aircraft operations are supported by the GNSS augmented with GBAS, SBAS and ABAS. SBAS or GBAS allow supporting navigation operations down to precision approaches. However, these augmentations do require an expensive network of reference receivers and real-time broadcast to the airborne user. To overcome, the ABAS system integrates on-board information provided by an INS so as to enhance the performance of the navigation system. In that scheme, INS is coupled with a GPS receiver in a GPS/baro-INS hybridization solution that is already performed on current commercial aircraft. This solution allows reaching better performance in terms of accuracy, integrity, availability and continuity than the two separated solutions. However the most stringent requirements for precision approaches or automatic landings cannot be fulfilled with the current hybridization. The main idea of this PhD study is then to extend the hybridization process by including other sensors already available on commercial aircraft or not and, to assess the performance reached by a global hybridization architecture. It aims at providing most of the navigation parameters in all operations with the required level of performance. The operations targeted by this hybridization are precision approaches, with a particular focus on CAT III precision approach and roll out on the runway. The study of video sensor has been particularly focused on in the thesis. Indeed video based navigation is a complete autonomous navigation opportunity only based on sensors that provide information from the dynamic of the vehicle and from the observation of the scenery. Moreover, from a possible compensation of any loss or degradation of a navigation system to the improvement of the navigation solution during the most critical operations, the interests of video are numerous