Design and integration of vision based sensors for unmanned aerial vehicles navigation and guidance

In this paper we present a novel Navigation and Guidance System (NGS) for Unmanned Aerial Vehicles (UAVs) based on Vision Based Navigation (VBN) and other avionics sensors. The main objective of our research is to design a lowcost and low-weight/volume NGS capable of providing the required level of performance in all flight phases of modern small- to medium-size UAVs, with a special focus on automated precision approach and landing, where VBN techniques can be fully exploited in a multisensory integrated architecture. Various existing techniques for VBN are compared and the Appearance-based Navigation (ABN) approach is selected for implementation

MIMO PID Controller Tuning Method for Quadrotor Based on LQR/LQG Theory

In this work, a new pre-tuning multivariable PID (Proportional Integral Derivative) controllers method for quadrotors is put forward. A procedure based on LQR/LQG (Linear Quadratic Regulator/Gaussian) theory is proposed for attitude and altitude control, which suposes a considerable simplification of the design problem due to only one pretuning parameter being used. With the aim to analyze the performance and robustness of the proposed method, a non-linear mathematical model of the DJI-F450 quadrotor is employed, where rotors dynamics, together with sensors drift/bias properties and noise characteristics of low-cost commercial sensors typically used in this type of applications are considered. In order to estimate the state vector and compensate bias/drift effects in the measures, a combination of filtering and data fusion algorithms (Kalman filter and Madgwick algorithm for attitude estimation) are proposed and implemented. Performance and robustness analysis of the control system is carried out by employing numerical simulations, which take into account the presence of uncertainty in the plant model and external disturbances. The obtained results show the proposed controller design method for multivariable PID controller is robust with respect to: (a) parametric uncertainty in the plant model, (b) disturbances acting at the plant input, (c) sensors measurement and estimation errors

Contributions to improve the technologies supporting unmanned aircraft operations

Mención Internacional en el título de doctorUnmanned Aerial Vehicles (UAVs), in their smaller versions known as drones, are becoming increasingly important in today's societies. The systems that make them up present a multitude of challenges, of which error can be considered the common denominator. The perception of the environment is measured by sensors that have errors, the models that interpret the information and/or define behaviors are approximations of the world and therefore also have errors. Explaining error allows extending the limits of deterministic models to address real-world problems. The performance of the technologies embedded in drones depends on our ability to understand, model, and control the error of the systems that integrate them, as well as new technologies that may emerge. Flight controllers integrate various subsystems that are generally dependent on other systems. One example is the guidance systems. These systems provide the engine's propulsion controller with the necessary information to accomplish a desired mission. For this purpose, the flight controller is made up of a control law for the guidance system that reacts to the information perceived by the perception and navigation systems. The error of any of the subsystems propagates through the ecosystem of the controller, so the study of each of them is essential. On the other hand, among the strategies for error control are state-space estimators, where the Kalman filter has been a great ally of engineers since its appearance in the 1960s. Kalman filters are at the heart of information fusion systems, minimizing the error covariance of the system and allowing the measured states to be filtered and estimated in the absence of observations. State Space Models (SSM) are developed based on a set of hypotheses for modeling the world. Among the assumptions are that the models of the world must be linear, Markovian, and that the error of their models must be Gaussian. In general, systems are not linear, so linearization are performed on models that are already approximations of the world. In other cases, the noise to be controlled is not Gaussian, but it is approximated to that distribution in order to be able to deal with it. On the other hand, many systems are not Markovian, i.e., their states do not depend only on the previous state, but there are other dependencies that state space models cannot handle. This thesis deals a collection of studies in which error is formulated and reduced. First, the error in a computer vision-based precision landing system is studied, then estimation and filtering problems from the deep learning approach are addressed. Finally, classification concepts with deep learning over trajectories are studied. The first case of the collection xviiistudies the consequences of error propagation in a machine vision-based precision landing system. This paper proposes a set of strategies to reduce the impact on the guidance system, and ultimately reduce the error. The next two studies approach the estimation and filtering problem from the deep learning approach, where error is a function to be minimized by learning. The last case of the collection deals with a trajectory classification problem with real data. This work completes the two main fields in deep learning, regression and classification, where the error is considered as a probability function of class membership.Los vehículos aéreos no tripulados (UAV) en sus versiones de pequeño tamaño conocidos como drones, van tomando protagonismo en las sociedades actuales. Los sistemas que los componen presentan multitud de retos entre los cuales el error se puede considerar como el denominador común. La percepción del entorno se mide mediante sensores que tienen error, los modelos que interpretan la información y/o definen comportamientos son aproximaciones del mundo y por consiguiente también presentan error. Explicar el error permite extender los límites de los modelos deterministas para abordar problemas del mundo real. El rendimiento de las tecnologías embarcadas en los drones, dependen de nuestra capacidad de comprender, modelar y controlar el error de los sistemas que los integran, así como de las nuevas tecnologías que puedan surgir. Los controladores de vuelo integran diferentes subsistemas los cuales generalmente son dependientes de otros sistemas. Un caso de esta situación son los sistemas de guiado. Estos sistemas son los encargados de proporcionar al controlador de los motores información necesaria para cumplir con una misión deseada. Para ello se componen de una ley de control de guiado que reacciona a la información percibida por los sistemas de percepción y navegación. El error de cualquiera de estos sistemas se propaga por el ecosistema del controlador siendo vital su estudio. Por otro lado, entre las estrategias para abordar el control del error se encuentran los estimadores en espacios de estados, donde el filtro de Kalman desde su aparición en los años 60, ha sido y continúa siendo un gran aliado para los ingenieros. Los filtros de Kalman son el corazón de los sistemas de fusión de información, los cuales minimizan la covarianza del error del sistema, permitiendo filtrar los estados medidos y estimarlos cuando no se tienen observaciones. Los modelos de espacios de estados se desarrollan en base a un conjunto de hipótesis para modelar el mundo. Entre las hipótesis se encuentra que los modelos del mundo han de ser lineales, markovianos y que el error de sus modelos ha de ser gaussiano. Generalmente los sistemas no son lineales por lo que se realizan linealizaciones sobre modelos que a su vez ya son aproximaciones del mundo. En otros casos el ruido que se desea controlar no es gaussiano, pero se aproxima a esta distribución para poder abordarlo. Por otro lado, multitud de sistemas no son markovianos, es decir, sus estados no solo dependen del estado anterior, sino que existen otras dependencias que los modelos de espacio de estados no son capaces de abordar. Esta tesis aborda un compendio de estudios sobre los que se formula y reduce el error. En primer lugar, se estudia el error en un sistema de aterrizaje de precisión basado en visión por computador. Después se plantean problemas de estimación y filtrado desde la aproximación del aprendizaje profundo. Por último, se estudian los conceptos de clasificación con aprendizaje profundo sobre trayectorias. El primer caso del compendio estudia las consecuencias de la propagación del error de un sistema de aterrizaje de precisión basado en visión artificial. En este trabajo se propone un conjunto de estrategias para reducir el impacto sobre el sistema de guiado, y en última instancia reducir el error. Los siguientes dos estudios abordan el problema de estimación y filtrado desde la perspectiva del aprendizaje profundo, donde el error es una función que minimizar mediante aprendizaje. El último caso del compendio aborda un problema de clasificación de trayectorias con datos reales. Con este trabajo se completan los dos campos principales en aprendizaje profundo, regresión y clasificación, donde se plantea el error como una función de probabilidad de pertenencia a una clase.I would like to thank the Ministry of Science and Innovation for granting me the funding with reference PRE2018-086793, associated to the project TEC2017-88048-C2-2-R, which provide me the opportunity to carry out all my PhD. activities, including completing an international research internship.Programa de Doctorado en Ciencia y Tecnología Informática por la Universidad Carlos III de MadridPresidente: Antonio Berlanga de Jesús.- Secretario: Daniel Arias Medina.- Vocal: Alejandro Martínez Cav

Low-cost vision sensors and integrated systems for unmanned aerial vehicle navigation

A novel low cost navigation system based on Vision Based Navigation (VBN) and other avionics sensors is presented, which is designed for small size Unmanned Aerial Vehicle (UAV) applications. The main objective of our research is to design a compact, light and relatively inexpensive system capable of providing the required navigation performance in all phases of flight of a small UAV, with a special focus on precision approach and landing, where Vision Based Navigation (VBN) techniques can be fully exploited in a multisensory integrated architecture. Various existing techniques for VBN are compared and the Appearance-based Navigation (ABN) approach is selected for implementation. Feature extraction and optical flow techniques are employed to estimate flight parameters such as roll angle, pitch angle, deviation from the runway and body rates. Additionally, we address the possible synergies between VBN, Global Navigation Satellite System (GNSS) and MEMS-IMU (Micro-Electromechanical System Inertial Measurement Unit) sensors and also the use of Aircraft Dynamics Models (ADMs) to provide additional information suitable to compensate for the shortcomings of VBN and MEMS-IMU sensors in high-dynamics attitude determination tasks. An Extended Kalman Filter (EKF) is developed to fuse the information provided by the different sensors and to provide estimates of position, velocity and attitude of the UAV platform in real-time. Two different integrated navigation system architectures are implemented. The first uses VBN at 20 Hz and GPS at 1 Hz to augment the MEMS-IMU running at 100 Hz. The second mode also includes the ADM (computations performed at 100 Hz) to provide augmentation of the attitude channel. Simulation of these two modes is performed in a significant portion of the AEROSONDE UAV operational flight envelope and performing a variety of representative manoeuvres (i.e., straight climb, level turning, turning descent and climb, straight descent, etc.). Simulation of the first integrated navigation system architecture (VBN/IMU/GPS) shows that the integrated system can reach position, velocity and attitude accuracies compatible with CAT-II precision approach requirements. Simulation of the second system architecture (VBN/IMU/GPS/ADM) also shows promising results since the achieved attitude accuracy is higher using the ADM/VBS/IMU than using VBS/IMU only. However, due to rapid divergence of the ADM virtual sensor, there is a need for frequent re-initialisation of the ADM data module, which is strongly dependent on the UAV flight dynamics and the specific manoeuvring transitions performed

Third-Person Autonomous Control using Deep RC

The ability to fly a remote control (RC) aircraft from a third-person perspective is a skill that many hobbyists and enthusiasts enjoy. With a little practice, an RC pilot can sense the aircraft’s orientation and apply the correct inputs for it to orbit, achieve specific mission objectives, or line-of-sight waypoints. The work done in [1] proved that third-person sensing of an aircraft’s attitude is possible. This work seeks to improve the deep learning methods, increase hardware capabilities through the addition of a turret and motion capture validation, and add control algorithms for complete autonomous flight. While the results are not comprehensive, this report gives the work accomplished thus far. Overall this report will show that substantial progress has been made towards third-person autonomous control

Exploring the Technical Advances and Limits of Autonomous UAVs for Precise Agriculture in Constrained Environments

In the field of precise agriculture with autonomous unmanned aerial vehicles (UAVs), the utilization of drones holds significant potential to transform crop monitoring, management, and harvesting techniques. However, despite the numerous benefits of UAVs in smart farming, there are still several technical challenges that need to be addressed in order to render their widespread adoption possible, especially in constrained environments. This paper provides a study of the technical aspect and limitations of autonomous UAVs in precise agriculture applications for constrained environments

Error Reduction in Vision-Based Multirotor Landing System

New applications are continuously appearing with drones as protagonists, but all of them share an essential critical maneuver—landing. New application requirements have led the study of novel landing strategies, in which vision systems have played and continue to play a key role. Generally, the new applications use the control and navigation systems embedded in the aircraft. However, the internal dynamics of these systems, initially focused on other tasks such as the smoothing trajectories between different waypoints, can trigger undesired behaviors. In this paper, we propose a landing system based on monocular vision and navigation information to estimate the helipad global position. In addition, the global estimation system includes a position error correction module by cylinder space transformation and a filtering system with a sliding window. To conclude, the landing system is evaluated with three quality metrics, showing how the proposed correction system together with stationary filtering improves the raw landing system.This research was partially funded by public research projects of Spanish Ministry of Science and Innovation, references PID2020-118249RB-C22 and PDC2021-121567-C22—AEI/10.13039/ 501100011033, and by the Madrid Government (Comunidad de Madrid, Spain) under the Multiannual Agreement with UC3M in the line of Excellence of University Professors, reference EPUC3M17

On the Complete Automation of Vertical Flight Aircraft Ship Landing

The current study focuses on developing an autonomous vertical flight aircraft ship landing system by directly automating the established Navy helicopter ship landing procedure. The central idea involves visually tracking a gyro-stabilized horizon bar installed on most Navy ships to approach and land vertically independent of deck motions. This was accomplished through the development of a rotorcraft flight dynamics modeling framework and vision-based control systems as well as conducting simulations and flight tests. The framework, named Texas A&M Rotorcraft Analysis Code (TRAC), was developed as a modular tool that could model any rotorcraft configuration at a low computational cost. A UH-60 helicopter was modeled as a baseline aircraft and validated using the US Army flight test data. A linear quadratic regulator (LQR) controller was utilized to stabilize and control the helicopter during autonomous ship landing simulations. The vision system was developed to obtain the visual information that a pilot perceives during ship approach and landing. It detects the ship at long-distance by utilizing machine/deep learning-based detection and at close range, it utilizes uniquely developed vision algorithms to detect the horizon bar to precisely estimate the aircraft position and orientation relative to the bar. It demonstrated 250 meters of detection range for a 6 x 6 ft sub-scale ship platform, which translates to a range of 17.3 kilometers for a full-scale 50 x 50 ft typical small ship. The distance and attitude estimations were validated using the measurements from an accurate 3D motion capture system (VICON), which demonstrated sub-centimeter and sub-degree accuracy. To control the aircraft based on the perceived visual information, both nonlinear control and deep reinforcement learning control strategies were developed. The nonlinear controller demonstrated robust tracking capability even with 0.5 seconds of time delay and estimation noise. When flight-tested in 5 m/s wind gust, the deep reinforcement learning control demonstrated superior disturbance rejection capability, with 50% reduced drift at a 3 times faster rate compared to conventional control systems. Both vision and control systems were implemented on a quadrotor unmanned aircraft and extensive flight tests were conducted to demonstrate accurate tracking in challenging conditions and safe vertical landing on a translating ship platform with 6 degrees of freedom motions