Perpetual flight in flow fields

Tese de Doutoramento. Engenharia Electrotécnica e de Computadores. Faculdade de Engenharia. Universidade do Porto. 201

Empirical evaluation of a Q-Learning Algorithm for Model-free Autonomous Soaring

Autonomous unpowered flight is a challenge for control and guidance systems: all the energy the aircraft might use during flight has to be harvested directly from the atmosphere. We investigate the design of an algorithm that optimizes the closed-loop control of a glider's bank and sideslip angles, while flying in the lower convective layer of the atmosphere in order to increase its mission endurance. Using a Reinforcement Learning approach, we demonstrate the possibility for real-time adaptation of the glider's behaviour to the time-varying and noisy conditions associated with thermal soaring flight. Our approach is online, data-based and model-free, hence avoids the pitfalls of aerological and aircraft modelling and allow us to deal with uncertainties and non-stationarity. Additionally, we put a particular emphasis on keeping low computational requirements in order to make on-board execution feasible. This article presents the stochastic, time-dependent aerological model used for simulation, together with a standard aircraft model. Then we introduce an adaptation of a Q-learning algorithm and demonstrate its ability to control the aircraft and improve its endurance by exploiting updrafts in non-stationary scenarios

UAS planning and trajectory generation for safe and long-duration oceanic and coastal missions.

La presente tesis doctoral, muestra el diseño de un sistema para la extensión de la duración de vuelo de sistemas autónomos no tripulados de tamaño pequeño. Este sistema fue diseñado en el contexto de misiones de vigilancia marítima y costera como parte del proyecto europeo MarineUAS. En este contexto, se han identificado tres problemas: 1) la necesidad de la estimación precisa de un campo de viento y la capacidad de identificación de fenómenos como el viento cortante o las ráfagas continuas y discretas para que puedan ser utilizadas potencialmente para la extracción de energía para mejorar la duración de vuelo. 2) La necesidad de generar trayectorias suaves para la extracción de energía considerando la dinámica de las plataformas de vuelo y 3) la habilidad de seguir dichas trayectorias. Para el primer problema, el uso de un método de computación directa permite determinar el campo de viento (velocidad y tasa de cambio de la velocidad de viento) sin la utilización de un estimador óptimo. Sin embargo, también se consideraron varios métodos y a partir de un análisis extenso se presentan diferentes comparativas de estos métodos, en el que se muestran las ventajas y desventajas de los mismos. Adicionalmente, la identificación de distintos fenómenos de viento, cómo las ráfagas, o el viento cortante, se logra a través de un innovador método que ejecuta una serie de pruebas estadísticas basadas en la distribución de Weibull y en distintos modelos dinámicos que consideran no solo la distribución del viento sino la interacción con el océano y la superficie en las respectivas capas límite. Para el segundo problema, una aproximación biomimética permitió el uso de un algoritmo complejo para la réplica de trajectorias de vuelo dinámico de aves. En dicho algoritmo se consideran observaciones presentadas por distintos científicos que permiten generar trayectorias paramétricas que consideran además restricciones cinemáticas de la plataforma en el diseño de las mismas. El tercer problema toma en consideración la curva generada y utiliza la teoría del campo de vectores para diseñar un controlador que permite seguir dicha trayectoria de manera eficiente y en tiempo real, respetando las leyes de control de bajo nivel en el autopiloto y permitiendo flexibilidad. Como complemento a este último sistema, se propone la reconfiguración dinámica de las misiones para mejorar el consumo energético durante el tiempo de vuelo considerando el viento predominante. Uno de los principales objectivos fue integrar, utilizando la metodología de ingeniería de sistemas, las distintias funciones anteriormente mencionadas en el que la ejecución de la misión fuese la prioridad. El principal logro fue haber realizado una extensa campaña experimental que permitió la validación del sistema en diferentes niveles, en el que se combinaron pruebas computacionales de alto y bajo nivel así como pruebas de campo en distintos escenarios y con distintas plataformas, lo cual permitió explorar la versatilidad del sistema. Los resultados muestran que se pueden lograr misiones más eficientes con mejoras de hasta un 20%en consumo de batería para misiones costeras. Finalmente, de los distintos análisis computacionales efectuados se concluye que el tiempo de ejecución de toda la función de extensión del vuelo es lo suficientemente pequeño para permitir la ejecución en tiempo real, lo cual, combinando con el diseño versátil en cuestión de arquitectura computacional, permiten la portabilidad del sistema así como la futura integración de funciones adicionales.In this thesis a system that aims to extend the flight duration of small Unmanned Aerial Systems (UAS) is presented. The system was designed in the context of oceanic and coastal surveillance missions as part of the MarineUAS European project. Three main problems were identified: 1) the need to accurately estimate the wind field and the capability to identify features of interest, such as, wind shear, and gusts that may be suitable to allow energy extraction to improve flight duration. 2) the need to generate smooth trajectories that extract energy, considering the UAS platform dynamics and 3) the ability to follow such paths. For the first problem, the use of a direct computation method allows determining the wind field (wind velocity and wind rate of change) without the use of an optimal estimator. Nevertheless, different wind velocity estimation methods are compared, and the pros and cons of each are exposed; in addition, the identification of features is accomplished with a novel approach that performs a real-time statistical analysis of the distribution of the wind field estimates, allowing the characterization of the shear components and also any other potential features, like continuous and discrete gusts considering complex models that take into account not only the phenomena but the interactions with the ground and ocean through their respective boundary layers. For the second problem, a biomimetic approach is presented, replicating the trajectories of soaring birds by considering observations of these birds and the replication of their swooping maneuvers using smooth parametrized curves. This allows flexibility in the curve design and also the incorporation of dynamic constraints of the platform on it. The solution of the third problem takes into account the smooth curve that was generated and among it, a type 1 Bishop moving frame is generated. Then, a novel adaptive control method based on the vector-field theory approach is proposed to calculate the error equations and the respective control law, which permits the tracking of the designed trajectory for dynamic soaring. Furthermore, an additional step was added, in which the surveillance mission is re-configured on a waypoint-to-waypoint basis for a more efficient flight considering the identified wind field. The result was that the execution of soaring trajectories would not be executed during all the mission, but only in specific legs that fulfill specific characteristics.The primary goal was to design algorithms that implement these functions and to integrate these functionalities in a systems-engineering approach, in which the mission execution is the main priority. An extensive experimental campaign was performed at different levels, in which software-in-the-loop and hardwarein- the-loop tests, together with field tests, were executed to demonstrate the efficiency of the various functions separately and integrated. The field tests and the simulations consider different scenarios and UAS platforms, showing the performance of the system in different conditions. The results showed that the system could execute a more efficient mission, with savings of up to 20% in battery consumption, with the so-called of the Flight-Duration-Enhancement-System (FDES). Finally, the computational analysis showed that the system could be executed in real-time with minimum latency despite the use of sophisticated algorithms; this, together with the chosen software and hardware architectures allows portability to other hardware components and the possibility of incorporating additional functions

Coordination on Systems of Multiple UAVs

Esta tesis trata acerca de métodos para coordinar las trayectorias de un sistema de Vehículos Aéreos no Tripulados y Autónomos (en adelante UAVs). El primer conjunto de técnicas desarrolladas durante la tesis se agrupan dentro de las técnicas de planificación de trayectorias. En este caso, el objetivo es generar planes de vuelo para un conjunto de vehículos coordinadamente de forma que no se produzcan colisiones entre ellos. Además, este tipo de técnicas puede usarse para modificar el plan de vuelo de un subconjunto de UAVs en tiempo real. Entre los algoritmos desarrollados en la tesis podemos destacar la adaptación de algoritmos evolutivos como los Algoritmos Genéticos y el Particle Swarm (Enjambre de Partículas), la incorporación de nuevas formas de muestreo del espacio para la aplicación del algoritmo Optimal Rapidly Exploring Random Trees (RRT*) en sistemas multi-UAV usando técnicas de muestreo novedosas. También se ha estudiado el comportamiento de parte de estos algoritmos en situaciones variables de incertidumbre del estado del sistema. En particular, se propone el uso del Filtro de Partículas para estimar la posición relativa entre varios UAVs. Además, se estudia la aplicación de métodos reactivos para la resolución de colisiones en tiempo real. Esta tesis propone un nuevo algoritmo para la resolución de colisiones entre múltiples UAVs en presencia de obstáculos fijos llamado G-ORCA. Este algoritmo soluciona varios problemas que han surgido al aplicar el algoritmo ORCA en su variante 3D en sistemas compuestos por vehículos reales. Su seguridad se ha demostrado tanto analíticamente, como empíricamente en pruebas con sistemas reales. De hecho, durante esta tesis numerosos experimentos en sistemas multi-UAV reales compuestos hasta por 4 UAVs han sido ejecutados. En dichos experimentos, se realiza una coordinación autónoma de UAVs en las que se asegura la ejecución de trayectorias libres de colisiones garantizando por tanto la seguridad del sistema. Una característica reseñable de esta tesis es que los algoritmos desarrollados han sido probados e integrados en sistemas más complejos que son usados en aplicaciones reales. En primer lugar, se presenta un sistema para aumentar la duración del vuelo de planeadores aprovechando las corrientes ascendentes de viento generadas por el calor (térmicas). En segundo lugar, un sistema de detección y resolución de colisiones coordinado para sistemas con múltiples UAVs reactivo ha sido diseñado, desarrollado y probado experimentalmente. Este sistema ha sido integrado dentro de un sistema automático de construcción de estructuras mediante múltiples UAVs.The aim of this thesis is to propose methods to coordinately generate trajectories for a system of Autonomous Unmanned Aerial Vehicles (UAVs). The first set of proposed techniques developed in this thesis can be defined as trajectory planning techniques. In this case, the objective is to generate coordinated flight plans for a system of UAVs in such a way that no collision are produced among each pair of UAVs. Besides, these techniques can be applied online in order to modify the original flight plan whenever a potential collision is detected. Amongst the developed algorithms in this thesis we can highlight the adaptation of evolutionary algorithms such as Genetic Algorithms and Particle Swarm, and the application of Optimal Rapidly Exploring Random Trees (RRT*) algorithm into a system of several UAVs with novel sampling techniques. In addition, many of these techniques have been adapted in order to be applicable when only uncertain knowledge of the state of the system is available. In particular, the use of the Particle Filter is proposed in order to estimate the relative position between UAVs. The estimation of the position as well as the uncertainty related to this estimation are then taken into account in the conflict resolution system. All techniques proposed in this thesis have been validated by performing several simulated and real tests. For this purpose, a method for randomly generating a huge test batch is presented in chapter 3. This will allow to test the behavior of the proposed methods in a great variety of situations. During the thesis, several real experimentations with fleets composed by up to four UAVs are presented. In these experiments, the UAVs in the system are automatically coordinated in order to ensure collision-free trajectories and thus guarantee the safety of the system. The other main topic of this thesis is the application of reactive methods for real-time conflict resolution. This thesis proposes a novel algorithm for collision resolution amongst multiple UAVs in the presence of static obstacles, which has been called Generalized-Optimal Reciprocal Collision Avoidance (G-ORCA). This algorithm overcomes several issues that have been detected into the algorithm 3D-ORCA in real applications. A remarkable characteristic of this thesis is that the developed algorithms have been applied as a part of more complex systems. First, a coordinated system for flight endurance extension of gliding aircrafts by profiting the ascending wind is presented. Second, a reactive collision avoidance block has been designed, developed and tested experimentally based in the aforementioned G-ORCA algorithm. This block has been integrated into a system for assembly construction with multiple UAVs

Navigation and autonomy of soaring unmanned aerial vehicles

The use of Unmanned Aerial Vehicles (UAV) has exploded over the last decade with the constant need to reduce costs while maintaining capability. Despite the relentless development of electronics and battery technology there is a sustained need to reduce the size and weight of the on-board systems to free-up payload capacity. One method of reducing the energy storage requirement of UAVs is to utilise naturally occurring sources of energy found in the atmosphere. This thesis explores the use of static and semi-dynamic soaring to extract energy from naturally occurring shallow layer cumulus convection to improve range, endurance and average speed. A simulation model of an X-Models XCalibur electric motor-glider is used in combination with a refined 4D parametric atmospheric model to simulate soaring flight. The parametric atmospheric model builds on previous successful models with refinements to more accurately describe the weather in northern Europe. The implementation of the variation of the MacCready setting is discussed. Methods for generating efficient trajectories are evaluated and recommendations are made regarding implementation. For micro to small UAVs to be able to track the desired trajectories a highly accurate Attitude Heading Reference System (AHRS) is needed. Detailed analysis of the practical implementation of advanced attitude determination is used to enable optimal execution of the trajectories generated. The new attitude determination methods are compared to existing Kalman and complimentary type filters. Analysis shows the methods developed are capable of providing accurate attitude determination with extremely low computational requirements, even during extreme manoeuvring. The new AHRS techniques reduce the need for powerful on-board microprocessors. This new AHRS technique is used as a foundation to develop a robust navigation filter capable of providing improved drift performance, over traditional filters, in the temporary absence of global navigation satellite information. All these algorithms have been verified by flight tests using a mixture of manned and unmanned aerial vehicles and avionics developed specifically for this thesis

Fusion of sensor information to measure the total energy of an aircraft and provide information about flight performance and local microclimate

The application of using Unmanned Aerial Vehicles (UAVs) to locate thermal updraft currents is a relatively new topic. It was first proposed in 1998 by John Wharington, and, subsequently, several researchers have developed algorithms to search and exploit thermals. However, few people have physically implemented a system and performed field testing. The aim of this project was to develop a low cost system to be carried on a glider to detect thermals effectively. A system was developed from the ground up and consisted of custom hardware and software that was developed specifically for aircraft. Data fusion was performed to estimate the attitude of the aircraft; this was done using a direction cosine (DCM) based method. Altitude and airspeed data were fused by estimating potential and kinetic energy respectively; thus determining the aircraft's total energy. This data was then interpreted to locate thermal activity. The system comprised an Inertial Measurement Unit (IMU), airspeed sensor, barometric altitude sensor, Global Positioning System (GPS), temperature sensor, SD card and a realtime telemetry link. These features allowed the system to determine aircraft position, height, airspeed and air temperature in realtime. A custom-designed radio controlled (RC) glider was constructed from composite materials in addition to a second 3.6 m production glider that was used during flight testing. Sensor calibration was done using a wind tunnel with custom designed apparatus that allowed a complete wing with its pitot tube to be tested in one operation. Flight testing was conducted in the field at several different locations over the course of six months. A total of 25 recorded flights were made during this period. Both thermal soaring and ridge soaring were performed to test the system under varying weather conditions. A telemetry link was developed to transfer data in realtime from the aircraft to a custom ground station. The recorded results were post-processed using Matlab and showed that the system was able to detect thermal updrafts. The sensors used in the system were shown to provide acceptable performance once some calibration had been performed. Sensor noise proved to be problematic, and time was spent alleviating its effects. Results showed that the system was able to measure airspeed to within ± 1 km/h. The standard deviation of the altitude estimate was determined to be 0.94 m. This was deemed to be satisfactory. The system was highly reliable and no faults occurred during operation. In conclusion, the project showed that inexpensive sensors and low power microcontrollers could be used very effectively for the application of detecting thermals

An aircraft and provide information about flight performance and local microclimate

Includes abstract.Includes bibliographical referencesThe application of using Unmanned Aerial Vehicles (UAVs) to locate thermal updraft currentsis a relatively new topic. It was first proposed in 1998 by John Wharington, and, subsequently, several researchers have developed algorithms to search and exploit thermals. However, few people have physically implemented a system and performed field testing. The aim of this project was to develop a low cost system to be carried on a glider to detect thermals effectively. A system was developed from the ground up and consisted of custom hardware and software that was developed specifically for aircraft. Data fusion was performed to estimate the attitude of the aircraft; this was done using a direction cosine (DCM) based method. Altitude and airspeed data were fused by estimating potential and kinetic energy respectively; thus determining the aircraft’s total energy. This data was then interpreted to locate thermal activity. The system comprised an Inertial Measurement Unit (IMU), airspeed sensor, barometric altitude sensor, Global Positioning System (GPS), temperature sensor, SD card and a realtime telemetry link. These features allowed the system to determine aircraft position, height, airspeed and air temperature in realtime. A custom-designed radio controlled (RC) glider was constructed from composite materials in addition to a second 3.6 m production glider that was used during flight testing. Sensor calibration was done using a wind tunnel with custom designed apparatus that allowed a complete wing with its pitot tube to be tested in one operation. Flight testing was conducted in the field at several different locations over the course of six months. A total of 25 recorded flights were made during this period. Both thermal soaring and ridge soaring were performed to test the system under varying weather conditions. A telemetry link was developed to transfer data in realtime from the aircraft to a custom ground station. The recorded results were post-processed using Matlab and showed that the system was able to detect thermal updrafts. The sensors used in the system were shown to provide acceptable performance once some calibration had been performed. Sensor noise proved to be problematic, and time was spent alleviating its effects