Empirical evaluation of a Q-Learning Algorithm for Model-free Autonomous Soaring

Autonomous unpowered flight is a challenge for control and guidance systems: all the energy the aircraft might use during flight has to be harvested directly from the atmosphere. We investigate the design of an algorithm that optimizes the closed-loop control of a glider's bank and sideslip angles, while flying in the lower convective layer of the atmosphere in order to increase its mission endurance. Using a Reinforcement Learning approach, we demonstrate the possibility for real-time adaptation of the glider's behaviour to the time-varying and noisy conditions associated with thermal soaring flight. Our approach is online, data-based and model-free, hence avoids the pitfalls of aerological and aircraft modelling and allow us to deal with uncertainties and non-stationarity. Additionally, we put a particular emphasis on keeping low computational requirements in order to make on-board execution feasible. This article presents the stochastic, time-dependent aerological model used for simulation, together with a standard aircraft model. Then we introduce an adaptation of a Q-learning algorithm and demonstrate its ability to control the aircraft and improve its endurance by exploiting updrafts in non-stationary scenarios

An End-to-End Platform for Autonomous Dynamic Soaring in Wind Shear

Despite advancements in our understanding of flight in modern times, birds remain unmatched when it comes to maneuverability and energy efficiency in flight; in particular seabirds like the albatross are known to travel vast distances without stopping for food by performing an aerobatic maneuver called dynamic soaring. When the maneuver is executed in the presence of a wind field that varies in strength of direction, the albatross extracts kinetic energy from the field. In this dissertation, we present an end-to-end system designed to exploit wind as the albatross does. The system we designed consists of a gliding platform outfitted with sensors and computational hardware, an on-board software platform that enables autonomy, and a ground platform for monitoring mission performance and issuing commands.We contribute the design of an airframe, the Fox, capable of performing dynamic soaring at low altitudes (~400m above sea level). We validate the airframe against expected stressors (vibration, coefficient of lift, temperature, and communication signal strength), and show in simulation it can complete a dynamic soaring orbit in wind shear that varies in maximum wind speed from 8 to 12 m/s. We show that this airframe can reach speeds exceeding 40 m/s while soaring.We fit the airframe with a commercial off-the-shelf autopilot, as well as a custom on-board-computing (OBC) solution to provide the necessary facilities to enable autonomy. The OBC generates dynamic soaring trajectories that fit a wind-field map that is built as the aircraft is deployed and controls the Fox to follow them by sending commands to the autopilot using a sample-based controller scheme. This process is monitored by human operators on the ground via a portable ground station that is linked to the Fox via a radio antenna. Field tests are presented that validate real-world controller performance against simulated results.Finally, we present a learning controller that learns from and out-performs the sample-based controller in simulation. While not field tested, we believe a self-optimizing controller of this form is necessary to enable autonomy of a soaring aircraft subject to extended mission durations.While dynamic soaring field tests were not pursued in this work, we hope this dissertation will be a blueprint for future researchers to finally achieve autonomous soaring