Reinforcement Learning for UAV Attitude Control

Autopilot systems are typically composed of an "inner loop" providing stability and control, while an "outer loop" is responsible for mission-level objectives, e.g. way-point navigation. Autopilot systems for UAVs are predominately implemented using Proportional, Integral Derivative (PID) control systems, which have demonstrated exceptional performance in stable environments. However more sophisticated control is required to operate in unpredictable, and harsh environments. Intelligent flight control systems is an active area of research addressing limitations of PID control most recently through the use of reinforcement learning (RL) which has had success in other applications such as robotics. However previous work has focused primarily on using RL at the mission-level controller. In this work, we investigate the performance and accuracy of the inner control loop providing attitude control when using intelligent flight control systems trained with the state-of-the-art RL algorithms, Deep Deterministic Gradient Policy (DDGP), Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO). To investigate these unknowns we first developed an open-source high-fidelity simulation environment to train a flight controller attitude control of a quadrotor through RL. We then use our environment to compare their performance to that of a PID controller to identify if using RL is appropriate in high-precision, time-critical flight control.Comment: 13 pages, 9 figure

A survey on fractional order control techniques for unmanned aerial and ground vehicles

In recent years, numerous applications of science and engineering for modeling and control of unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs) systems based on fractional calculus have been realized. The extra fractional order derivative terms allow to optimizing the performance of the systems. The review presented in this paper focuses on the control problems of the UAVs and UGVs that have been addressed by the fractional order techniques over the last decade

Experimental Results of Concurrent Learning Adaptive Controllers

Commonly used Proportional-Integral-Derivative based UAV flight controllers are often seen to provide adequate trajectory-tracking performance only after extensive tuning. The gains of these controllers are tuned to particular platforms, which makes transferring controllers from one UAV to other time-intensive. This paper suggests the use of adaptive controllers in speeding up the process of extracting good control performance from new UAVs. In particular, it is shown that a concurrent learning adaptive controller improves the trajectory tracking performance of a quadrotor with baseline linear controller directly imported from another quadrotors whose inertial characteristics and throttle mapping are very di fferent. Concurrent learning adaptive control uses specifi cally selected and online recorded data concurrently with instantaneous data and is capable of guaranteeing tracking error and weight error convergence without requiring persistency of excitation. Flight-test results are presented on indoor quadrotor platforms operated in MIT's RAVEN environment. These results indicate the feasibility of rapidly developing high-performance UAV controllers by using adaptive control to augment a controller transferred from another UAV with similar control assignment structure.United States. Office of Naval Research. Multidisciplinary University Research Initiative (Grant N000141110688)National Science Foundation (U.S.). Graduate Research Fellowship Program (Grant 0645960)Boeing Scientific Research Laboratorie

Automatic control of a multirotor

Objective of this thesis is to describe the design and realisation phases of a multirotor to be used for low risk and cost aerial observation. Starting point of this activity was a wide literature study related to the technological evolution of multirotors design and to the state of the art. Firstly the most common multirotor configurations were defined and, according to a size and performance based evaluation, the most suitable one was chosen. A detailed computer aided design model was drawn as basis for the realisation of two prototypes. The realised multirotors were “X-shaped” octorotors with eight coaxially coupled motors. The mathematical model of the multirotor dynamics was studied. “Proportional Integral Derivative” and “Linear Quadratic” algorithms were chosen as techniques to regulate the attitude dynamics of the multirotor. These methods were tested with a nonlinear model simulation developed in the Matlab Simulink environment. In the meanwhile the Arduino board was selected as the best compromise between costs and performance and the above mentioned algorithms were implemented using this platform thanks to its main characteristic of being completely “open source”. Indeed the multirotor was conceived to be a serviceable tool for the public utility and, at the same time, to be an accessible device for research and studies. The behaviour of the physical multirotor was evaluated with a test bench designed to isolate the rotation about one single body axis at a time. The data of the experimental tests were gathered in real time using a custom Matlab code and several indoor tests allowed the “fine tuning” of the controllers gains. Afterwards a portable “ground station” was conceived and realised in adherence with the real scenarios users needs. Several outdoor experimental flights were executed with successful results and the data gathered during the outdoor tests were used to evaluate some key performance indicators as the endurance and the maximum allowable payload mass. Then the fault tolerance of the control system was evaluated simulating and experimenting the loss of one motor; even in this critical condition the system exhibited an acceptable behaviour. The reached project readiness allowed to meet some potential users as the “Turin Fire Department” and to cooperate with them in a simulated emergency. During this event the multirotor was used to gather and transmit real time aerial images for an improved “situation awareness”. Finally the study was extended to more innovative control techniques like the neural networks based ones. Simulations results demonstrated their effectiveness; nevertheless the inherent complexity and the unreliability outside the training ranges could have a catastrophic impact on the airworthiness. This is a factor that cannot be neglected especially in the applications related to flying platforms. Summarising, this research work was addressed mainly to the operating procedures for implementing automatic control algorithms to real platforms. All the design aspects, from the preliminary multirotor configuration choice to the tests in possible real scenarios, were covered obtaining performances comparable with other commercial of-the-shelf platforms

Internal Model Control Tuned Proportional Integral Derivative for Quadrotor Unmanned Aerial Vehicle Dynamic Model

In recent times, there are has been growing substantive attention to the quadrotor Unmanned Aerial Vehicle (UAV) stability control. However, inherent nonlinearity is a major challenge with this control technique, this paper, therefore, developed a PID based Internal Model Control (IMC) method for the dynamic model of quadrotor UAV. The versatility and simplicity of the Proportional-Integral-Derivative (PID) controller enable it to enjoy wide usage and acceptability as stability control methods for the unmanned aerial vehicles. The aim of this paper is to use the PID controller with IMC to control a UAV. The proposed approach - IMC-PID control method -was simulated using MATLAB software and X-plane flight simulator. Thereafter, a comparative analysis of the IMC-PID control method with Chien-Hrones-Reswick, Cohen-coon, and Ziegler Nichols based PID Controllers was done using pitch and altitude as performance metrics. Keywords: Internal Model Control,MATLAB/Simulink, Proportional Integral Derivative, Quadrotor, Unmanned Aerial Vehicle (UAV),X-Plane, DOI: 10.7176/CTI/9-01 Publication date: April 30th 202

Fuzzy Gain-Scheduling PID for UAV Position and Altitude Controllers

Unmanned aerial vehicle (UAV) applications have evolved to a wide range of fields in the last decade. One of the main challenges in autonomous tasks is the UAV stability during maneuvers. Thus, attitude and position control play a crucial role in stabilizing the vehicle in the desired orientation and path. Many control techniques have been developed for this. However, proportional integral derivative (PID) controllers are often used due their structure and efficiency. Despite PID’s good performance, different requirements may be present at different mission stages. The main contribution of this research work is the development of a novel strategy based on a fuzzy-gain scheduling mechanism to adjust the PID controller to stabilize both position and altitude. This control strategy must be effective, simple, and robust to uncertainties and external disturbances. The Robot Operating System (ROS) integrates the proposed system and the flight control unit. The obtained results showed that the proposed approach was successfully applied to the trajectory tracking and revealed a good performance compared to conventional PID and in the presence of noises. In the tests, the position controller was only affected when the altitude error was higher, with an error of 2% lower.publishedVersio

Development of Robust Control Laws for Disturbance Rejection in Rotorcraft UAVs

Inherent stability inside the flight envelope must be guaranteed in order to safely introduce private and commercial UAV systems into the national airspace. The rejection of unknown external wind disturbances offers a challenging task due to the limited available information about the unpredictable and turbulent characteristics of the wind. This thesis focuses on the design, development and implementation of robust control algorithms for disturbance rejection in rotorcraft UAVs. The main focus is the rejection of external disturbances caused by wind influences. Four control algorithms are developed in an effort to mitigate wind effects: baseline nonlinear dynamic inversion (NLDI), a wind rejection extension for the NLDI, NLDI with adaptive artificial neural networks (ANN) augmentation, and NLDI with L1 adaptive control augmentation. A simulation environment is applied to evaluate the performance of these control algorithms under external wind conditions using a Monte Carlo analysis. Outdoor flight test results are presented for the implementation of the baseline NLDI, NLDI augmented with adaptive ANN and NLDI augmented with L1 adaptive control algorithms in a DJI F330 Flamewheel quadrotor UAV system. A set of metrics is applied to compare and evaluate the overall performance of the developed control algorithms under external wind disturbances. The obtained results show that the extended NLDI exhibits undesired characteristics while the augmentation of the baseline NLDI control law with adaptive ANN and L1 output-feedback adaptive control improve the robustness of the translational and rotational dynamics of a rotorcraft UAV in the presence of wind disturbances

Geometric Adaptive Control for a Quadrotor UAV with Wind Disturbance Rejection

This paper presents a geometric adaptive control scheme for a quadrotor unmanned aerial vehicle, where the effects of unknown, unstructured disturbances are mitigated by a multilayer neural network that is adjusted online. The stability of the proposed controller is analyzed with Lyapunov stability theory on the special Euclidean group, and it is shown that the tracking errors are uniformly ultimately bounded with an ultimate bound that can be abridged arbitrarily. A mathematical model of wind disturbance on the quadrotor dynamics is presented, and it is shown that the proposed adaptive controller is capable of rejecting the effects of wind disturbances successfully. These are illustrated by numerical examples