Learning Unmanned Aerial Vehicle Control for Autonomous Target Following

While deep reinforcement learning (RL) methods have achieved unprecedented successes in a range of challenging problems, their applicability has been mainly limited to simulation or game domains due to the high sample complexity of the trial-and-error learning process. However, real-world robotic applications often need a data-efficient learning process with safety-critical constraints. In this paper, we consider the challenging problem of learning unmanned aerial vehicle (UAV) control for tracking a moving target. To acquire a strategy that combines perception and control, we represent the policy by a convolutional neural network. We develop a hierarchical approach that combines a model-free policy gradient method with a conventional feedback proportional-integral-derivative (PID) controller to enable stable learning without catastrophic failure. The neural network is trained by a combination of supervised learning from raw images and reinforcement learning from games of self-play. We show that the proposed approach can learn a target following policy in a simulator efficiently and the learned behavior can be successfully transferred to the DJI quadrotor platform for real-world UAV control

Accurate Tracking of Aggressive Quadrotor Trajectories using Incremental Nonlinear Dynamic Inversion and Differential Flatness

Autonomous unmanned aerial vehicles (UAVs) that can execute aggressive (i.e., high-speed and high-acceleration) maneuvers have attracted significant attention in the past few years. This paper focuses on accurate tracking of aggressive quadcopter trajectories. We propose a novel control law for tracking of position and yaw angle and their derivatives of up to fourth order, specifically, velocity, acceleration, jerk, and snap along with yaw rate and yaw acceleration. Jerk and snap are tracked using feedforward inputs for angular rate and angular acceleration based on the differential flatness of the quadcopter dynamics. Snap tracking requires direct control of body torque, which we achieve using closed-loop motor speed control based on measurements from optical encoders attached to the motors. The controller utilizes incremental nonlinear dynamic inversion (INDI) for robust tracking of linear and angular accelerations despite external disturbances, such as aerodynamic drag forces. Hence, prior modeling of aerodynamic effects is not required. We rigorously analyze the proposed control law through response analysis, and we demonstrate it in experiments. The controller enables a quadcopter UAV to track complex 3D trajectories, reaching speeds up to 12.9 m/s and accelerations up to 2.1g, while keeping the root-mean-square tracking error down to 6.6 cm, in a flight volume that is roughly 18 m by 7 m and 3 m tall. We also demonstrate the robustness of the controller by attaching a drag plate to the UAV in flight tests and by pulling on the UAV with a rope during hover.Comment: To be published in IEEE Transactions on Control Systems Technology. Revision: new set of experiments at increased speed (up to 12.9 m/s), updated controller design using quaternion representation, new video available at https://youtu.be/K15lNBAKDC

UAV Model-based Flight Control with Artificial Neural Networks: A Survey

Model-Based Control (MBC) techniques have dominated flight controller designs for Unmanned Aerial Vehicles (UAVs). Despite their success, MBC-based designs rely heavily on the accuracy of the mathematical model of the real plant and they suffer from the explosion of complexity problem. These two challenges may be mitigated by Artificial Neural Networks (ANNs) that have been widely studied due to their unique features and advantages in system identification and controller design. Viewed from this perspective, this survey provides a comprehensive literature review on combined MBC-ANN techniques that are suitable for UAV flight control, i.e., low-level control. The objective is to pave the way and establish a foundation for efficient controller designs with performance guarantees. A reference template is used throughout the survey as a common basis for comparative studies to fairly determine capabilities and limitations of existing research. The end-result offers supported information for advantages, disadvantages and applicability of a family of relevant controllers to UAV prototypes

LQ Servo control design with Kalman filter for a quadrotor UAV

This paper deals with reference signal tracking control of a quadrotor UAV (Unmanned Aerial Vehicle). Before controller design, a nonlinear simulation model is needed, to be the base of design and the first testbed for the resulted controllers. The next step should be the linearization of the nonlinear model in hovering mode, and the reduction of the resulted linear model. The reduced linear model is controllable and observable. The control goal was to track a spatial trajectory with the helicopter center of gravity. For this purpose, an LQ Servo controller (with double integrator) was designed, augmented with a Kalman filter state observer. The resultant controller provided good tracking performance for a slowly varying reference signal, also on the nonlinear model! After the transient response, the tracking error was below 1 cm which provides safe handling even in indoor applications. The time of transients was approximately 4 seconds which is acceptable

Real-time UAV Complex Missions Leveraging Self-Adaptive Controller with Elastic Structure

The expectation of unmanned air vehicles (UAVs) pushes the operation environment to narrow spaces, where the systems may fly very close to an object and perform an interaction. This phase brings the variation in UAV dynamics: thrust and drag coefficient of the propellers might change under different proximity. At the same time, UAVs may need to operate under external disturbances to follow time-based trajectories. Under these challenging conditions, a standard controller approach may not handle all missions with a fixed structure, where there may be a need to adjust its parameters for each different case. With these motivations, practical implementation and evaluation of an autonomous controller applied to a quadrotor UAV are proposed in this work. A self-adaptive controller based on a composite control scheme where a combination of sliding mode control (SMC) and evolving neuro-fuzzy control is used. The parameter vector of the neuro-fuzzy controller is updated adaptively based on the sliding surface of the SMC. The autonomous controller possesses a new elastic structure, where the number of fuzzy rules keeps growing or get pruned based on bias and variance balance. The interaction of the UAV is experimentally evaluated in real time considering the ground effect, ceiling effect and flight through a strong fan-generated wind while following time-based trajectories.Comment: 18 page

Design and Implementation of an Artificial Neural Network Controller for Quadrotor Flight in Confined Environment

Quadrotors offer practical solutions for many applications, such as emergency rescue, surveillance, military operations, videography and many more. For this reason, they have recently attracted the attention of research and industry. Even though they have been intensively studied, quadrotors still suffer from some challenges that limit their use, such as trajectory measurement, attitude estimation, obstacle avoidance, safety precautions, and land cybersecurity. One major problem is flying in a confined environment, such as closed buildings and tunnels, where the aerodynamics around the quadrotor are affected by close proximity objects, which result in tracking performance deterioration, and sometimes instability. To address this problem, researchers followed three different approaches; the Modeling approach, which focuses on the development of a precise dynamical model that accounts for the different aerodynamic effects, the Sensor Integration approach, which focuses on the addition of multiple sensors to the quadrotor and applying algorithms to stabilize the quadrotor based on their measurements, and the Controller Design approach, which focuses on the development of an adaptive and robust controller. In this research, a learning controller is proposed as a solution for the issue of quadrotor trajectory control in confined environments. This controller utilizes Artificial Neural Networks to adjust for the unknown aerodynamics on-line. A systematic approach for controller design is developed, so that, the approach could be followed for the development of controllers for other nonlinear systems of similar form. One goal for this research is to develop a global controller that could be applied to any quadrotor with minimal adjustment. A novel Artificial Neural Network structure is presented that increases learning efficiency and speed. In addition, a new learning algorithm is developed for the Artificial Neural Network, when utilized with the developed controller. Simulation results for the designed controller when applied to the Qball-X4 quadrotor are presented that show the effectiveness of the proposed Artificial Neural Network structure and the developed learning algorithm in the presence of variety of different unknown aerodynamics. These results are confirmed with real time experimentation, as the developed controller was successfully applied to Quanser’s Qball-X4 quadrotor for the flight control in confined environment. The practical challenges associated with the application of such a controller for quadrotor flight in confined environment are analyzed and adequately resolved to achieve an acceptable tracking performance