Reinforcement Learning for UAV Attitude Control

Autopilot systems are typically composed of an "inner loop" providing stability and control, while an "outer loop" is responsible for mission-level objectives, e.g. way-point navigation. Autopilot systems for UAVs are predominately implemented using Proportional, Integral Derivative (PID) control systems, which have demonstrated exceptional performance in stable environments. However more sophisticated control is required to operate in unpredictable, and harsh environments. Intelligent flight control systems is an active area of research addressing limitations of PID control most recently through the use of reinforcement learning (RL) which has had success in other applications such as robotics. However previous work has focused primarily on using RL at the mission-level controller. In this work, we investigate the performance and accuracy of the inner control loop providing attitude control when using intelligent flight control systems trained with the state-of-the-art RL algorithms, Deep Deterministic Gradient Policy (DDGP), Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO). To investigate these unknowns we first developed an open-source high-fidelity simulation environment to train a flight controller attitude control of a quadrotor through RL. We then use our environment to compare their performance to that of a PID controller to identify if using RL is appropriate in high-precision, time-critical flight control.Comment: 13 pages, 9 figure

Skyport airframe: design and manufacturing

Many rural areas of developing countries lack the necessary transportation infrastructure to have reliable access to basic needs. This is particularly true for medical supplies. To combat the issue of insufficient access to vaccines in developing areas, the SkyPort project has developed the SkyPort UAV (Unmanned Aerial Vehicle). The SkyPort UAV has the vertical takeoff and landing (VTOL) capabilities of a quadcopter, as well as the efficient, sustained flight of a fixed-wing aircraft. It provides a cheaper, quicker, and safer delivery method than existing alternatives for vaccines in areas that lack a reliable transportation infrastructure. The role of the SkyPort Airframe Design Team was to design and build the primary support structure of the UAV, which will house the payload, controls, and propulsion systems being designed by the other two SkyPort teams. The airframe consists of a lightweight and durable fuselage, wing, tail, and framing subsystems and it is designed to be modular so that parts are easy to replace and require minimal maintenance. Primary materials used in construction were foam, carbon fiber, and aluminum. Testing of the frame yielded a weight of 8.63 kg, minimum foam strength of 1.70 MPa, and a minimum factor of safety of 16 for the structural members of the frame. Although the weight of the airframe is higher than the desired weight, this was necessary in order to satisfy the strength requirements and protect sensitive electrical components during initial flight tests. In the future, this extra weight could be decreased by using less carbon fiber, lower density foam, smaller, lighter material for the structural members, or smaller fasteners

Online Deep Learning for Improved Trajectory Tracking of Unmanned Aerial Vehicles Using Expert Knowledge

This work presents an online learning-based control method for improved trajectory tracking of unmanned aerial vehicles using both deep learning and expert knowledge. The proposed method does not require the exact model of the system to be controlled, and it is robust against variations in system dynamics as well as operational uncertainties. The learning is divided into two phases: offline (pre-)training and online (post-)training. In the former, a conventional controller performs a set of trajectories and, based on the input-output dataset, the deep neural network (DNN)-based controller is trained. In the latter, the trained DNN, which mimics the conventional controller, controls the system. Unlike the existing papers in the literature, the network is still being trained for different sets of trajectories which are not used in the training phase of DNN. Thanks to the rule-base, which contains the expert knowledge, the proposed framework learns the system dynamics and operational uncertainties in real-time. The experimental results show that the proposed online learning-based approach gives better trajectory tracking performance when compared to the only offline trained network.Comment: corrected version accepted for ICRA 201

Design of a quadcopter to work at high temperatures

The project develops the design of a quadcopter to work within industrial plants which can be found even at 80 degrees Celsius. These plants should be checked as a way of detecting faults or cracks to prevent other serious incidents that may arise. Both the whole building as well as industrial machinery, which are inside the plant, should be inspected without the need to wait until the infrastructure is fully cooled down. Both external mechanical defense to get close to surfaces, adapting to customer specifications, as well as mechanical and electronic components in the multicopter are designed. It shall support all the requested temperature at least 80 degrees.El proyecto desarrolla el diseño de un cuadricóptero para trabajar dentro de plantas industriales que se pueden encontrar hasta una temperatura de 80 grados. Estos edificios deben ser revisados continuamente como una forma de detectar fallas o grietas que puedan evitar otros incidentes más graves que pudieran surgir. Todo el edificio, así como la maquinaria industrial que están dentro de la planta, deben ser inspeccionados sin la necesidad de esperar hasta que la infraestructura está totalmente enfriada ...Ingeniería Industria

Linear Hamilton Jacobi Bellman Equations in High Dimensions

The Hamilton Jacobi Bellman Equation (HJB) provides the globally optimal solution to large classes of control problems. Unfortunately, this generality comes at a price, the calculation of such solutions is typically intractible for systems with more than moderate state space size due to the curse of dimensionality. This work combines recent results in the structure of the HJB, and its reduction to a linear Partial Differential Equation (PDE), with methods based on low rank tensor representations, known as a separated representations, to address the curse of dimensionality. The result is an algorithm to solve optimal control problems which scales linearly with the number of states in a system, and is applicable to systems that are nonlinear with stochastic forcing in finite-horizon, average cost, and first-exit settings. The method is demonstrated on inverted pendulum, VTOL aircraft, and quadcopter models, with system dimension two, six, and twelve respectively.Comment: 8 pages. Accepted to CDC 201

Autonomous moving target-tracking for a UAV quadcopter based on fuzzy-PI.

Moving target-tracking is an attractive application for quadcopters and a very challenging, complicated field of research due to the complex dynamics of a quadcopter and the varying speed of the moving target with time. For this reason, various control algorithms have been developed to track a moving target using a camera. In this paper, a Fuzzy-PI controller is developed to adjust the parameters of the PI controller using the position and change of position data as input. The proposed controller is compared to a gain-scheduled PID controller instead of the typical PID controller. To verify the performance of the developed system and distinguish which one has better performance, several experiments of a quadcopter tracking a moving target are conducted under the varying speed of the moving target, indoor and outdoor and during day and night. The obtained results indicate that the proposed controller works well for tracking a moving target under different scenarios, especially during night