Deep Reinforcement Learning Attitude Control of Fixed-Wing UAVs Using Proximal Policy Optimization

Contemporary autopilot systems for unmanned aerial vehicles (UAVs) are far more limited in their flight envelope as compared to experienced human pilots, thereby restricting the conditions UAVs can operate in and the types of missions they can accomplish autonomously. This paper proposes a deep reinforcement learning (DRL) controller to handle the nonlinear attitude control problem, enabling extended flight envelopes for fixed-wing UAVs. A proof-of-concept controller using the proximal policy optimization (PPO) algorithm is developed, and is shown to be capable of stabilizing a fixed-wing UAV from a large set of initial conditions to reference roll, pitch and airspeed values. The training process is outlined and key factors for its progression rate are considered, with the most important factor found to be limiting the number of variables in the observation vector, and including values for several previous time steps for these variables. The trained reinforcement learning (RL) controller is compared to a proportional-integral-derivative (PID) controller, and is found to converge in more cases than the PID controller, with comparable performance. Furthermore, the RL controller is shown to generalize well to unseen disturbances in the form of wind and turbulence, even in severe disturbance conditions.Comment: 11 pages, 3 figures, 2019 International Conference on Unmanned Aircraft Systems (ICUAS

Aerial Vehicles

This book contains 35 chapters written by experts in developing techniques for making aerial vehicles more intelligent, more reliable, more flexible in use, and safer in operation.It will also serve as an inspiration for further improvement of the design and application of aeral vehicles. The advanced techniques and research described here may also be applicable to other high-tech areas such as robotics, avionics, vetronics, and space

Optimal trajectory generation with DMOC versus NTG : application to an underwater glider and a JPL aerobot.

Optimal trajectory generation is an essential part for robotic explorers to execute the total exploration of deep oceans or outer space planets while curiosity of human and technology advancements of society both require robots to search for unknown territories efficiently and safely. As one of state-of-the-art optimal trajectory generation methodologies, Nonlinear Trajectory Generation (NTG) combines with B-spline, nonlinear programming, differential flatness technique to generate optimal trajectories for modelled mechanical systems. While Discrete Mechanics and Optimal Control (DMOC) is a newly proposed optimal control method for mechanical systems, it is based on direct discretization of Lagrange-d\u27Alembert principle. In this dissertation, NTG is utilized to generate trajectories for an underwater glider with a 3D B-spline ocean current model. The optimal trajectories are corresponding well with the Lagrangian Coherent Structures (LCS). Then NTG is utilized to generate 3D opportunistic trajectories for a JPL (Jet Propulsion Laboratory) Aerobot by taking advantage of wind velocity. Since both DMOC and NTG are methods which can generate optimal trajectories for mechanical systems, their differences in theory and application are investigated. In a simple ocean current example and a more complex ocean current model, DMOC with discrete Euler-Lagrange constraints generates local optimal solutions with different initial guesses while NTG is also generating similar solutions with more computation time and comparable energy consumption. DMOC is much easier to implement than NTG because in order to generate good solutions in NTG, its variables need to be correctly defined as B-spline variables with rightly-chosen orders. Finally, the MARIT (Multiple Air Robotics Indoor Testbed) is established with a Vicon 8i motion capture system. Six Mcam 2 cameras connected with a datastation are able to track real-time coordinates of a draganflyer helicopter. This motion capture system establishes a good foundation for future NTG and DMOC algorithms verifications

Model predictive cooperative localization control of multiple UAVs using potential function sensor constraints: a workflow to create sensor constraint based potential functions for the control of cooperative localization scenarios with mobile robots.

The global localization of multiple mobile robots can be achieved cost efficiently by localizing one robot globally and the others in relation to it using local sensor data. However, the drawback of this cooperative localization is the requirement of continuous sensor information. Due to a limited sensor perception space, the tracking task to continuously maintain this sensor information is challenging. To address this problem, this contribution is presenting a model predictive control (MPC) approach for such cooperative localization scenarios. In particular, the present work shows a novel workflow to describe sensor limitations with the help of potential functions. In addition, a compact motion model for multi-rotor drones is introduced to achieve MPC real-time capability. The effectiveness of the presented approach is demonstrated in a numerical simulation, an experimental indoor scenario with two quadrotors as well as multiple indoor scenarios of a quadrotor obstacle evasion maneuver

Hardware, Software, and Low-Level Control Scheme Development for a Real-Time Autonomous Rover

The objective of this research is to develop a low-cost autonomous rover platform for experiments in autonomous navigation. This thesis describes the design, development, and testing of an autonomous rover platform, based on the commercial, off-the-shelf Tamiya TXT-1 radio controlled vehicle. This vehicle is outfitted with an onboard computer based on the Mini-ITX architecture and an array of sensors for localization and obstacle avoidance, and programmed with Matlab/SimulinkRTM Real-Time Workshop (RTW) utilizing the Linux Real-Time Application Interface (RTAI) operating system.;First, a kinematic model is developed and verified for the rover. Then a proportional-integral-derivative (PID) feedback controller is developed for translational and rotational velocity regulation. Finally, a hybrid navigation controller is developed combining a potential field controller and an obstacle avoidance controller for waypoint tracking.;Experiments are performed to verify the functionality of the kinematic model and the PID velocity controller, and to demonstrate the capabilities of the hybrid navigation controller. These experiments prove that the rover is capable of successfully navigating in an unknown indoor environment. Suggestions for future research include the integration of additional sensors for localization and creation of multiple platforms for autonomous coordination experiments

Optimal fault-tolerant flight control for aircraft with actuation impairments

Current trends towards greater complexity and automation are leaving modern technological systems increasingly vulnerable to faults. Without proper action, a minor error may lead to devastating consequences. In flight control, where the controllability and dynamic stability of the aircraft primarily rely on the control surfaces and engine thrust, faults in these effectors result in a higher extent of risk for these aspects. Moreover, the operation of automatic flight control would be suddenly disturbed. To address this problem, different methodologies of designing optimal flight controllers are presented in this thesis. For multiple-input multiple-output (MIMO) systems, the feedback optimal control is a prominent technique that solves a multi-objective cost function, which includes, for instance, tracking requirements and control energy minimisation. The first proposed method is based on a linear quadratic regulator (LQR) control law augmented with a fault-compensation scheme. This fault-tolerant system handles the situation in an adaptive way by solving the optimisation cost function and considering fault information, while assuming an effective fault detection system is available. The developed scheme was tested in a six-degrees-of-freedom nonlinear environment to validate the linear-based controller. Results showed that this fault tolerant control (FTC) strategy managed to handle high magnitudes of the actuator’s loss of effciency faults. Although the rise time of aircraft response became slower, overshoot and settling errors were minimised, and the stability of the aircraft was maintained. Another FTC approach has been developed utilising the features of controller robustness against the system parametric uncertainties, without the need for reconfiguration or adaptation. Two types of control laws were established under this scheme, the H∞ and µ-synthesis controllers. Both were tested in a nonlinear environment for three points in the flight envelope: ascending, cruising, and descending. The H∞ controller maintained the requirements in the intact case; while in fault, it yielded non-robust high-frequency control surface deflections. The µ-synthesis, on the other hand, managed to handle the constraints of the system and accommodate faults reaching 30% loss of effciency in actuation. The final approach is based on the control allocation technique. It considers the tracking requirements and the constraints of the actuators in the design process. To accommodate lock-in-place faults, a new control effort redistribution scheme was proposed using the fuzzy logic technique, assuming faults are provided by a fault detection system. The results of simulation testing on a Boeing 747 multi-effector model showed that the system managed to handle these faults and maintain good tracking and stability performance, with some acceptable degradation in particular fault scenarios. The limitations of the controller to handle a high degree of faults were also presented

Multi-Agent Systems

This Special Issue ""Multi-Agent Systems"" gathers original research articles reporting results on the steadily growing area of agent-oriented computing and multi-agent systems technologies. After more than 20 years of academic research on multi-agent systems (MASs), in fact, agent-oriented models and technologies have been promoted as the most suitable candidates for the design and development of distributed and intelligent applications in complex and dynamic environments. With respect to both their quality and range, the papers in this Special Issue already represent a meaningful sample of the most recent advancements in the field of agent-oriented models and technologies. In particular, the 17 contributions cover agent-based modeling and simulation, situated multi-agent systems, socio-technical multi-agent systems, and semantic technologies applied to multi-agent systems. In fact, it is surprising to witness how such a limited portion of MAS research already highlights the most relevant usage of agent-based models and technologies, as well as their most appreciated characteristics. We are thus confident that the readers of Applied Sciences will be able to appreciate the growing role that MASs will play in the design and development of the next generation of complex intelligent systems. This Special Issue has been converted into a yearly series, for which a new call for papers is already available at the Applied Sciences journal’s website: https://www.mdpi.com/journal/applsci/special_issues/Multi-Agent_Systems_2019

NASA thesaurus. Volume 2: Access vocabulary

The access vocabulary, which is essentially a permuted index, provides access to any word or number in authorized postable and nonpostable terms. Additional entries include postable and nonpostable terms, other word entries and pseudo-multiword terms that are permutations of words that contain words within words. The access vocabulary contains almost 42,000 entries that give increased access to the hierarchies in Volume 1 - Hierarchical Listing

NASA thesaurus. Volume 2: Access vocabulary

The Access Vocabulary, which is essentially a permuted index, provides access to any word or number in authorized postable and nonpostable terms. Additional entries include postable and nonpostable terms, other word entries, and pseudo-multiword terms that are permutations of words that contain words within words. The Access Vocabulary contains 40,738 entries that give increased access to the hierarchies in Volume 1 - Hierarchical Listing