Design of Controllers for a Multiple Input Multiple Output System

A method of controller design for multiple input multiple output (MIMO) system is needed that will not give the high order controllers of modern control theory but will be more systematic than the “ad hoc” method. The objective of this method of design for multiple input multiple output systems is to find a controller of fixed order with performance specifications taken into consideration. An inner approximation of the stabilizing set is found through the algorithm discussed in Keel and Bhattacharyya’s "Fixed order multivariable controller synthesis: A new algorithm." The set satisfying the performance is then approximated through one of two algorithms; a hybrid of two optimization algorithms or the grid algorithm found in Lampton’s "Reinforcement Learning of a Morphing Airfoil-Policy and Discrete Learning Analysis." The method is then applied to five models of four aircraft; Commander 700, X-29, X-38, and F-5A using controllers of first and second orders

Structural and Aerodynamic Interaction Computational Tool for Highly Reconfigurable Wings

Morphing air vehicles enable more efficient and capable multi-role aircraft by adapting their shape to reach an ideal configuration in an ever-changing environment. Morphing capability is envisioned to have a profound impact on the future of the aerospace industry, and a reconfigurable wing is a significant element of a morphing aircraft. This thesis develops two tools for analyzing wing configurations with multiple geometric degrees-of-freedom: the structural tool and the aerodynamic and structural interaction tool. Linear Space Frame Finite Element Analysis with Euler-Bernoulli beam theory is used to develop the structural analysis morphing tool for modeling a given wing structure with variable geometric parameters including wing span, aspect ratio, sweep angle, dihedral angle, chord length, thickness, incidence angle, and twist angle. The structural tool is validated with linear Euler-Bernoulli beam models using a commercial finite element software program, and the tool is shown to match within 1% compared to all test cases. The verification of the structural tool uses linear and nonlinear Timoshenko beam models, 3D brick element wing models at various sweep angles, and a complex wing structural model of an existing aircraft. The beam model verification demonstrated the tool matches the Timoshenko models within 3%, but the comparisons to complex wing models show the limitations of modeling a wing structure using beam elements. The aerodynamic and structural interaction tool is developed to integrate a constant strength source doublet panel method aerodynamic tool, developed externally to this work, with the structural tool. The load results provided by the aerodynamic tool are used as inputs to the structural tool, giving a quasi-static aeroelastically deflected wing shape. An iterative version of the interaction tool uses the deflected wing shape results from the structural tool as new inputs for the aerodynamic tool in order to investigate the geometric convergence of an aeroelastically deflected wing shape. The findings presented in this thesis show that geometric convergence of the deflected wing shape is not attained using the chosen iterative method, but other potential methods are proposed for future work. The tools presented in the thesis are capable of modeling a wide range of wing configurations, and they may ultimately be utilized by Machine Learning algorithms to learn the ideal wing configuration for given flight conditions and develop control laws for a flyable morphing air vehicle

Human-in-the-Loop Methods for Data-Driven and Reinforcement Learning Systems

Recent successes combine reinforcement learning algorithms and deep neural networks, despite reinforcement learning not being widely applied to robotics and real world scenarios. This can be attributed to the fact that current state-of-the-art, end-to-end reinforcement learning approaches still require thousands or millions of data samples to converge to a satisfactory policy and are subject to catastrophic failures during training. Conversely, in real world scenarios and after just a few data samples, humans are able to either provide demonstrations of the task, intervene to prevent catastrophic actions, or simply evaluate if the policy is performing correctly. This research investigates how to integrate these human interaction modalities to the reinforcement learning loop, increasing sample efficiency and enabling real-time reinforcement learning in robotics and real world scenarios. This novel theoretical foundation is called Cycle-of-Learning, a reference to how different human interaction modalities, namely, task demonstration, intervention, and evaluation, are cycled and combined to reinforcement learning algorithms. Results presented in this work show that the reward signal that is learned based upon human interaction accelerates the rate of learning of reinforcement learning algorithms and that learning from a combination of human demonstrations and interventions is faster and more sample efficient when compared to traditional supervised learning algorithms. Finally, Cycle-of-Learning develops an effective transition between policies learned using human demonstrations and interventions to reinforcement learning. The theoretical foundation developed by this research opens new research paths to human-agent teaming scenarios where autonomous agents are able to learn from human teammates and adapt to mission performance metrics in real-time and in real world scenarios.Comment: PhD thesis, Aerospace Engineering, Texas A&M (2020). For more information, see https://vggoecks.com

Reinforcement Learning of a Morphing Airfoil-Policy and Discrete Learning Analysis

Casting the problem of morphing a microair vehicle as a reinforcement-learning problem to achieve desired tasks or performance is a candidate approach for handling many of the unique challenges associated with such small aircraft. This paper presents an early stage in the development of learning how and when to morph a micro air vehicle by develop-ing an episodic unsupervised learning algorithm using the Q-learning method to learn the shape and shape change policy of a single morphing airfoil. Reinforcement is addressed by reward functions based on airfoil properties, such as lift coefficient, representing desired performance for specified flight conditions. The reinforcement learning as it is applied to morphing is integrated with a computational model of an airfoil. The methodology is demon-strated with numerical examples of an NACA type airfoil that autonomously morphs in two degrees of freedom, thickness and camber, to a shape that corresponds to specified goal requirements. Because of the continuous nature of the thickness and camber of the airfoil, this paper addresses the convergence of the learning algorithm given several discretizations. Convergence is also analyzed with three candidate policies: 1) a fully random exploration policy, 2) a policy annealing from random exploration to exploitation, and 3) an annealin

Discretization and Approximation Methods for Reinforcement Learning of Highly Reconfigurable Systems

There are a number of techniques that are used to solve reinforcement learning problems, but very few that have been developed for and tested on highly reconfigurable systems cast as reinforcement learning problems. Reconfigurable systems refers to a vehicle (air, ground, or water) or collection of vehicles that can change its geometrical features, i.e. shape or formation, to perform tasks that the vehicle could not otherwise accomplish. These systems tend to be optimized for several operating conditions, and then controllers are designed to reconfigure the system from one operating condition to another. Q-learning, an unsupervised episodic learning technique that solves the reinforcement learning problem, is an attractive control methodology for reconfigurable systems. It has been successfully applied to a myriad of control problems, and there are a number of variations that were developed to avoid or alleviate some limitations in earlier version of this approach. This dissertation describes the development of three modular enhancements to the Q-learning algorithm that solve some of the unique problems that arise when working with this class of systems, such as the complex interaction of reconfigurable parameters and computationally intensive models of the systems. A multi-resolution state-space discretization method is developed that adaptively rediscretizes the state-space by progressively finer grids around one or more distinct Regions Of Interest within the state or learning space. A genetic algorithm that autonomously selects the basis functions to be used in the approximation of the action-value function is applied periodically throughout the learning process. Policy comparison is added to monitor the state of the policy encoded in the action-value function to prevent unnecessary episodes at each level of discretization. This approach is validated on several problems including an inverted pendulum, reconfigurable airfoil, and reconfigurable wing. Results show that the multi-resolution state-space discretization method reduces the number of state-action pairs, often by an order of magnitude, required to achieve a specific goal and the policy comparison prevents unnecessary episodes once the policy has converged to a usable policy. Results also show that the genetic algorithm is a promising candidate for the selection of basis functions for function approximation of the action-value function