Optimal control of a helicopter unmanned aerial vehicle (UAV)

This thesis addresses optimal control of a helicopter unmanned aerial vehicle (UAV). Helicopter UAVs may be widely used for both military and civilian operations. Because these helicopters are underactuated nonlinear mechanical systems, high-performance controller design for them presents a challenge. This thesis presents an optimal controller design via both state and output feedback for trajectory tracking of a helicopter UAV using a neural network (NN). The state and output-feedback control system utilizes the backstepping methodology, employing kinematic and dynamic controllers while the output feedback approach uses an observer in addition to these controllers. The online approximator-based dynamic controller learns the Hamilton-Jacobi-Bellman (HJB) equation in continuous time and calculates the corresponding optimal control input to minimize the HJB equation forward-in-time. Optimal tracking is accomplished with a single NN utilized for cost function approximation. The overall closed-loop system stability is demonstrated using Lyapunov analysis. Simulation results are provided to demonstrate the effectiveness of the proposed control design for trajectory tracking. A description of the hardware for confirming the theoretical approach, and a discussion of material pertaining to the algorithms used and methods employed specific to the hardware implementation is also included. Additional attention is devoted to challenges in implementation as well as to opportunities for further research in this field. This thesis is presented in the form of two papers --Abstract, page iv

Online Trimming of Flight Dynamics Models Using the 2Simulate Realtime Simulation Framework

Real-time simulation requires the definition of scenarios, which may be defined on the ground or in the air. In-flight scenarios usually begin in a trimmed flight condition, so that the simulation model is in a steady state. At the German Aerospace Center (DLR), the Air Vehicle Simulator (AVES) is operated for flight research and simulator studies. Research simulators require a high degree of flexibility to modify scenarios. Moreover, the flexibility to modify the simulation model poses a challenge to both the user interface and the trim procedure. Therefore, an adjustable user interface is required to allow the user to run the trim calculation online for immediate operation in a simulation run. This paper presents an implementation of a generic approach to trim flight dynamic models for use in research flight simulators. This implies both the implementation of a graphical user interface as well as the trim algorithm and their interaction. An example is demonstrated for a helicopter model used for research activities at DLR

Neuro-Optimal Control of Helicopter UAVs

Helicopter UAVs can be extensively used for military missions as well as in civil operations, ranging from multi- role combat support and search and rescue, to border surveillance and forest fire monitoring. Helicopter UAVs are underactuated nonlinear mechanical systems with correspondingly challenging controller designs. This paper presents an optimal controller design for the regulation and vertical tracking of an underactuated helicopter using an adaptive critic neural network framework. The online approximator-based controller learns the infinite- horizoncontinuous-time Hamilton-Jacobi-Bellman(HJB) equation and then calculates the corresponding optimal control input that minimizes the HJB equation forward-in-time. In the proposed technique, optimal regulation and vertical tracking is accomplished by a single neural network (NN) with a second NN necessary for the virtual controller. Both of the NNs are tuned online using novel weight update laws. Simulation results are included to demonstrate the effectiveness of the proposed control design in hovering applications. (Unmanned Systems Technology XIII, edited by Douglas W. Gage, Charles M. Shoemaker, Robert E. Karlsen, Grant R. Gerhart

Optimal aeroelastic trim for rotorcraft with constrained, non-unique trim solutions

New rotorcraft configurations are emerging, such as the optimal speed helicopter and slowed-rotor compound helicopter which, due to variable rotor speed and redundant lifting components, have non-unique trim solution spaces. The combination of controls and rotor speed that produce the best steady-flight condition is sought among all the possible solutions. This work develops the concept of optimal rotorcraft trim and explores its application to advanced rotorcraft configurations with non-unique, constrained trim solutions. The optimal trim work is based on the nonlinear programming method of the generalized reduced gradient (GRG) and is integrated into a multi-body, comprehensive aeroelastic rotorcraft code. In addition to the concept of optimal trim, two further developments are presented that allow the extension of optimal trim to rotorcraft with rotors that operate over a wide range of rotor speeds. The first is the concept of variable rotor speed trim with special application to rotors operating in steady autorotation. The technique developed herein treats rotor speed as a trim variable and uses a Newton-Raphson iterative method to drive the rotor speed to zero average torque simultaneously with other dependent trim variables. The second additional contribution of this thesis is a novel way to rapidly approximate elastic rotor blade stresses and strains in the aeroelastic trim analysis for structural constraints. For rotors that operate over large angular velocity ranges, rotor resonance and increased flapping conditions are encountered that can drive the maximum cross-sectional stress and strain to levels beyond endurance limits; such conditions must be avoided. The method developed herein captures the maximum cross-sectional stress/strain based on the trained response of an artificial neural network (ANN) surrogate as a function of 1-D beam forces and moments. The stresses/strains are computed simultaneously with the optimal trim and are used as constraints in the optimal trim solution. Finally, an optimal trim analysis is applied to a high-speed compound gyroplane configuration, which has two distinct rotor speed control methods, with the purpose of maximizing the vehicle cruise efficiency while maintaining rotor blade strain below endurance limit values.Ph.D.Committee Chair: Dimitri N. Mavris; Committee Co-Chair: Daniel P Schrage; Committee Member: David A. Peters; Committee Member: Dewey H. Hodges; Committee Member: J.V.R. Prasa

Continuous-Time Reinforcement Learning: New Design Algorithms with Theoretical Insights and Performance Guarantees

Continuous-time nonlinear optimal control problems hold great promise in real-world applications. After decades of development, reinforcement learning (RL) has achieved some of the greatest successes as a general nonlinear control design method. However, a recent comprehensive analysis of state-of-the-art continuous-time RL (CT-RL) methods, namely, adaptive dynamic programming (ADP)-based CT-RL algorithms, reveals they face significant design challenges due to their complexity, numerical conditioning, and dimensional scaling issues. Despite advanced theoretical results, existing ADP CT-RL synthesis methods are inadequate in solving even small, academic problems. The goal of this work is thus to introduce a suite of new CT-RL algorithms for control of affine nonlinear systems. Our design approach relies on two important factors. First, our methods are applicable to physical systems that can be partitioned into smaller subproblems. This constructive consideration results in reduced dimensionality and greatly improved intuitiveness of design. Second, we introduce a new excitation framework to improve persistence of excitation (PE) and numerical conditioning performance via classical input/output insights. Such a design-centric approach is the first of its kind in the ADP CT-RL community. In this paper, we progressively introduce a suite of (decentralized) excitable integral reinforcement learning (EIRL) algorithms. We provide convergence and closed-loop stability guarantees, and we demonstrate these guarantees on a significant application problem of controlling an unstable, nonminimum phase hypersonic vehicle (HSV)

Data-Driven Robust Control of Unknown MIMO Nonlinear System Subject to Input Saturations and Disturbances

This paper presented a new data-driven robust control scheme for unknown nonlinear systems in the presence of input saturation and external disturbances. According to the input and output data of the nonlinear system, a recurrent neural network (RNN) data-driven model is established to reconstruct the dynamics of the nonlinear system. An adaptive output-feedback controller is developed to approximate the unknown disturbances and a novel input saturation compensation method is used to attenuate the effect of the input saturation. Under the proposed adaptive control scheme, the uniformly ultimately bounded convergence of all the signals of the closed-loop nonlinear system is guaranteed via Lyapunov analysis. The simulation results are given to show the effectiveness of the proposed data-driven robust controller

Lyapunov based optimal control of a class of nonlinear systems

Optimal control of nonlinear systems is in fact difficult since it requires the solution to the Hamilton-Jacobi-Bellman (HJB) equation which has no closed-form solution. In contrast to offline and/or online iterative schemes for optimal control, this dissertation in the form of five papers focuses on the design of iteration free, online optimal adaptive controllers for nonlinear discrete and continuous-time systems whose dynamics are completely or partially unknown even when the states not measurable. Thus, in Paper I, motivated by homogeneous charge compression ignition (HCCI) engine dynamics, a neural network-based infinite horizon robust optimal controller is introduced for uncertain nonaffine nonlinear discrete-time systems. First, the nonaffine system is transformed into an affine-like representation while the resulting higher order terms are mitigated by using a robust term. The optimal adaptive controller for the affinelike system solves HJB equation and identifies the system dynamics provided a target set point is given. Since it is difficult to define the set point a priori in Paper II, an extremum seeking control loop is designed while maximizing an uncertain output function. On the other hand, Paper III focuses on the infinite horizon online optimal tracking control of known nonlinear continuous-time systems in strict feedback form by using state and output feedback by relaxing the initial admissible controller requirement. Paper IV applies the optimal controller from Paper III to an underactuated helicopter attitude and position tracking problem. In Paper V, the optimal control of nonlinear continuous-time systems in strict feedback form from Paper III is revisited by using state and output feedback when the internal dynamics are unknown. Closed-loop stability is demonstrated for all the controller designs developed in this dissertation by using Lyapunov analysis --Abstract, page iv