Optimal adaptive control of time-delay dynamical systems with known and uncertain dynamics

Delays are found in many industrial pneumatic and hydraulic systems, and as a result, the performance of the overall closed-loop system deteriorates unless they are explicitly accounted. It is also possible that the dynamics of such systems are uncertain. On the other hand, optimal control of time-delay systems in the presence of known and uncertain dynamics by using state and output feedback is of paramount importance. Therefore, in this research, a suite of novel optimal adaptive control (OAC) techniques are undertaken for linear and nonlinear continuous time-delay systems in the presence of uncertain system dynamics using state and/or output feedback. First, the optimal regulation of linear continuous-time systems with state and input delays by utilizing a quadratic cost function over infinite horizon is addressed using state and output feedback. Next, the optimal adaptive regulation is extended to uncertain linear continuous-time systems under a mild assumption that the bounds on system matrices are known. Subsequently, the event-triggered optimal adaptive regulation of partially unknown linear continuous time systems with state-delay is addressed by using integral reinforcement learning (IRL). It is demonstrated that the optimal control policy renders asymptotic stability of the closed-loop system provided the linear time-delayed system is controllable and observable. The proposed event-triggered approach relaxed the need for continuous availability of state vector and proven to be zeno-free. Finally, the OAC using IRL neural network based control of uncertain nonlinear time-delay systems with input and state delays is investigated. An identifier is proposed for nonlinear time-delay systems to approximate the system dynamics and relax the need for the control coefficient matrix in generating the control policy. Lyapunov analysis is utilized to design the optimal adaptive controller, derive parameter/weight tuning law and verify stability of the closed-loop system”--Abstract, page iv

Discrete-Time Neural Network Output Feedback Control of Nonlinear Systems in Non-Strict Feedback Form

An adaptive neural network (NN)-based output feedback controller is proposed to deliver a desired tracking performance for a class of discrete-time nonlinear systems, which is represented in non-strict feedback form. The NN backstepping approach is utilized to design the adaptive output feedback controller consisting of: 1) a NN observer to estimate the system states with the input-output data, and 2) two NNs to generate the virtual and actual control inputs, respectively. The non-causal problem in the discrete-time backstepping design is avoided by using the universal NN approximator. The persistence excitation (PE) condition is relaxed both in the NN observer and NN controller design. The uniformly ultimate boundedness (UUB) of the closed-loop tracking error, the state estimation errors and the NN weight estimates is shown

Lyapunov based optimal control of a class of nonlinear systems

Optimal control of nonlinear systems is in fact difficult since it requires the solution to the Hamilton-Jacobi-Bellman (HJB) equation which has no closed-form solution. In contrast to offline and/or online iterative schemes for optimal control, this dissertation in the form of five papers focuses on the design of iteration free, online optimal adaptive controllers for nonlinear discrete and continuous-time systems whose dynamics are completely or partially unknown even when the states not measurable. Thus, in Paper I, motivated by homogeneous charge compression ignition (HCCI) engine dynamics, a neural network-based infinite horizon robust optimal controller is introduced for uncertain nonaffine nonlinear discrete-time systems. First, the nonaffine system is transformed into an affine-like representation while the resulting higher order terms are mitigated by using a robust term. The optimal adaptive controller for the affinelike system solves HJB equation and identifies the system dynamics provided a target set point is given. Since it is difficult to define the set point a priori in Paper II, an extremum seeking control loop is designed while maximizing an uncertain output function. On the other hand, Paper III focuses on the infinite horizon online optimal tracking control of known nonlinear continuous-time systems in strict feedback form by using state and output feedback by relaxing the initial admissible controller requirement. Paper IV applies the optimal controller from Paper III to an underactuated helicopter attitude and position tracking problem. In Paper V, the optimal control of nonlinear continuous-time systems in strict feedback form from Paper III is revisited by using state and output feedback when the internal dynamics are unknown. Closed-loop stability is demonstrated for all the controller designs developed in this dissertation by using Lyapunov analysis --Abstract, page iv

Lyapunov-Based Robust and Adaptive Control Design for nonlinear Uncertain Systems

The control of systems with uncertain nonlinear dynamics is an important field of control science attracting decades of focus. In this dissertation, four different control strategies are presented using sliding mode control, adaptive control, dynamic compensation, and neural network for a nonlinear aeroelastic system with bounded uncertainties and external disturbance. In Chapter 2, partial state feedback adaptive control designs are proposed for two different aeroelastic systems operating in unsteady flow. In Chapter 3, a continuous robust control design is proposed for a class of single input and single output system with uncertainties. An aeroelastic system with a trailingedge flap as its control input will be considered as the plant for demonstration of effectiveness of the controller. The controller is proved to be robust by both athematical proof and simulation results. In Chapter 3, a robust output feedback control strategy is discussed for the vibration suppression of an aeroelastic system operating in an unsteady incompressible flowfield. The aeroelastic system is actuated using a combination of leading-edge (LE) and trailing-edge (TE) flaps in the presence of different kinds of gust disturbances. In Chapter 5, a neural-network based model-free controller is designed for an aeroelastic system operating at supersonic speed. The controller is shown to be able to effectively asymptotically stabilize the system via both a Lyapunov-based stability proof and numerical simulation results

Event-triggered near optimal adaptive control of interconnected systems

Increased interest in complex interconnected systems like smart-grid, cyber manufacturing have attracted researchers to develop optimal adaptive control schemes to elicit a desired performance when the complex system dynamics are uncertain. In this dissertation, motivated by the fact that aperiodic event sampling saves network resources while ensuring system stability, a suite of novel event-sampled distributed near-optimal adaptive control schemes are introduced for uncertain linear and affine nonlinear interconnected systems in a forward-in-time and online manner. First, a novel stochastic hybrid Q-learning scheme is proposed to generate optimal adaptive control law and to accelerate the learning process in the presence of random delays and packet losses resulting from the communication network for an uncertain linear interconnected system. Subsequently, a novel online reinforcement learning (RL) approach is proposed to solve the Hamilton-Jacobi-Bellman (HJB) equation by using neural networks (NNs) for generating distributed optimal control of nonlinear interconnected systems using state and output feedback. To relax the state vector measurements, distributed observers are introduced. Next, using RL, an improved NN learning rule is derived to solve the HJB equation for uncertain nonlinear interconnected systems with event-triggered feedback. Distributed NN identifiers are introduced both for approximating the uncertain nonlinear dynamics and to serve as a model for online exploration. Next, the control policy and the event-sampling errors are considered as non-cooperative players and a min-max optimization problem is formulated for linear and affine nonlinear systems by using zero-sum game approach for simultaneous optimization of both the control policy and the event based sampling instants. The net result is the development of optimal adaptive event-triggered control of uncertain dynamic systems --Abstract, page iv

Nonlinear system identification and control using dynamic multi-time scales neural networks

In this thesis, on-line identification algorithm and adaptive control design are proposed for nonlinear singularly perturbed systems which are represented by dynamic neural network model with multi-time scales. A novel on-line identification law for the Neural Network weights and linear part matrices of the model has been developed to minimize the identification errors. Based on the identification results, an adaptive controller is developed to achieve trajectory tracking. The Lyapunov synthesis method is used to conduct stability analysis for both identification algorithm and control design. To further enhance the stability and performance of the control system, an improved . dynamic neural network model is proposed by replacing all the output signals from the plant with the state variables of the neural network. Accordingly, the updating laws are modified with a dead-zone function to prevent parameter drifting. By combining feedback linearization with one of three classical control methods such as direct compensator, sliding mode controller or energy function compensation scheme, three different adaptive controllers have been proposed for trajectory tracking. New Lyapunov function analysis method is applied for the stability analysis of the improved identification algorithm and three control systems. Extensive simulation results are provided to support the effectiveness of the proposed identification algorithms and control systems for both dynamic NN models

Reinforcement Learning Based Dual-Control Methodology for Complex Nonlinear Discrete-Time Systems with Application to Spark Engine EGR Operation

A novel reinforcement-learning-based dual-control methodology adaptive neural network (NN) controller is developed to deliver a desired tracking performance for a class of complex feedback nonlinear discrete-time systems, which consists of a second-order nonlinear discrete-time system in nonstrict feedback form and an affine nonlinear discrete-time system, in the presence of bounded and unknown disturbances. For example, the exhaust gas recirculation (EGR) operation of a spark ignition (SI) engine is modeled by using such a complex nonlinear discrete-time system. A dual-controller approach is undertaken where primary adaptive critic NN controller is designed for the nonstrict feedback nonlinear discrete-time system whereas the secondary one for the affine nonlinear discrete-time system but the controllers together offer the desired performance. The primary adaptive critic NN controller includes an NN observer for estimating the states and output, an NN critic, and two action NNs for generating virtual control and actual control inputs for the nonstrict feedback nonlinear discrete-time system, whereas an additional critic NN and an action NN are included for the affine nonlinear discrete-time system by assuming the state availability. All NN weights adapt online towards minimization of a certain performance index, utilizing gradient-descent-based rule. Using Lyapunov theory, the uniformly ultimate boundedness (UUB) of the closed-loop tracking error, weight estimates, and observer estimates are shown. The adaptive critic NN controller performance is evaluated on an SI engine operating with high EGR levels where the controller objective is to reduce cyclic dispersion in heat release while minimizing fuel intake. Simulation and experimental results indicate that engine out emissions drop significantly at 20% EGR due to reduction in dispersion in heat release thus verifying the dual-control approach

Optimal Adaptive Output Regulation of Uncertain Nonlinear Discrete-Time Systems using Lifelong Concurrent Learning

This Paper Addresses Neural Network (NN) based Optimal Adaptive Regulation of Uncertain Nonlinear Discrete-Time Systems in Affine Form using Output Feedback Via Lifelong Concurrent Learning. First, an Adaptive NN Observer is Introduced to Estimate Both the State Vector and Control Coefficient Matrix, and its NN Weights Are Adjusted using Both Output Error and Concurrent Learning Term to Relax the Persistency Excitation (PE) Condition. Next, by Utilizing an Actor-Critic Framework for Estimating the Value Functional and Control Policy, the Critic Network Weights Are Tuned Via Both Temporal Different Error and Concurrent Learning Schemes through a Replay Buffer. the Actor NN Weights Are Tuned using Control Policy Errors. to Attain Lifelong Learning for Performing Effectively during Multiple Tasks, an Elastic Weight Consolidation Term is Added to the Critic NN Weight Tuning Law. the State Estimation, Regulation, and the Weight Estimation Errors of the Observer, Actor and Critic NNs Are Demonstrated to Be Bounded When Performing Tasks by using Lyapunov Analysis. Simulation Results Are Carried Out to Verify the Effectiveness of the Proposed Approach on a Vander Pol Oscillator. Finally, Extension to Optimal Tracking is Given Briefly

Adaptive Predictive Control Using Neural Network for a Class of Pure-feedback Systems in Discrete-time

10.1109/TNN.2008.2000446IEEE Transactions on Neural Networks1991599-1614ITNN