Finite-horizon optimal control of linear and a class of nonlinear systems

Traditionally, optimal control of dynamical systems with known system dynamics is obtained in a backward-in-time and offline manner either by using Riccati or Hamilton-Jacobi-Bellman (HJB) equation. In contrast, in this dissertation, finite-horizon optimal regulation has been investigated for both linear and nonlinear systems in a forward-in-time manner when system dynamics are uncertain. Value and policy iterations are not used while the value function (or Q-function for linear systems) and control input are updated once a sampling interval consistent with standard adaptive control. First, the optimal adaptive control of linear discrete-time systems with unknown system dynamics is presented in Paper I by using Q-learning and Bellman equation while satisfying the terminal constraint. A novel update law that uses history information of the cost to go is derived. Paper II considers the design of the linear quadratic regulator in the presence of state and input quantization. Quantization errors are eliminated via a dynamic quantizer design and the parameter update law is redesigned from Paper I. Furthermore, an optimal adaptive state feedback controller is developed in Paper III for the general nonlinear discrete-time systems in affine form without the knowledge of system dynamics. In Paper IV, a NN-based observer is proposed to reconstruct the state vector and identify the dynamics so that the control scheme from Paper III is extended to output feedback. Finally, the optimal regulation of quantized nonlinear systems with input constraint is considered in Paper V by introducing a non-quadratic cost functional. Closed-loop stability is demonstrated for all the controller designs developed in this dissertation by using Lyapunov analysis while all the proposed schemes function in an online and forward-in-time manner so that they are practically viable --Abstract, page iv

Robust Adaptive Fuzzy Output Tracking Control for a Class of Twin-Roll Strip Casting Systems

This paper is concerned with the adaptive fuzzy control problem for a class of twin-roll strip casting systems. By using fuzzy logic systems (FLSs) to approximate the compounded nonlinear functions, a novel robust output tracking controller with adaptation laws is designed based on the high gain observer. First, the nonlinear dynamic equations for the roll gap and the molten steel level are constructed, respectively. Then, the mean value theorem is employed to transform the nonaffine nonlinear systems to the corresponding affine nonlinear systems. Moreover, it is also proved that all the closed-loop signals are bounded and the systems output tracking errors can converge to the desired neighborhoods of the origin via the Lyapunov stability analysis. Finally, simulation results, based on semiexperimental system dynamic model and parameters, are worked out to show the effectiveness of the proposed adaptive fuzzy design method

Adaptive Predictive Control Using Neural Network for a Class of Pure-feedback Systems in Discrete-time

10.1109/TNN.2008.2000446IEEE Transactions on Neural Networks1991599-1614ITNN

Near Optimal Neural Network-Based Output Feedback Control of Affine Nonlinear Discrete-Time Systems

In this paper, a novel online reinforcement learning neural network (NN)-based optimal output feedback controller, referred to as adaptive critic controller, is proposed for affine nonlinear discrete-time systems, to deliver a desired tracking performance. The adaptive critic design consist of three entities, an observer to estimate the system states, an action network that produces optimal control input and a critic that evaluates the performance of the action network. The critic is termed adaptive as it adapts itself to output the optimal cost-to-go function which is based on the standard Bellman equation. By using the Lyapunov approach, the uniformly ultimate boundedness (UUB) of the estimation and tracking errors and weight estimates is demonstrated. The effectiveness of the controller is evaluated for the task of nanomanipulation in a simulation environment

Lyapunov based optimal control of a class of nonlinear systems

Optimal control of nonlinear systems is in fact difficult since it requires the solution to the Hamilton-Jacobi-Bellman (HJB) equation which has no closed-form solution. In contrast to offline and/or online iterative schemes for optimal control, this dissertation in the form of five papers focuses on the design of iteration free, online optimal adaptive controllers for nonlinear discrete and continuous-time systems whose dynamics are completely or partially unknown even when the states not measurable. Thus, in Paper I, motivated by homogeneous charge compression ignition (HCCI) engine dynamics, a neural network-based infinite horizon robust optimal controller is introduced for uncertain nonaffine nonlinear discrete-time systems. First, the nonaffine system is transformed into an affine-like representation while the resulting higher order terms are mitigated by using a robust term. The optimal adaptive controller for the affinelike system solves HJB equation and identifies the system dynamics provided a target set point is given. Since it is difficult to define the set point a priori in Paper II, an extremum seeking control loop is designed while maximizing an uncertain output function. On the other hand, Paper III focuses on the infinite horizon online optimal tracking control of known nonlinear continuous-time systems in strict feedback form by using state and output feedback by relaxing the initial admissible controller requirement. Paper IV applies the optimal controller from Paper III to an underactuated helicopter attitude and position tracking problem. In Paper V, the optimal control of nonlinear continuous-time systems in strict feedback form from Paper III is revisited by using state and output feedback when the internal dynamics are unknown. Closed-loop stability is demonstrated for all the controller designs developed in this dissertation by using Lyapunov analysis --Abstract, page iv

Online Reinforcement Learning Neural Network Controller Design for Nanomanipulation

In this paper, a novel reinforcement learning neural network (NN)-based controller, referred to adaptive critic controller, is proposed for affine nonlinear discrete-time systems with applications to nanomanipulation. In the online NN reinforcement learning method, one NN is designated as the critic NN, which approximates the long-term cost function by assuming that the states of the nonlinear systems is available for measurement. An action NN is employed to derive an optimal control signal to track a desired system trajectory while minimizing the cost function. Online updating weight tuning schemes for these two NNs are also derived. By using the Lyapunov approach, the uniformly ultimate boundedness (UUB) of the tracking error and weight estimates is shown. Nanomanipulation implies manipulating objects with nanometer size. It takes several hours to perform a simple task in the nanoscale world. To accomplish the task automatically the proposed online learning control design is evaluated for the task of nanomanipulation and verified in the simulation environmen

Robust Output Regulation for Autonomous Robots:self-learning mechanisms, task-space control and multi-agent systems

This thesis focuses on robust output regulation for autonomous robots. The control objective of output regulation is to design a feedback controller to achieve asymptotic tracking and/or disturbance rejection for a class of exogenous reference and/or disturbance while maintaining closed-loop stability. We investigate three research problems that pertain to the constructive design of robust output regulation for fully actuated Euler-Lagrange systems from centralized to distributed fashions. The first one is the global robust output regulation of second-order affine nonlinear systems with input disturbances that encompass the fully-actuated Euler-Lagrange systems. Based on a certainty equivalence principle method, we proposed a novel class of nonlinear internal models taking a cascade interconnection structure with strictly relaxed conditions than before. The second one is the output regulation for robot manipulators working in task-space. An internal model-based adaptive controller is designed to cope with uncertain manipulator kinematic and dynamic parameters, as well as unknown periodic reference trajectories generated by harmonic oscillators. The last one is the formation control of manipulators’ end-effector subject to external disturbances or parameter uncertainties. We present and analyze gradient descent-based distributed formation controllers for end-effectors. Internal models are used to reject external disturbances. Moreover, by introducing an extra integrator and an adaptive estimator for gravitational compensation and stabilization, respectively, we extend the proposed gradient-based design to the case where the plant parameters are not exactly known