Reinforcement-Learning-Based Output-Feedback Control of Nonstrict Nonlinear Discrete-Time Systems with Application to Engine Emission Control

A novel reinforcement-learning-based output adaptive neural network (NN) controller, which is also referred to as the adaptive-critic NN controller, is developed to deliver the desired tracking performance for a class of nonlinear discrete-time systems expressed in nonstrict feedback form in the presence of bounded and unknown disturbances. The adaptive-critic NN controller consists of an observer, a critic, and two action NNs. The observer estimates the states and output, and the two action NNs provide virtual and actual control inputs to the nonlinear discrete-time system. The critic approximates a certain strategic utility function, and the action NNs minimize the strategic utility function and control inputs. All NN weights adapt online toward minimization of a performance index, utilizing the gradient-descent-based rule, in contrast with iteration-based adaptive-critic schemes. Lyapunov functions are used to show the stability of the closed-loop tracking error, weights, and observer estimates. Separation and certainty equivalence principles, persistency of excitation condition, and linearity in the unknown parameter assumption are not needed. Experimental results on a spark ignition (SI) engine operating lean at an equivalence ratio of 0.75 show a significant (25%) reduction in cyclic dispersion in heat release with control, while the average fuel input changes by less than 1% compared with the uncontrolled case. Consequently, oxides of nitrogen (NOx) drop by 30%, and unburned hydrocarbons drop by 16% with control. Overall, NOx\u27s are reduced by over 80% compared with stoichiometric levels

Reinforcement Learning Based Dual-Control Methodology for Complex Nonlinear Discrete-Time Systems with Application to Spark Engine EGR Operation

A novel reinforcement-learning-based dual-control methodology adaptive neural network (NN) controller is developed to deliver a desired tracking performance for a class of complex feedback nonlinear discrete-time systems, which consists of a second-order nonlinear discrete-time system in nonstrict feedback form and an affine nonlinear discrete-time system, in the presence of bounded and unknown disturbances. For example, the exhaust gas recirculation (EGR) operation of a spark ignition (SI) engine is modeled by using such a complex nonlinear discrete-time system. A dual-controller approach is undertaken where primary adaptive critic NN controller is designed for the nonstrict feedback nonlinear discrete-time system whereas the secondary one for the affine nonlinear discrete-time system but the controllers together offer the desired performance. The primary adaptive critic NN controller includes an NN observer for estimating the states and output, an NN critic, and two action NNs for generating virtual control and actual control inputs for the nonstrict feedback nonlinear discrete-time system, whereas an additional critic NN and an action NN are included for the affine nonlinear discrete-time system by assuming the state availability. All NN weights adapt online towards minimization of a certain performance index, utilizing gradient-descent-based rule. Using Lyapunov theory, the uniformly ultimate boundedness (UUB) of the closed-loop tracking error, weight estimates, and observer estimates are shown. The adaptive critic NN controller performance is evaluated on an SI engine operating with high EGR levels where the controller objective is to reduce cyclic dispersion in heat release while minimizing fuel intake. Simulation and experimental results indicate that engine out emissions drop significantly at 20% EGR due to reduction in dispersion in heat release thus verifying the dual-control approach

Barrier Lyapunov function-based adaptive fuzzy attitude tracking control for rigid satellite with input delay and output constraint

This paper investigates the adaptive attitude tracking problem for the rigid satellite involving output constraint, input saturation, input time delay, and external disturbance by integrating barrier Lyapunov function (BLF) and prescribed performance control (PPC). In contrast to the existing approaches, the input delay is addressed by Pade approximation, and the actual control input concerning saturation is obtained by utilizing an auxiliary variable that simplifies the controller design with respect to mean value methods or Nussbaum function-based strategies. Due to the implementation of the BLF control, together with an interval notion-based PPC strategy, not only the system output but also the transformed error produced by PPC are constrained. An adaptive fuzzy controller is then constructed and the predesigned constraints for system output and the transformed error will not be violated. In addition, a smooth switch term is imported into the controller such that the finite time convergence for all error variables is guaranteed for a certain case while the singularity problem is avoided. Finally, simulations are provided to show the effectiveness and potential of the proposed new design techniques

Asymmetric bounded neural control for an uncertain robot by state feedback and output feedback

In this paper, an adaptive neural bounded control scheme is proposed for an n-link rigid robotic manipulator with unknown dynamics. With the combination of the neural approximation and backstepping technique, an adaptive neural network control policy is developed to guarantee the tracking performance of the robot. Different from the existing results, the bounds of the designed controller are known a priori, and they are determined by controller gains, making them applicable within actuator limitations. Furthermore, the designed controller is also able to compensate the effect of unknown robotic dynamics. Via the Lyapunov stability theory, it can be proved that all the signals are uniformly ultimately bounded. Simulations are carried out to verify the effectiveness of the proposed scheme

Neural network control of nonstrict feedback and nonaffine nonlinear discrete-time systems with application to engine control

In this dissertation, neural networks (NN) approximate unknown nonlinear functions in the system equations, unknown control inputs, and cost functions for two different classes of nonlinear discrete-time systems. Employing NN in closed-loop feedback systems requires that weight update algorithms be stable...Controllers are developed and applied to a nonlinear, discrete-time system of equations for a spark ignition engine model to reduce the cyclic dispersion of heat release --Abstract, page iv

Adaptive Approximation-Based Control for Nonlinear Systems: A Unified Solution with Accurate and Inaccurate Measurements

A unified solution to adaptive approximation-based control for nonlinear systems with accurate and inaccurate state measurement is synthesized in this study. Starting from the standard adaptive approximation-based controller with accurate state measurement, its corresponding physical interpretation, stability conclusion, and learning ability are rigorously addressed when facing additive measurement inaccuracy, and explicit answers are obtained in the framework of both controller matching and system matching. Finally, it proves that, with a certain condition, the standard adaptive approximation-based controller works as a unified solution for the cases with accurate and inaccurate measurement, and the solution can be extended to the nonlinear system control problems with extra unknown dynamics or faults in actuator and/or process dynamics. A single-link robot arm example is used for the simulation demonstration of the unified solution

Unknown dynamics estimator-based output-feedback control for nonlinear pure-feedback systems

Most existing adaptive control designs for nonlinear pure-feedback systems have been derived based on backstepping or dynamic surface control (DSC) methods, requiring full system states to be measurable. The neural networks (NNs) or fuzzy logic systems (FLSs) used to accommodate uncertainties also impose demanding computational cost and sluggish convergence. To address these issues, this paper proposes a new output-feedback control for uncertain pure-feedback systems without using backstepping and function approximator. A coordinate transform is first used to represent the pure-feedback system in a canonical form to evade using the backstepping or DSC scheme. Then the Levant's differentiator is used to reconstruct the unknown states of the derived canonical system. Finally, a new unknown system dynamics estimator with only one tuning parameter is developed to compensate for the lumped unknown dynamics in the feedback control. This leads to an alternative, simple approximation-free control method for pure-feedback systems, where only the system output needs to be measured. The stability of the closed-loop control system, including the unknown dynamics estimator and the feedback control is proved. Comparative simulations and experiments based on a PMSM test-rig are carried out to test and validate the effectiveness of the proposed method