Neural network optimal control for nonlinear system based on zero-sum differential game

summary:In this paper, for a class of the complex nonlinear system control problems, based on the two-person zero-sum game theory, combined with the idea of approximate dynamic programming(ADP), the constrained optimization control problem is solved for the nonlinear systems with unknown system functions and unknown time-varying disturbances. In order to obtain the approximate optimal solution of the zero-sum game, the multilayer neural network is used to fit the evaluation network, the execution network and the disturbance network of ADP respectively. The Lyapunov stability theory is used to prove the uniform convergence, and the system control output converges to the neighborhood of the target reference value. Finally, the simulation example verifies the effectiveness of the algorithm

Koopman Operator Theory and The Applied Perspective of Modern Data-Driven Systems

Recent theoretical developments in dynamical systems and machine learning have allowed researchers to re-evaluate how dynamical systems are modeled and controlled. In this thesis, Koopman operator theory is used to model dynamical systems and obtain optimal control solutions for nonlinear systems using sampled system data. The Koopman operator is obtained using data generated from a real physical system or from an analytical model which describes the physical system under nominal conditions. One of the critical advantages of the Koopman operator is that the response of the nonlinear system can be obtained from an equivalent infinite dimensional linear system. This is achieved by exploiting the topological structure associated with the spectrum of the Koopman operator and the Koopman eigenfunctions. The main contributions of this thesis are threefold. First, we provide a data-driven approach for system identification, and a model-based approach for obtaining an analytic change of coordinates associated with the principle Koopman eigenfunctions for systems with hyperbolic equilibrium points. A new derivation of the Hamilton-Jacobi equations associated with the infinite time horizon nonlinear optimal control problem is obtained using the Koopman generator. Then, a learning algorithm called Koopman Policy Iteration is used to obtain the solution to the infinite horizon nonlinear optimal fixed point regulation problem without state and input constraints. Finally, the finite time nonlinear optimal control problem with state and input constraints is solved using a receding horizon optimization approach called dual mode model predictive control using Koopman eigenfunctions. Evidence supporting the convergence of these methods are provided using analytical examples

Continual Reinforcement Learning Formulation For Zero-Sum Game-Based Constrained Optimal Tracking

This study provides a novel reinforcement learning-based optimal tracking control of partially uncertain nonlinear discrete-time (DT) systems with state constraints using zero-sum game (ZSG) formulation. To address optimal tracking, a novel augmented system consisting of tracking error and its integral value, along with an uncertain desired trajectory, is constructed. A barrier function (BF) with a tradeoff factor is incorporated into the cost function to keep the state trajectories to remain within a compact set and to balance safety with optimality. Next, by using the modified value functional, the ZSG formulation is introduced wherein an actor–critic neural network (NN) framework is employed to approximate the value functional, optimal control input, and worst disturbance. The critic NN weights are tuned once at the sample instants and then iteratively within sampling instants. Using control input errors, the actor NN weights are adjusted once a sampling instant. The concurrent learning term in the critic weight tuning law overcomes the need for the persistency excitation (PE) condition. Further, a weight consolidation scheme is incorporated into the critic update law to attain lifelong learning by overcoming catastrophic forgetting. Finally, a numerical example supports the analytical claims

Decentralized adaptive neural network control of interconnected nonlinear dynamical systems with application to power system

Traditional nonlinear techniques cannot be directly applicable to control large scale interconnected nonlinear dynamic systems due their sheer size and unavailability of system dynamics. Therefore, in this dissertation, the decentralized adaptive neural network (NN) control of a class of nonlinear interconnected dynamic systems is introduced and its application to power systems is presented in the form of six papers. In the first paper, a new nonlinear dynamical representation in the form of a large scale interconnected system for a power network free of algebraic equations with multiple UPFCs as nonlinear controllers is presented. Then, oscillation damping for UPFCs using adaptive NN control is discussed by assuming that the system dynamics are known. Subsequently, the dynamic surface control (DSC) framework is proposed in continuous-time not only to overcome the need for the subsystem dynamics and interconnection terms, but also to relax the explosion of complexity problem normally observed in traditional backstepping. The application of DSC-based decentralized control of power system with excitation control is shown in the third paper. On the other hand, a novel adaptive NN-based decentralized controller for a class of interconnected discrete-time systems with unknown subsystem and interconnection dynamics is introduced since discrete-time is preferred for implementation. The application of the decentralized controller is shown on a power network. Next, a near optimal decentralized discrete-time controller is introduced in the fifth paper for such systems in affine form whereas the sixth paper proposes a method for obtaining the L2-gain near optimal control while keeping a tradeoff between accuracy and computational complexity. Lyapunov theory is employed to assess the stability of the controllers --Abstract, page iv

Multi-H∞ controls for unknown input-interference nonlinear system with reinforcement learning

This article studies the multi-H∞ controls for the input-interference nonlinear systems via adaptive dynamic programming (ADP) method, which allows for multiple inputs to have the individual selfish component of the strategy to resist weighted interference. In this line, the ADP scheme is used to learn the Nash-optimization solutions of the input-interference nonlinear system such that multiple H∞ performance indices can reach the defined Nash equilibrium. First, the input-interference nonlinear system is given and the Nash equilibrium is defined. An adaptive neural network (NN) observer is introduced to identify the input-interference nonlinear dynamics. Then, the critic NNs are used to learn the multiple H∞ performance indices. A novel adaptive law is designed to update the critic NN weights by minimizing the Hamiltonian-Jacobi-Isaacs (HJI) equation, which can be used to directly calculate the multi-H∞ controls effectively by using input-output data such that the actor structure is avoided. Moreover, the control system stability and updated parameter convergence are proved. Finally, two numerical examples are simulated to verify the proposed ADP scheme for the input-interference nonlinear system