Optimal Control Synthesis of a Class of Nonlinear Systems Using Single Network Adaptive Critics

Adaptive critic (AC) neural network solutions to optimal control designs using dynamic programming has reduced the need of complex computations and storage requirements that typical dynamic programming requires. In this paper, a single network adaptive critic (SNAC) is presented. This approach is applicable to a class of nonlinear systems where the optimal control (stationary) equation is explicitly solvable for control in terms of state and costate variables. The SNAC architecture offers three potential advantages; a simpler architecture, significant savings of computational load and reduction in approximation errors. In order to demonstrate these benefits, a real-life micro-electro-mechanical-system (MEMS) problem has been solved. This demonstrates that the SNAC technique is applicable for complex engineering systems. Both AC and SNAC approaches are compared in terms of some metrics

Dynamic Re-Optimization of a Spacecraft Attitude Controller in the Presence of Uncertainties

Online trained neural networks have become popular in recent years in the design of robust and adaptive controllers for dynamic systems with uncertainties due to their universal function approximation capabilities. This paper discusses a technique that dynamically reoptimizes a Single Network Adaptive Critic (SNAC) based optimal controller in the presence of unmodeled plant uncertainties. The SNAC based optimal controller designed for the nominal plant model no more retains optimality in the presence of uncertainties/unmodeled dynamics that may creep up in the system equations during operation. This calls for a strategy to re-optimize the existing SNAC controller with respect to the original cost function but corresponding to new constraint (state) equations. The controller re-optimization is carried out in two steps: (i) synthesis of a set of online neural networks that capture the uncertainties in the plant equations on-line (ii) reoptimization of the existing SNAC controller to drive the states of the plant to a desired reference by minimizing the original cost function. This approach has been applied in the online reoptimization of a spacecraft attitude controller and numerical results from simulation studies are presented here

Issues on Stability of ADP Feedback Controllers for Dynamical Systems

This paper traces the development of neural-network (NN)-based feedback controllers that are derived from the principle of adaptive/approximate dynamic programming (ADP) and discusses their closed-loop stability. Different versions of NN structures in the literature, which embed mathematical mappings related to solutions of the ADP-formulated problems called “adaptive critics” or “action-critic” networks, are discussed. Distinction between the two classes of ADP applications is pointed out. Furthermore, papers in “model-free” development and model-based neurocontrollers are reviewed in terms of their contributions to stability issues. Recent literature suggests that work in ADP-based feedback controllers with assured stability is growing in diverse forms

Modeling and Control of Re-Entry Heat Transfer Problem using Neural Networks

A nonlinear optimal re-entry temperature control problem is solved using single network adaptive critic (SNAC) technique. The nonlinear model developed and used accounts for conduction, convection and radiation at high temperature, represents the dynamics of heat transfer in a cooling fin for an object re-entering the earth\u27s atmosphere. Simulation results demonstrate that the control synthesis technique presented is very effective in obtaining a desired temperature profile over a wide envelope of initial temperature distribution

Dynamic Re-Optimization of a MEMS Controller in Presence of Unmodeled Uncertainties

Online trained neural networks have become popular in recent years in designing robust and adaptive controllers for dynamic systems with uncertainties in their system equations because of their universal function approximation property. This paper discusses a technique that dynamically reoptimizes a Single Network Adaptive Critic (SNAC) based optimal controller in the presence of unmodeled uncertainties. The controller design is carried out in two steps: (i) synthesis of a set of online neural networks that capture the uncertainties in the plant equations on-line (ii) re-optimization of the existing optimal controller to drive the states of the plant to a desired reference by minimizing a predefined cost function. The neural network weight update rule for the online networks has been derived using Lyapunov theory that guarantees stability of the error dynamics as well as boundedness of the weights. This approach has been applied in the online reoptimization of a micro-electromechanical device controller and numerical results from simulation studies are presented here

Optimization based control design techniques for distributed parameter systems

The study presents optimization based control design techniques for the systems that are governed by partial differential equations. A control technique is developed for systems that are actuated at the boundary. The principles of dynamic inversion and constrained optimization theory are used to formulate a feedback controller. This control technique is demonstrated for heat equations and thermal convection loops. This technique is extended to address a practical issue of parameter uncertainty in a class of systems. An estimator is defined for unknown parameters in the system. The Lyapunov stability theory is used to derive an update law of these parameters. The estimator is used to design an adaptive controller for the system. A second control technique is presented for a class of second order systems that are actuated in-domain. The technique of proper orthogonal decomposition is used first to develop an approximate model. This model is then used to design optimal feedback controller. Approximate dynamic programming based neural network architecture is used to synthesize a sub-optimal controller. This control technique is demonstrated to stabilize the heave dynamics of a flexible aircraft wings. The third technique is focused on the optimal control of stationary thermally convected fluid flows from the numerical point of view. To overcome the computational requirement, optimization is carried out using reduced order model. The technique of proper orthogonal decomposition is used to develop reduced order model. An example of chemical vapor deposition reactor is considered to examine this control technique --Abstract, page iii

State-Constrained Agile Missile Control with Adaptive-Critic-Based Neural Networks

In this study, we develop an adaptive-critic-based controller to steer an agile missile that has a constraint on the minimum flight Mach number from various initial Mach numbers to a given final Mach number in minimum time while completely reversing its flightpath angle. This class of bounded state space, free final time problems is very difficult to solve due to discontinuities in costates at the constraint boundaries. We use a two-neural-network structure called adaptive critic in this study to carry out the optimization process. This structure obtains an optimal controller through solving optimal control-related equations resulting from a Hamiltonian formulation. Detailed derivations of equations and conditions on the constraint boundary are provided. For numerical experiments, we consider vertical plane scenarios. Flight Mach number and the flightpath angle are the states and the aerodynamic angle of attack is treated as the control. Numerical results bring out some attractive features of the adaptive critic approach and show that this formulation works very well in guiding the missile to its final conditions for this state constrained optimization problem from an envelope of initial conditions

Approximate dynamic programming based solutions for fixed-final-time optimal control and optimal switching

Optimal solutions with neural networks (NN) based on an approximate dynamic programming (ADP) framework for new classes of engineering and non-engineering problems and associated difficulties and challenges are investigated in this dissertation. In the enclosed eight papers, the ADP framework is utilized for solving fixed-final-time problems (also called terminal control problems) and problems with switching nature. An ADP based algorithm is proposed in Paper 1 for solving fixed-final-time problems with soft terminal constraint, in which, a single neural network with a single set of weights is utilized. Paper 2 investigates fixed-final-time problems with hard terminal constraints. The optimality analysis of the ADP based algorithm for fixed-final-time problems is the subject of Paper 3, in which, it is shown that the proposed algorithm leads to the global optimal solution providing certain conditions hold. Afterwards, the developments in Papers 1 to 3 are used to tackle a more challenging class of problems, namely, optimal control of switching systems. This class of problems is divided into problems with fixed mode sequence (Papers 4 and 5) and problems with free mode sequence (Papers 6 and 7). Each of these two classes is further divided into problems with autonomous subsystems (Papers 4 and 6) and problems with controlled subsystems (Papers 5 and 7). Different ADP-based algorithms are developed and proofs of convergence of the proposed iterative algorithms are presented. Moreover, an extension to the developments is provided for online learning of the optimal switching solution for problems with modeling uncertainty in Paper 8. Each of the theoretical developments is numerically analyzed using different real-world or benchmark problems --Abstract, page v