Optimal Control Synthesis of a Class of Nonlinear Systems Using Single Network Adaptive Critics

Adaptive critic (AC) neural network solutions to optimal control designs using dynamic programming has reduced the need of complex computations and storage requirements that typical dynamic programming requires. In this paper, a single network adaptive critic (SNAC) is presented. This approach is applicable to a class of nonlinear systems where the optimal control (stationary) equation is explicitly solvable for control in terms of state and costate variables. The SNAC architecture offers three potential advantages; a simpler architecture, significant savings of computational load and reduction in approximation errors. In order to demonstrate these benefits, a real-life micro-electro-mechanical-system (MEMS) problem has been solved. This demonstrates that the SNAC technique is applicable for complex engineering systems. Both AC and SNAC approaches are compared in terms of some metrics

Dynamic Re-Optimization of a Spacecraft Attitude Controller in the Presence of Uncertainties

Online trained neural networks have become popular in recent years in the design of robust and adaptive controllers for dynamic systems with uncertainties due to their universal function approximation capabilities. This paper discusses a technique that dynamically reoptimizes a Single Network Adaptive Critic (SNAC) based optimal controller in the presence of unmodeled plant uncertainties. The SNAC based optimal controller designed for the nominal plant model no more retains optimality in the presence of uncertainties/unmodeled dynamics that may creep up in the system equations during operation. This calls for a strategy to re-optimize the existing SNAC controller with respect to the original cost function but corresponding to new constraint (state) equations. The controller re-optimization is carried out in two steps: (i) synthesis of a set of online neural networks that capture the uncertainties in the plant equations on-line (ii) reoptimization of the existing SNAC controller to drive the states of the plant to a desired reference by minimizing the original cost function. This approach has been applied in the online reoptimization of a spacecraft attitude controller and numerical results from simulation studies are presented here

Modeling and Control of Re-Entry Heat Transfer Problem using Neural Networks

A nonlinear optimal re-entry temperature control problem is solved using single network adaptive critic (SNAC) technique. The nonlinear model developed and used accounts for conduction, convection and radiation at high temperature, represents the dynamics of heat transfer in a cooling fin for an object re-entering the earth\u27s atmosphere. Simulation results demonstrate that the control synthesis technique presented is very effective in obtaining a desired temperature profile over a wide envelope of initial temperature distribution

Dynamic Re-Optimization of a MEMS Controller in Presence of Unmodeled Uncertainties

Online trained neural networks have become popular in recent years in designing robust and adaptive controllers for dynamic systems with uncertainties in their system equations because of their universal function approximation property. This paper discusses a technique that dynamically reoptimizes a Single Network Adaptive Critic (SNAC) based optimal controller in the presence of unmodeled uncertainties. The controller design is carried out in two steps: (i) synthesis of a set of online neural networks that capture the uncertainties in the plant equations on-line (ii) re-optimization of the existing optimal controller to drive the states of the plant to a desired reference by minimizing a predefined cost function. The neural network weight update rule for the online networks has been derived using Lyapunov theory that guarantees stability of the error dynamics as well as boundedness of the weights. This approach has been applied in the online reoptimization of a micro-electromechanical device controller and numerical results from simulation studies are presented here

Optimal Controller Synthesis of Variable-Time Impulsive Problems using Single Network Adaptive Critics

This paper presents a systematic approach to solve for the optimal control of a variable-time impulsive system. First, optimality condition for a variable-time impulsive system is derived using the calculus of variations method. Next, a single network adaptive critic technique is proposed to numerically solve for the optimal control and the detailed algorithm is presented. Finally, two examples-one linear and one nonlinear-are solved applying the conditions derived and the algorithm proposed. Numerical results demonstrate the power of the neural network based adaptive critic method in solving this class of problems

Optimal Neuro-Controller Synthesis for Impulse-Driven System

This paper presents a new controller design technique for systems driven with impulse inputs. Necessary conditions for optimal impulse control are derived. A neural network structure to solve the resulting equations is presented. The solution concepts are illustrated with a few example problems that exhibit increasing levels of difficulty. Two linear problems-one scalar and one vector-and a benchmark nonlinear problem-Van Der Pol oscillator-are used as case studies. Numerical results show the efficacy of the new solution process for impulse driven systems. Since the theoretical development and the design technique are free from restrictive assumptions, this technique is applicable to many problems in engineering and science

Issues on Stability of ADP Feedback Controllers for Dynamical Systems

This paper traces the development of neural-network (NN)-based feedback controllers that are derived from the principle of adaptive/approximate dynamic programming (ADP) and discusses their closed-loop stability. Different versions of NN structures in the literature, which embed mathematical mappings related to solutions of the ADP-formulated problems called “adaptive critics” or “action-critic” networks, are discussed. Distinction between the two classes of ADP applications is pointed out. Furthermore, papers in “model-free” development and model-based neurocontrollers are reviewed in terms of their contributions to stability issues. Recent literature suggests that work in ADP-based feedback controllers with assured stability is growing in diverse forms

Development and Implementation of New Nonlinear Control Concepts for a UA

A reconfigurable flight control method is developed to be implemented on an Unmanned Aircraft (UA), a thirty percent scale model of the Cessna 150. This paper presents the details of the UAV platform, system identification, reconfigurable controller design, development, and implementation on the UA to analyze the performance metrics. A Crossbow Inertial Measurement Unit provides the roll, pitch and yaw accelerations and rates along with the roll and pitch. The 100400 mini-air data boom from spaceage control provides the airspeed, altitude, angle of attack and the side slip angles. System identification is accomplished by commanding preprogrammed inputs to the control surfaces and correlating the corresponding variations at the outputs. A Single Network Adaptive Critic, which is a neural network based optimal controller, is developed as part of a nonlinear flight control system. An online learning neural network is augmented to form an outer loop to reconfigure and supplement the optimal controller to guarantee a practical stability for the airplane. This paper also presents some simulations from the hardware-in-the-loop testing and concludes with an analysis of the flight performance metrics for the controller under investigation

Optimization based control design techniques for distributed parameter systems

The study presents optimization based control design techniques for the systems that are governed by partial differential equations. A control technique is developed for systems that are actuated at the boundary. The principles of dynamic inversion and constrained optimization theory are used to formulate a feedback controller. This control technique is demonstrated for heat equations and thermal convection loops. This technique is extended to address a practical issue of parameter uncertainty in a class of systems. An estimator is defined for unknown parameters in the system. The Lyapunov stability theory is used to derive an update law of these parameters. The estimator is used to design an adaptive controller for the system. A second control technique is presented for a class of second order systems that are actuated in-domain. The technique of proper orthogonal decomposition is used first to develop an approximate model. This model is then used to design optimal feedback controller. Approximate dynamic programming based neural network architecture is used to synthesize a sub-optimal controller. This control technique is demonstrated to stabilize the heave dynamics of a flexible aircraft wings. The third technique is focused on the optimal control of stationary thermally convected fluid flows from the numerical point of view. To overcome the computational requirement, optimization is carried out using reduced order model. The technique of proper orthogonal decomposition is used to develop reduced order model. An example of chemical vapor deposition reactor is considered to examine this control technique --Abstract, page iii

Near-Optimal Control of a Quadcopter Using Reinforcement Learning

This paper presents a novel control method for quadcopters that achieves near-optimal tracking control for input-affine nonlinear quadcopter dynamics. The method uses a reinforcement learning algorithm called Single Network Adaptive Critics (SNAC), which approximates a solution to the discrete-time Hamilton-Jacobi-Bellman (DT-HJB) equation using a single neural network trained offline. The control method involves two SNAC controllers, with the outer loop controlling the linear position and velocities (position control) and the inner loop controlling the angular position and velocities (attitude control). The resulting quadcopter controller provides optimal feedback control and tracks a trajectory for an infinite-horizon, and it is compared with commercial optimal control software. Furthermore, the closed-loop controller can control the system with any initial conditions within the domain of training. Overall, this research demonstrates the benefits of using SNAC for nonlinear control, showing its ability to achieve near-optimal tracking control while reducing computational complexity. This paper provides insights into a new approach for controlling quadcopters, with potential applications in various fields such as aerial surveillance, delivery, and search and rescue