Adaptive dynamic programming-based controller with admittance adaptation for robot–environment interaction

The problem of optimal tracking control for robot–environment interaction is studied in this article. The environment is regarded as a linear system and an admittance control with iterative linear quadratic regulator method is obtained to guarantee the compliant behaviour. Meanwhile, an adaptive dynamic programming-based controller is proposed. Under adaptive dynamic programming frame, the critic network is performed with radial basis function neural network to approximate the optimal cost, and the neural network weight updating law is incorporated with an additional stabilizing term to eliminate the requirement for the initial admissible control. The stability of the system is proved by Lyapunov theorem. The simulation results demonstrate the effectiveness of the proposed control scheme

Discrete Globalised Dual Heuristic Dynamic Programming in Control of the Two-Wheeled Mobile Robot

Network-based control systems have been emerging technologies in the control of nonlinear systems over the past few years. This paper focuses on the implementation of the approximate dynamic programming algorithm in the network-based tracking control system of the two-wheeled mobile robot, Pioneer 2-DX. The proposed discrete tracking control system consists of the globalised dual heuristic dynamic programming algorithm, the PD controller, the supervisory term, and an additional control signal. The structure of the supervisory term derives from the stability analysis realised using the Lyapunov stability theorem. The globalised dual heuristic dynamic programming algorithm consists of two structures: the actor and the critic, realised in a form of neural networks. The actor generates the suboptimal control law, while the critic evaluates the realised control strategy by approximation of value function from the Bellman’s equation. The presented discrete tracking control system works online, the neural networks’ weights adaptation process is realised in every iteration step, and the neural networks preliminary learning procedure is not required. The performance of the proposed control system was verified by a series of computer simulations and experiments realised using the wheeled mobile robot Pioneer 2-DX

Neural-Network Vector Controller for Permanent-Magnet Synchronous Motor Drives: Simulated and Hardware-Validated Results

This paper focuses on current control in a permanentmagnet synchronous motor (PMSM). The paper has two main objectives: The first objective is to develop a neural-network (NN) vector controller to overcome the decoupling inaccuracy problem associated with conventional PI-based vector-control methods. The NN is developed using the full dynamic equation of a PMSM, and trained to implement optimal control based on approximate dynamic programming. The second objective is to evaluate the robust and adaptive performance of the NN controller against that of the conventional standard vector controller under motor parameter variation and dynamic control conditions by (a) simulating the behavior of a PMSM typically used in realistic electric vehicle applications and (b) building an experimental system for hardware validation as well as combined hardware and simulation evaluation. The results demonstrate that the NN controller outperforms conventional vector controllers in both simulation and hardware implementation

Approximate Dynamic Programming Based Optimal Neurocontrol Synthesis of a Chemical Reactor Process Using Proper Orthogonal Decomposition

The concept of approximate dynamic programming and adaptive critic neural network based optimal controller is extended in this study to include systems governed by partial differential equations. An optimal controller is synthesized for a dispersion type tubular chemical reactor, which is governed by two coupled nonlinear partial differential equations. It consists of three steps: First, empirical basis functions are designed using the Proper Orthogonal Decomposition technique and a low-order lumped parameter system to represent the infinite-dimensional system is obtained by carrying out a Galerkin projection. Second, approximate dynamic programming technique is applied in a discrete time framework, followed by the use of a dual neural network structure called adaptive critics, to obtain optimal neurocontrollers for this system. In this structure, one set of neural networks captures the relationship between the state variables and the control, whereas the other set captures the relationship between the state and the costate variables. Third, the lumped parameter control is then mapped back to the spatial dimension using the same basis functions to result in a feedback control. Numerical results are presented that illustrate the potential of this approach. It should be noted that the procedure presented in this study can be used in synthesizing optimal controllers for a fairly general class of nonlinear distributed parameter systems

Neural-network based vector control of VSCHVDC transmission systems

The application of high-voltage dc (HVDC) using voltage-source converters (VSC) has surged recently in electric power transmission and distribution systems. An optimal vector control of a VSC-HVDC system which uses an artificial neural network to implement an approximate dynamic programming algorithm and is trained with Levenberg-Marquardt is introduced in this paper. The proposed neural network vector control algorithm is analyzed in comparison with standard vector control methods for various HVDC control requirements, including dc voltage, active and reactive power control, and ac system voltage support. Assessment of the resulting closed-loop control shows that the neural network vector control approach has superior performance and works efficiently within and beyond the constraints of the HVDC system, for instance, converter rated power and saturation of PWM modulation

A Proportional-Integrator Type Adaptive Critic Design-Based Neurocontroller for a Static Compensator in a Multimachine Power System

A novel nonlinear optimal controller for a static compensator (STATCOM) connected to a power system, using artificial neural networks, is presented in this paper. The action dependent heuristic dynamic programming, a member of the adaptive critic designs family is used for the design of the STATCOM neurocontroller. This neurocontroller provides optimal control based on reinforcement learning and approximate dynamic programming. Using a proportional-integrator approach, the proposed neurocontroller is capable of dealing with actual rather than deviation signals. Simulation results are provided to show that the proposed controller outperforms a conventional PI controller for a STATCOM in a small and large multimachine power system during large-scale faults, as well as small disturbances

The Heuristic Dynamic Programming Approach in Boost Converters

In this study, a heuristic dynamic programming controller is proposed to control a boost converter. Conventional controllers such as proportional-integral-derivative (PID) or proportional-integral (PI) are designed based on the linearized small-signal model near the operating point. Therefore, the performance of the controller during the start-up, the load change, or the input voltage variation is not optimal since the system model changes by varying the operating point. The heuristic dynamic programming controller optimally controls the boost converter by following the approximate dynamic programming. The advantage of the HDP is that the neural network-based characteristic of the proposed controller enables boost converters to easily cope with large disturbances. An HDP with a well-trained critic and action networks can perform as an optimal controller for the boost converter. To compare the effectiveness of the traditional PI-based and the HDP boost converter, the simulation results are provided

Neural network optimal control for nonlinear system based on zero-sum differential game

summary:In this paper, for a class of the complex nonlinear system control problems, based on the two-person zero-sum game theory, combined with the idea of approximate dynamic programming(ADP), the constrained optimization control problem is solved for the nonlinear systems with unknown system functions and unknown time-varying disturbances. In order to obtain the approximate optimal solution of the zero-sum game, the multilayer neural network is used to fit the evaluation network, the execution network and the disturbance network of ADP respectively. The Lyapunov stability theory is used to prove the uniform convergence, and the system control output converges to the neighborhood of the target reference value. Finally, the simulation example verifies the effectiveness of the algorithm