Fully Evolvable Optimal Neurofuzzy Controller Using Adaptive Critic Designs

A near-optimal neurofuzzy external controller is designed in this paper for a static compensator (STATCOM) in a multimachine power system. The controller provides an auxiliary reference signal for the STATCOM in such a way that it improves the damping of the rotor speed deviations of its neighboring generators. A zero-order Takagi-Sugeno fuzzy rule base constitutes the core of the controller. A heuristic dynamic programming (HDP) based approach is used to further train the controller and enable it to provide nonlinear near-optimal control at different operating conditions of the power system. Based on the connectionist systems theory, the parameters of the neurofuzzy controller, including the membership functions, undergo training. Simulation results are provided that compare the performance of the neurofuzzy controller with and without updating the fuzzy set parameters. Simulation results indicate that updating the membership functions can noticeably improve the performance of the controller and reduce the size of the STATCOM, which leads to lower capital investment

Adaptive dynamic programming with eligibility traces and complexity reduction of high-dimensional systems

This dissertation investigates the application of a variety of computational intelligence techniques, particularly clustering and adaptive dynamic programming (ADP) designs especially heuristic dynamic programming (HDP) and dual heuristic programming (DHP). Moreover, a one-step temporal-difference (TD(0)) and n-step TD (TD(λ)) with their gradients are utilized as learning algorithms to train and online-adapt the families of ADP. The dissertation is organized into seven papers. The first paper demonstrates the robustness of model order reduction (MOR) for simulating complex dynamical systems. Agglomerative hierarchical clustering based on performance evaluation is introduced for MOR. This method computes the reduced order denominator of the transfer function by clustering system poles in a hierarchical dendrogram. Several numerical examples of reducing techniques are taken from the literature to compare with our work. In the second paper, a HDP is combined with the Dyna algorithm for path planning. The third paper uses DHP with an eligibility trace parameter (λ) to track a reference trajectory under uncertainties for a nonholonomic mobile robot by using a first-order Sugeno fuzzy neural network structure for the critic and actor networks. In the fourth and fifth papers, a stability analysis for a model-free action-dependent HDP(λ) is demonstrated with batch- and online-implementation learning, respectively. The sixth work combines two different gradient prediction levels of critic networks. In this work, we provide a convergence proofs. The seventh paper develops a two-hybrid recurrent fuzzy neural network structures for both critic and actor networks. They use a novel n-step gradient temporal-difference (gradient of TD(λ)) of an advanced ADP algorithm called value-gradient learning (VGL(λ)), and convergence proofs are given. Furthermore, the seventh paper is the first to combine the single network adaptive critic with VGL(λ). --Abstract, page iv