Neural Networks: Training and Application to Nonlinear System Identification and Control

This dissertation investigates training neural networks for system identification and classification. The research contains two main contributions as follow:1. Reducing number of hidden layer nodes using a feedforward componentThis research reduces the number of hidden layer nodes and training time of neural networks to make them more suited to online identification and control applications by adding a parallel feedforward component. Implementing the feedforward component with a wavelet neural network and an echo state network provides good models for nonlinear systems.The wavelet neural network with feedforward component along with model predictive controller can reliably identify and control a seismically isolated structure during earthquake. The network model provides the predictions for model predictive control. Simulations of a 5-story seismically isolated structure with conventional lead-rubber bearings showed significant reductions of all response amplitudes for both near-field (pulse) and far-field ground motions, including reduced deformations along with corresponding reduction in acceleration response. The controller effectively regulated the apparent stiffness at the isolation level. The approach is also applied to the online identification and control of an unmanned vehicle. Lyapunov theory is used to prove the stability of the wavelet neural network and the model predictive controller. 2. Training neural networks using trajectory based optimization approachesTraining neural networks is a nonlinear non-convex optimization problem to determine the weights of the neural network. Traditional training algorithms can be inefficient and can get trapped in local minima. Two global optimization approaches are adapted to train neural networks and avoid the local minima problem. Lyapunov theory is used to prove the stability of the proposed methodology and its convergence in the presence of measurement errors. The first approach transforms the constraint satisfaction problem into unconstrained optimization. The constraints define a quotient gradient system (QGS) whose stable equilibrium points are local minima of the unconstrained optimization. The QGS is integrated to determine local minima and the local minimum with the best generalization performance is chosen as the optimal solution. The second approach uses the QGS together with a projected gradient system (PGS). The PGS is a nonlinear dynamical system, defined based on the optimization problem that searches the components of the feasible region for solutions. Lyapunov theory is used to prove the stability of PGS and QGS and their stability under presence of measurement noise

Generalised additive multiscale wavelet models constructed using particle swarm optimisation and mutual information for spatio-temporal evolutionary system representation

A new class of generalised additive multiscale wavelet models (GAMWMs) is introduced for high dimensional spatio-temporal evolutionary (STE) system identification. A novel two-stage hybrid learning scheme is developed for constructing such an additive wavelet model. In the first stage, a new orthogonal projection pursuit (OPP) method, implemented using a particle swarm optimisation(PSO) algorithm, is proposed for successively augmenting an initial coarse wavelet model, where relevant parameters of the associated wavelets are optimised using a particle swarm optimiser. The resultant network model, obtained in the first stage, may however be a redundant model. In the second stage, a forward orthogonal regression (FOR) algorithm, implemented using a mutual information method, is then applied to refine and improve the initially constructed wavelet model. The proposed two-stage hybrid method can generally produce a parsimonious wavelet model, where a ranked list of wavelet functions, according to the capability of each wavelet to represent the total variance in the desired system output signal is produced. The proposed new modelling framework is applied to real observed images, relative to a chemical reaction exhibiting a spatio-temporal evolutionary behaviour, and the associated identification results show that the new modelling framework is applicable and effective for handling high dimensional identification problems of spatio-temporal evolution sytems

Lattice dynamical wavelet neural networks implemented using particle swarm optimisation for spatio-temporal system identification

Starting from the basic concept of coupled map lattices, a new family of adaptive wavelet neural networks, called lattice dynamical wavelet neural networks (LDWNN), is introduced for spatiotemporal system identification, by combining an efficient wavelet representation with a coupled map lattice model. A new orthogonal projection pursuit (OPP) method, coupled with a particle swarm optimisation (PSO) algorithm, is proposed for augmenting the proposed network. A novel two-stage hybrid training scheme is developed for constructing a parsimonious network model. In the first stage, by applying the orthogonal projection pursuit algorithm, significant wavelet-neurons are adaptively and successively recruited into the network, where adjustable parameters of the associated waveletneurons are optimised using a particle swarm optimiser. The resultant network model, obtained in the first stage, may however be redundant. In the second stage, an orthogonal least squares (OLS) algorithm is then applied to refine and improve the initially trained network by removing redundant wavelet-neurons from the network. The proposed two-stage hybrid training procedure can generally produce a parsimonious network model, where a ranked list of wavelet-neurons, according to the capability of each neuron to represent the total variance in the system output signal is produced. Two spatio-temporal system identification examples are presented to demonstrate the performance of the proposed new modelling framework

Nonlinear self-tuning control for power oscillation damping

Power systems exhibit nonlinear behavior especially during disturbances, necessitating the application of appropriate nonlinear control techniques. Lack of availability of accurate and updated models for the whole power system adds to the challenge. Conventional damping control design approaches consider a single operating condition of the system, which are obviously simple but tend to lack performance robustness. Objective of this research work is to design a measurement based self-tuning controller, which does not rely on accurate models and deals with nonlinearities in system response. Designed controller is required to ensure settling of inter-area oscillations within 10−12s, following disturbance such as a line outage. The neural network (NN) model is illustrated for the representation of nonlinear power systems. An optimization based algorithm, Levenberg-Marquardt (LM), for online estimation of power system dynamic behavior is proposed in batch mode to improve the model estimation. Careful study shows that the LM algorithm yields better closed loop performance, compared to conventional recursive least square (RLS) approach with the pole-shifting controller (PSC) in linear framework. Exploiting the capability of LM, a special form of neural network compatible with feedback linearization technique, is applied. Validation of the performance of proposed algorithm is done through the modeling and simulating heavy loading of transmission lines, when the nonlinearities are pronounced. Nonlinear NN model in the Feedback Linearization (FLNN) form gives better estimation than the autoregressive with an external input (ARX) form. The proposed identifier (FLNN with LM algorithm) is then tested on a 4−machine, 2−area power system in conjunction with the feedback linearization controller (FBLC) under varying operating conditions. This case study indicates that the developed closed loop strategy performs better than the linear NN with PSC. Extension of FLNN with FBLC structure in a multi-variable setup is also done. LM algorithm is successfully employed with the multi-input multi-output FLNN structure in a sliding window batch mode, and FBLC controller generates multiple control signals for FACTS. Case studies on a large scale 16−machine, 5−area power system are reported for different power flow scenarios, to prove the superiority of proposed schemes: both MIMO and MISO against a conventional model based controller. A coefficient vector for FBLC is derived, and utilized online at each time instant, to enhance the damping performance of controller, transforming into a time varying controller

emgr - The Empirical Gramian Framework

System Gramian matrices are a well-known encoding for properties of input-output systems such as controllability, observability or minimality. These so-called system Gramians were developed in linear system theory for applications such as model order reduction of control systems. Empirical Gramian are an extension to the system Gramians for parametric and nonlinear systems as well as a data-driven method of computation. The empirical Gramian framework - emgr - implements the empirical Gramians in a uniform and configurable manner, with applications such as Gramian-based (nonlinear) model reduction, decentralized control, sensitivity analysis, parameter identification and combined state and parameter reduction

Disturbance Observer-based Robust Control and Its Applications: 35th Anniversary Overview

Disturbance Observer has been one of the most widely used robust control tools since it was proposed in 1983. This paper introduces the origins of Disturbance Observer and presents a survey of the major results on Disturbance Observer-based robust control in the last thirty-five years. Furthermore, it explains the analysis and synthesis techniques of Disturbance Observer-based robust control for linear and nonlinear systems by using a unified framework. In the last section, this paper presents concluding remarks on Disturbance Observer-based robust control and its engineering applications.Comment: 12 pages, 4 figure

A brief review of neural networks based learning and control and their applications for robots

As an imitation of the biological nervous systems, neural networks (NN), which are characterized with powerful learning ability, have been employed in a wide range of applications, such as control of complex nonlinear systems, optimization, system identification and patterns recognition etc. This article aims to bring a brief review of the state-of-art NN for the complex nonlinear systems. Recent progresses of NNs in both theoretical developments and practical applications are investigated and surveyed. Specifically, NN based robot learning and control applications were further reviewed, including NN based robot manipulator control, NN based human robot interaction and NN based behavior recognition and generation