Generalised additive multiscale wavelet models constructed using particle swarm optimisation and mutual information for spatio-temporal evolutionary system representation

A new class of generalised additive multiscale wavelet models (GAMWMs) is introduced for high dimensional spatio-temporal evolutionary (STE) system identification. A novel two-stage hybrid learning scheme is developed for constructing such an additive wavelet model. In the first stage, a new orthogonal projection pursuit (OPP) method, implemented using a particle swarm optimisation(PSO) algorithm, is proposed for successively augmenting an initial coarse wavelet model, where relevant parameters of the associated wavelets are optimised using a particle swarm optimiser. The resultant network model, obtained in the first stage, may however be a redundant model. In the second stage, a forward orthogonal regression (FOR) algorithm, implemented using a mutual information method, is then applied to refine and improve the initially constructed wavelet model. The proposed two-stage hybrid method can generally produce a parsimonious wavelet model, where a ranked list of wavelet functions, according to the capability of each wavelet to represent the total variance in the desired system output signal is produced. The proposed new modelling framework is applied to real observed images, relative to a chemical reaction exhibiting a spatio-temporal evolutionary behaviour, and the associated identification results show that the new modelling framework is applicable and effective for handling high dimensional identification problems of spatio-temporal evolution sytems

A new class of wavelet networks for nonlinear system identification

A new class of wavelet networks (WNs) is proposed for nonlinear system identification. In the new networks, the model structure for a high-dimensional system is chosen to be a superimposition of a number of functions with fewer variables. By expanding each function using truncated wavelet decompositions, the multivariate nonlinear networks can be converted into linear-in-the-parameter regressions, which can be solved using least-squares type methods. An efficient model term selection approach based upon a forward orthogonal least squares (OLS) algorithm and the error reduction ratio (ERR) is applied to solve the linear-in-the-parameters problem in the present study. The main advantage of the new WN is that it exploits the attractive features of multiscale wavelet decompositions and the capability of traditional neural networks. By adopting the analysis of variance (ANOVA) expansion, WNs can now handle nonlinear identification problems in high dimensions

Identification of partial differential equation models for a class of multiscale spatio-temporal dynamical systems

In this paper, the identification of a class of multiscale spatio-temporal dynamical sys-tems, which incorporate multiple spatial scales, from observations is studied. The proposed approach is a combination of Adams integration and an orthogonal least squares algorithm, in which the multiscale operators are expanded, using polynomials as basis functions, and the spatial derivatives are estimated by finite difference methods. The coefficients of the polynomials can vary with respect to the space domain to represent the feature of multiple scales involved in the system dynamics and are approximated using a B-spline wavelet multi-resolution analysis (MRA). The resulting identified models of the spatio-temporal evolution form a system of partial differential equations with different spatial scales. Examples are provided to demonstrate the efficiency of the proposed method

Estimation of spatial derivatives and identification of continuous spatio-temporal dynamical systems

A new approach for the estimation of spatial derivatives and the identification of a class of continuous spatio-temporal dynamical systems from experimental data is presented in this study. The proposed identification approach is a combination of implicit Adams integration and an orthogonal forward regression algorithm (OFR), in which the operators are expanded using polynomials as basis functions. The noisy experimental data are de-noised by using biorthogonal spline wavelet filters and the spatial derivatives are estimated using a multiresolution analysis method. Finally a bootstrap method is applied to refine the identified parameters from the OFR algorithm. The resulting identified models of the spatio-temporal evolution form a system of partial differential equations. Examples are provided to demonstrate the efficiency of the proposed method

Identification of partial differential equation models for continuous spatio-temporal dynamical systems

The identification of a class of continuous spatio-temporal dynamical systems from observations is presented in this paper. The proposed approach is a combination of implicit Adams integration and an orthogonal least squares algorithm, in which the operators are expanded using polynomials as basis functions and the spatial derivatives are estimated by finite difference methods. The resulting identified models of the spatio-temporal evolution form a system of partial differential equations. Examples are provided to demonstrate the efficiency of the proposed methods

Time-varying system identification using an ultra-orthogonal forward regression and multiwavelet basis functions with applications to EEG

A new parametric approach is proposed for nonlinear and non-stationary system identification based on a time-varying nonlinear autoregressive with exogenous input (TV-NARX) model. The time-varying coefficients of the TV-NARX model are expanded using multi- wavelet basis functions and the model is thus transformed into a time-invariant regression problem. An ultra-orthogonal forward regression (UOFR) algorithm aided by mutual information (MI) is designed to identify a parsimonious model structure and estimate the associated model parameters. The UOFR-MI algorithm which uses not only the observed data themselves but also weak derivatives of the signals is more powerful in model structure detection. The proposed approach combining the advantages of both the basis function expansion method and the UOFR-MI algorithm is proved to be capable of tracking the change of time-varying parameters effectively in both numerical simulations and the real EEG data

The wavelet-NARMAX representation : a hybrid model structure combining polynomial models with multiresolution wavelet decompositions

A new hybrid model structure combing polynomial models with multiresolution wavelet decompositions is introduced for nonlinear system identification. Polynomial models play an important role in approximation theory, and have been extensively used in linear and nonlinear system identification. Wavelet decompositions, in which the basis functions have the property of localization in both time and frequency, outperform many other approximation schemes and offer a flexible solution for approximating arbitrary functions. Although wavelet representations can approximate even severe nonlinearities in a given signal very well, the advantage of these representations can be lost when wavelets are used to capture linear or low-order nonlinear behaviour in a signal. In order to sufficiently utilise the global property of polynomials and the local property of wavelet representations simultaneously, in this study polynomial models and wavelet decompositions are combined together in a parallel structure to represent nonlinear input-output systems. As a special form of the NARMAX model, this hybrid model structure will be referred to as the WAvelet-NARMAX model, or simply WANARMAX. Generally, such a WANARMAX representation for an input-output system might involve a large number of basis functions and therefore a great number of model terms. Experience reveals that only a small number of these model terms are significant to the system output. A new fast orthogonal least squares algorithm, called the matching pursuit orthogonal least squares (MPOLS) algorithm, is also introduced in this study to determine which terms should be included in the final model

An adaptive orthogonal search algorithm for model subset selection and non-linear system identification

A new adaptive orthogonal search (AOS) algorithm is proposed for model subset selection and non-linear system identification. Model structure detection is a key step in any system identification problem. This consists of selecting significant model terms from a redundant dictionary of candidate model terms, and determining the model complexity (model length or model size). The final objective is to produce a parsimonious model that can well capture the inherent dynamics of the underlying system. In the new AOS algorithm, a modified generalized cross-validation criterion, called the adjustable prediction error sum of squares (APRESS), is introduced and incorporated into a forward orthogonal search procedure. The main advantage of the new AOS algorithm is that the mechanism is simple and the implementation is direct and easy, and more importantly it can produce efficient model subsets for most non-linear identification problems

Identification of time-varying systems using multiresolution wavelet models

Identification of linear and nonlinear time-varying systems is investigated and a new wavelet model identification algorithm is introduced. By expanding each time-varying coefficient using a multiresolution wavelet expansion, the time-varying problem is reduced to a time invariant problem and the identification reduces to regressor selection and parameter estimation. Several examples are included to illustrate the application of the new algorithm

Lattice dynamical wavelet neural networks implemented using particle swarm optimization for spatio-temporal system identification

In this brief, by combining an efficient wavelet representation with a coupled map lattice model, a new family of adaptive wavelet neural networks, called lattice dynamical wavelet neural networks (LDWNNs), is introduced for spatio-temporal system identification. A new orthogonal projection pursuit (OPP) method, coupled with a particle swarm optimization (PSO) algorithm, is proposed for augmenting the proposed network. A novel two-stage hybrid training scheme is developed for constructing a parsimonious network model. In the first stage, by applying the OPP algorithm, significant wavelet neurons are adaptively and successively recruited into the network, where adjustable parameters of the associated wavelet neurons are optimized using a particle swarm optimizer. The resultant network model, obtained in the first stage, however, may be redundant. In the second stage, an orthogonal least squares algorithm is then applied to refine and improve the initially trained network by removing redundant wavelet neurons from the network. An example for a real spatio-temporal system identification problem is presented to demonstrate the performance of the proposed new modeling framework