Fuzzy jump wavelet neural network based on rule induction for dynamic nonlinear system identification with real data applications

Aim Fuzzy wavelet neural network (FWNN) has proven to be a promising strategy in the identification of nonlinear systems. The network considers both global and local properties, deals with imprecision present in sensory data, leading to desired precisions. In this paper, we proposed a new FWNN model nominated “Fuzzy Jump Wavelet Neural Network” (FJWNN) for identifying dynamic nonlinear-linear systems, especially in practical applications. Methods The proposed FJWNN is a fuzzy neural network model of the Takagi-Sugeno-Kang type whose consequent part of fuzzy rules is a linear combination of input regressors and dominant wavelet neurons as a sub-jump wavelet neural network. Each fuzzy rule can locally model both linear and nonlinear properties of a system. The linear relationship between the inputs and the output is learned by neurons with linear activation functions, whereas the nonlinear relationship is locally modeled by wavelet neurons. Orthogonal least square (OLS) method and genetic algorithm (GA) are respectively used to purify the wavelets for each sub-JWNN. In this paper, fuzzy rule induction improves the structure of the proposed model leading to less fuzzy rules, inputs of each fuzzy rule and model parameters. The real-world gas furnace and the real electromyographic (EMG) signal modeling problem are employed in our study. In the same vein, piecewise single variable function approximation, nonlinear dynamic system modeling, and Mackey–Glass time series prediction, ratify this method superiority. The proposed FJWNN model is compared with the state-of-the-art models based on some performance indices such as RMSE, RRSE, Rel ERR%, and VAF%. Results The proposed FJWNN model yielded the following results: RRSE (mean±std) of 10e-5±6e-5 for piecewise single-variable function approximation, RMSE (mean±std) of 2.6–4±2.6e-4 for the first nonlinear dynamic system modelling, RRSE (mean±std) of 1.59e-3±0.42e-3 for Mackey–Glass time series prediction, RMSE of 0.3421 for gas furnace modelling and VAF% (mean±std) of 98.24±0.71 for the EMG modelling of all trial signals, indicating a significant enhancement over previous methods. Conclusions The FJWNN demonstrated promising accuracy and generalization while moderating network complexity. This improvement is due to applying main useful wavelets in combination with linear regressors and using fuzzy rule induction. Compared to the state-of-the-art models, the proposed FJWNN yielded better performance and, therefore, can be considered a novel tool for nonlinear system identificationPeer ReviewedPostprint (published version

Recurrent error-based ridge polynomial neural networks for time series forecasting

Time series forecasting has attracted much attention due to its impact on many practical applications. Neural networks (NNs) have been attracting widespread interest as a promising tool for time series forecasting. The majority of NNs employ only autoregressive (AR) inputs (i.e., lagged time series values) when forecasting time series. Moving-average (MA) inputs (i.e., errors) however have not adequately considered. The use of MA inputs, which can be done by feeding back forecasting errors as extra network inputs, alongside AR inputs help to produce more accurate forecasts. Among numerous existing NNs architectures, higher order neural networks (HONNs), which have a single layer of learnable weights, were considered in this research work as they have demonstrated an ability to deal with time series forecasting and have an simple architecture. Based on two HONNs models, namely the feedforward ridge polynomial neural network (RPNN) and the recurrent dynamic ridge polynomial neural network (DRPNN), two recurrent error-based models were proposed. These models were called the ridge polynomial neural network with error feedback (RPNN-EF) and the ridge polynomial neural network with error-output feedbacks (RPNN-EOF). Extensive simulations covering ten time series were performed. Besides RPNN and DRPNN, a pi-sigma neural network and a Jordan pi-sigma neural network were used for comparison. Simulation results showed that introducing error feedback to the models lead to significant forecasting performance improvements. Furthermore, it was found that the proposed models outperformed many state-of-the-art models. It was concluded that the proposed models have the capability to efficiently forecast time series and that practitioners could benefit from using these forecasting models

Nonlinear dynamics and modeling of heart and brain signals

Ph.DDOCTOR OF PHILOSOPH

Real-Time Localization of Epileptogenic Foci EEG Signals: An FPGA-Based Implementation

The epileptogenic focus is a brain area that may be surgically removed to control of epileptic seizures. Locating it is an essential and crucial step prior to the surgical treatment. However, given the difficulty of determining the localization of this brain region responsible of the initial seizure discharge, many works have proposed machine learning methods for the automatic classification of focal and non-focal electroencephalographic (EEG) signals. These works use automatic classification as an analysis tool for helping neurosurgeons to identify focal areas off-line, out of surgery, during the processing of the huge amount of information collected during several days of patient monitoring. In turn, this paper proposes an automatic classification procedure capable of assisting neurosurgeons online, during the resective epilepsy surgery, to refine the localization of the epileptogenic area to be resected, if they have doubts. This goal requires a real-time implementation with as low a computational cost as possible. For that reason, this work proposes both a feature set and a classifier model that minimizes the computational load while preserving the classification accuracy at 95.5%, a level similar to previous works. In addition, the classification procedure has been implemented on a FPGA device to determine its resource needs and throughput. Thus, it can be concluded that such a device can embed the whole classification process, from accepting raw signals to the delivery of the classification results in a cost-effective Xilinx Spartan-6 FPGA device. This real-time implementation begins providing results after a 5 s latency, and later, can deliver floating-point classification results at 3.5 Hz rate, using overlapped time-windows

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Design and implementation of a soft computing-based controller for a complex mechanical system

Soft-Computing basierende Regler beinhalten Algorithmen, die im Bereich des Maschinellen Lernens einzuordnen sind. Diese Regler sind in der Lage eine geeignete Steuerungsstrategie durch direkte Interaktion mit einer dynamischen Regelstrecke zu entwerfen. Sowohl klassische als auch moderne Reglerentwurfsmethoden hangen von der Genauigkeit des verwendeten dynamischen Systemmodells ab, was insbesondere bei steigender Komplexitat des Systems und auftretenden Modellunsicherheiten nicht mehr uneingeschrankt gewahrleistet werden kann. Die Ziele von Soft- Computing basierenden Reglern sind die Verbesserung der Gute des Regelverhaltens und eine geeignete Anpassung der Regler ohne eine mathematische Modellbildung auf Grundlage von physikalischen Gesetzen. Im Rahmen dieser Arbeit werden funf Algorithmen zur Modellbildung und Regelung dynamischer Systeme untersucht, welche auf dem Mehrschichten-Perzeptron-Netzwerk (Multi-Layer Perceptron network, MLP), auf der Methode der Support Vector Machine (SVM), der Gau-Prozesse, der radialen Basisfunktionen (Radial Basis Functions, RBF) sowie der Fuzzy-Inferenz-Systeme basieren. Im Anschluss an die Darstellung der zugrunde liegenden mathematischen Zusammenhange dieser Methoden sowie deren Hauptanwendungsfelder im Bereich der Modellbildung und Regelung dynamischer Systeme wird eine systematische Evaluierung der funf Methoden diskutiert. Anhand der Verwendung quantitativer Gutekennziern werden diese Methoden fur die Verwendung in der Modellbildung und Regelung dynamischer Systeme vergleichbar gegenubergestellt. Basierend auf den Ergebnissen der Evaluierung wird der SVM-basierte Algorithmus als Kernalgorithmus des Soft-Computing basierenden Reglers verwendet. Der vorgestellte Regler besteht aus zwei Hauptteilen, wobei der erste Teil aus einer Modellfunktion der dynamischen Regelstrecke und einem SVM-basierten Beobachter besteht, und der zweite Teil basierend auf dem Systemmodell eine geeignete Regelstrategie generiert. Die Verikation des SVM-basierten Regleralgorithmus erfolgt anhand eines FEM-Modells eines dynamischen elastischen Balken bzw. einseitig eingespannten elastischen Balkens. Dieses Modell kann z. B. als Ersatzmodell fur das mechanische Verhalten eines exiblen Roboterarms oder einer Flugzeugtrag ache verwendet werden. Der Hauptteil der Modellfunktion besteht aus einem automatischen Systemidentikationsalgorithmus, der auch die Integration eines systematischen Modellbildungsansatzes fur dynamische Systeme ermoglicht.Die Ergebnisse des SVM-basierten Beobachter zeigen ahnliches Verhalten zum Kalman- Bucy Beobachter. Auch die Sensitivitatsanalyse der Parameter zeigt eine bessere Gute der SVM-basierten Beobachter im Vergleich mit den Kalman-Bucy Beobachtern. Im Anschluss wird der SVM-basierte Regler zur Schwingungsregelung des Kragtragers verwendet. Hierbei werden vergleichbare Ergebnisse zum LQR-Regler erzielt. Eine experimentelle Validierung des SVM basierten Reglers erfolgt an Versuchsst anden eines elastischen Biegebalkens sowie eines invertierten Biegebalkens. Die Zustandsbeobachtung fuhrt zu vergleichbaren Ergebnissen verglichen mit einem Kalman-Bucy Beobachter. Auch die Modellbildung des elastischen Balkens fuhrt zu guten Ubereinstimmungen. Die Regelgute des Soft-Computing basierenden Reglers wurde am Versuchsstand des invertierten Biegebalkens experimentell erprobt. Es wird deutlich, dass Ergebnisse im Rahmen der erforderlichen Vorgaben erzielt werden konnen.The focus of this thesis is to obtain a soft computing-based controller for complex mechanical system. soft computing based controllers are based on machine learning algorithm that able to develop suitable control strategies by direct interaction with targeted dynamic systems. Classical and modern control design methods depend on the accuracy of the system dynamic model which cannot be achieved due to the dynamic system complexity and modeling uncertainties. A soft computing-based controller aims to improve the performance of the close loop system and to give the controller adaptation ability as well as to reduce the need for mathematical modeling based on physical laws. In this work ve dierent softcomputing algorithms used in the eld of modeling and controlling dynamic systems are investigated.These algorithms are Multi-Layer Perceptron(MLP) network, Support Vector Machine (SVM),Gaussian process, Radial Basis Function (RBF), and Fuzzy Inference System (FIS). The basic mathematical description of each algorithm is given. Additionally, the most recent applications in modeling and controlling of dynamic system are summarized. A systematic evaluation of the ve algorithms is proposed. The goal of the evaluation is to provide quantitative measure of the performance of soft computing algorithms when used in modeling and controlling a dynamic system. Based on the evaluation, the SVM algorithm is selected as the core learning algorithm for the soft computing based controller. The controller has two main units. The rst unit has two functions of modeling dynamic system and obtaining a SVM-based observer. The second unit is in charge of generating suitable control strategy based on the dynamic model obtained. The verication of the controller using SVM algorithm is done using an elastic cantilever beam modeled using Finite Element Method (FEM). An elastic cantilever beam can be considered as a representation of exible single-link manipulator or aircraft wing. In the core of the modeling unit, an automatic system identication algorithm which allows a systematic modeling approach of dynamic systems is implemented. The results show that the system dynamic model using SVM algorithm is accurate with respect to the FEM model. As for the SVM-based observer the results show that it has good estimation in comparison with to dierent Kalman-Bucy observers. The sensitivity to parameters variations analysis shows that the SVM-based observer has better performance than Kalman-Bucy observer. The SVM based controller is used to control the vibration of the cantilever beam; the results show that the model reference controller using SVM has a similar performance to LQR controller. The validation of the controller using SVM algorithm is carried out using the elastic cantilever beam test rig and the inverted cantilever beam test rig. The states estimation using SVM-based observer of the elastic cantilever beam test rig is successful and accurate compared to a Kalman-Bucy observer. Modeling of the elastic cantilever beam using the SVM algorithm shows good accuracy. The performance of controller is tested on the inverted cantilever beam test rig. The results show that required performance objective can be realized using this control strategy

Signal Processing Research Program

Contains table of contents for Part III, table of contents for Section 1, an introduction and reports on fourteen research projects.Charles S. Draper Laboratory Contract DL-H-404158U.S. Navy - Office of Naval Research Grant N00014-89-J-1489National Science Foundation Grant MIP 87-14969Battelle LaboratoriesTel-Aviv University, Department of Electronic SystemsU.S. Army Research Office Contract DAAL03-86-D-0001The Federative Republic of Brazil ScholarshipSanders Associates, Inc.Bell Northern Research, Ltd.Amoco Foundation FellowshipGeneral Electric FellowshipNational Science Foundation FellowshipU.S. Air Force - Office of Scientific Research FellowshipU.S. Navy - Office of Naval Research Grant N00014-85-K-0272Natural Science and Engineering Research Council of Canada - Science and Technology Scholarshi

DYNAMIC SELF-ORGANISED NEURAL NETWORK INSPIRED BY THE IMMUNE ALGORITHM FOR FINANCIAL TIME SERIES PREDICTION AND MEDICAL DATA CLASSIFICATION

Artificial neural networks have been proposed as useful tools in time series analysis in a variety of applications. They are capable of providing good solutions for a variety of problems, including classification and prediction. However, for time series analysis, it must be taken into account that the variables of data are related to the time dimension and are highly correlated. The main aim of this research work is to investigate and develop efficient dynamic neural networks in order to deal with data analysis issues. This research work proposes a novel dynamic self-organised multilayer neural network based on the immune algorithm for financial time series prediction and biomedical signal classification, combining the properties of both recurrent and self-organised neural networks. The first case study that has been addressed in this thesis is prediction of financial time series. The financial time series signal is in the form of historical prices of different companies. The future prediction of price in financial time series enables businesses to make profits by predicting or simply guessing these prices based on some historical data. However, the financial time series signal exhibits a highly random behaviour, which is non-stationary and nonlinear in nature. Therefore, the prediction of this type of time series is very challenging. In this thesis, a number of experiments have been simulated to evaluate the ability of the designed recurrent neural network to forecast the future value of financial time series. The resulting forecast made by the proposed network shows substantial profits on financial historical signals when compared to the self-organised hidden layer inspired by immune algorithm and multilayer perceptron neural networks. These results suggest that the proposed dynamic neural networks has a better ability to capture the chaotic movement in financial signals. The second case that has been addressed in this thesis is for predicting preterm birth and diagnosing preterm labour. One of the most challenging tasks currently facing the healthcare community is the identification of preterm labour, which has important significances for both healthcare and the economy. Premature birth occurs when the baby is born before completion of the 37-week gestation period. Incomplete understanding of the physiology of the uterus and parturition means that premature labour prediction is a difficult task. The early prediction of preterm births could help to improve prevention, through appropriate medical and lifestyle interventions. One promising method is the use of Electrohysterography. This method records the uterine electrical activity during pregnancy. In this thesis, the proposed dynamic neural network has been used for classifying between term and preterm labour using uterine signals. The results indicated that the proposed network generated improved classification accuracy in comparison to the benchmarked neural network architectures