Harmonic Identification Using an Echo State Network for Adaptive Control of an Active Filter in an Electric Ship

A shunt active filter is a power electronic device used in a power system to decrease ldquoharmonic current pollutionrdquo caused by nonlinear loads. The Echo State Network (ESN) has been widely used as an effective system identifier with much faster training speed than the other Recurrent Neural Networks (RNNs). However, only a few attempts have been made to use an ESN as a system controller. As the first attempt to use an ESN in indirect neurocontrol, this paper proposes an indirect adaptive neurocontrol scheme using two ESNs to control a shunt active filter in a multiple-reference frame. As the first step in the proposed neurocontrol scheme, an online system identifier using an ESN is implemented in the Innovative Integration M67 card consisting of the TMS320C6701 processor to identify the load harmonics in a typical electric ship power system. The shunt active filter and the ship power system are simulated using a Real-Time Digital Simulator (RTDS) system. The required computational effort and the system identification accuracy of an ESN with different dynamic reservoir size are discussed, which can provide useful information for similar applications in the future. The testing results in the real-time implementation show that the ESN is capable of providing fast and accurate system identification for the indirect neurocontrol of a shunt active filter

A PSO with Quantum Infusion Algorithm for Training Simultaneous Recurrent Neural Networks

Simultaneous recurrent neural network (SRN) is one of the most powerful neural network architectures well suited for estimation and control of complex time varying nonlinear dynamic systems. SRN training is a difficult problem especially if multiple inputs and multiple outputs (MIMO) are involved. Particle swarm optimization with quantum infusion (PSO-QI) is introduced in this paper for training such SRNs. In order to illustrate the capability of the PSO-QI training algorithm, a wide area monitor (WAM) for a power system is developed using a multiple inputs multiple outputs Elman SRN. The SRN estimates speed deviations of four generators in a multimachine power system. Since MIMO structured SRNs are hard to train, a two step approach for training is presented with PSO-QI. The performance of PSO-QI is compared to that of the standard PSO algorithm. Results demonstrate that the SRN trained with the PSO-QI in the two step approach tracks the speed deviations of the generators with the minimum error

Neural network for estimating and compensating the nonlinear characteristics of nonstationary complex systems

Issued as final reportNational Science Foundation (U.S

Online Design of an Echo State Network Based Wide Area Monitor for a Multimachine Power System

With deregulation and growth of the power industry, many power system elements such as generators, transmission lines, are driven to operate near their maximum capacity, especially those serving heavy load centres. Wide Area Controllers (WACs) using wide area or global signals can provide remote auxiliary control signals to local controllers such as automatic voltage regulators, power system stabilizers, etc. To damp out system oscillations. However, since the power system is highly nonlinear with fast changing dynamics, it is a challenging problem to design an online system monitor/estimator, which can provide dynamic intra-area and inter-area information such speed deviations of generators to an adaptive WAC continuously. This paper presents a new kind of recurrent neural networks, called the Echo State Network (ESN), for the online design of a Wide Area Monitor (WAM) for a multimachine power system. A single ESN is used to predict the speed deviations of four generators in two different areas. The performance of this ESN WAM is evaluated for small and large disturbances on the power system. Results for an ESN based WAM and a Time-Delayed Neural Network (TDNN)-based WAM are presented and compared. The advantages of the ESN WAM are that it learns the dynamics of the power system in a shorter training time with a higher accuracy and with considerably fewer weights to be adapted compared to the design-based on a TDNN

Reservoir Computing: computation with dynamical systems

In het onderzoeksgebied Machine Learning worden systemen onderzocht die kunnen leren op basis van voorbeelden. Binnen dit onderzoeksgebied zijn de recurrente neurale netwerken een belangrijke deelgroep. Deze netwerken zijn abstracte modellen van de werking van delen van de hersenen. Zij zijn in staat om zeer complexe temporele problemen op te lossen maar zijn over het algemeen zeer moeilijk om te trainen. Recentelijk zijn een aantal gelijkaardige methodes voorgesteld die dit trainingsprobleem elimineren. Deze methodes worden aangeduid met de naam Reservoir Computing. Reservoir Computing combineert de indrukwekkende rekenkracht van recurrente neurale netwerken met een eenvoudige trainingsmethode. Bovendien blijkt dat deze trainingsmethoden niet beperkt zijn tot neurale netwerken, maar kunnen toegepast worden op generieke dynamische systemen. Waarom deze systemen goed werken en welke eigenschappen bepalend zijn voor de prestatie is evenwel nog niet duidelijk. Voor dit proefschrift is onderzoek gedaan naar de dynamische eigenschappen van generieke Reservoir Computing systemen. Zo is experimenteel aangetoond dat de idee van Reservoir Computing ook toepasbaar is op niet-neurale netwerken van dynamische knopen. Verder is een maat voorgesteld die gebruikt kan worden om het dynamisch regime van een reservoir te meten. Tenslotte is een adaptatieregel geïntroduceerd die voor een breed scala reservoirtypes de dynamica van het reservoir kan afregelen tot het gewenste dynamisch regime. De technieken beschreven in dit proefschrift zijn gedemonstreerd op verschillende academische en ingenieurstoepassingen

Deep neural network acoustic models for multi-dialect Arabic speech recognition

Speech is a desirable communication method between humans and computers. The major concerns of the automatic speech recognition (ASR) are determining a set of classification features and finding a suitable recognition model for these features. Hidden Markov Models (HMMs) have been demonstrated to be powerful models for representing time varying signals. Artificial Neural Networks (ANNs) have also been widely used for representing time varying quasi-stationary signals. Arabic is one of the oldest living languages and one of the oldest Semitic languages in the world, it is also the fifth most generally used language and is the mother tongue for roughly 200 million people. Arabic speech recognition has been a fertile area of reasearch over the previous two decades, as attested by the various papers that have been published on this subject. This thesis investigates phoneme and acoustic models based on Deep Neural Networks (DNN) and Deep Echo State Networks for multi-dialect Arabic Speech Recognition. Moreover, the TIMIT corpus with a wide variety of American dialects is also aimed to evaluate the proposed models. The availability of speech data that is time-aligned and labelled at phonemic level is a fundamental requirement for building speech recognition systems. A developed Arabic phoneme database (APD) was manually timed and phonetically labelled. This dataset was constructed from the King Abdul-Aziz Arabic Phonetics Database (KAPD) database for Saudi Arabia dialect and the Centre for Spoken Language Understanding (CSLU2002) database for different Arabic dialects. This dataset covers 8148 Arabic phonemes. In addition, a corpus of 120 speakers (13 hours of Arabic speech) randomly selected from the Levantine Arabic dialect database that is used for training and 24 speakers (2.4 hours) for testing are revised and transcription errors were manually corrected. The selected dataset is labelled automatically using the HTK Hidden Markov Model toolkit. TIMIT corpus is also used for phone recognition and acoustic modelling task. We used 462 speakers (3.14 hours) for training and 24 speakers (0.81 hours) for testing. For Automatic Speech Recognition (ASR), a Deep Neural Network (DNN) is used to evaluate its adoption in developing a framewise phoneme recognition and an acoustic modelling system for Arabic speech recognition. Restricted Boltzmann Machines (RBMs) DNN models have not been explored for any Arabic corpora previously. This allows us to claim priority for adopting this RBM DNN model for the Levantine Arabic acoustic models. A post-processing enhancement was also applied to the DNN acoustic model outputs in order to improve the recognition accuracy and to obtain the accuracy at a phoneme level instead of the frame level. This post process has significantly improved the recognition performance. An Echo State Network (ESN) is developed and evaluated for Arabic phoneme recognition with different learning algorithms. This investigated the use of the conventional ESN trained with supervised and forced learning algorithms. A novel combined supervised/forced supervised learning algorithm (unsupervised adaptation) was developed and tested on the proposed optimised Arabic phoneme recognition datasets. This new model is evaluated on the Levantine dataset and empirically compared with the results obtained from the baseline Deep Neural Networks (DNNs). A significant improvement on the recognition performance was achieved when the ESN model was implemented compared to the baseline RBM DNN model’s result. The results show that the ESN model has a better ability for recognizing phonemes sequences than the DNN model for a small vocabulary size dataset. The adoption of the ESNs model for acoustic modeling is seen to be more valid than the adoption of the DNNs model for acoustic modeling speech recognition, as ESNs are recurrent models and expected to support sequence models better than the RBM DNN models even with the contextual input window. The TIMIT corpus is also used to investigate deep learning for framewise phoneme classification and acoustic modelling using Deep Neural Networks (DNNs) and Echo State Networks (ESNs) to allow us to make a direct and valid comparison between the proposed systems investigated in this thesis and the published works in equivalent projects based on framewise phoneme recognition used the TIMIT corpus. Our main finding on this corpus is that ESN network outperform time-windowed RBM DNN ones. However, our developed system ESN-based shows 10% lower performance when it was compared to the other systems recently reported in the literature that used the same corpus. This due to the hardware availability and not applying speaker and noise adaption that can improve the results in this thesis as our aim is to investigate the proposed models for speech recognition and to make a direct comparison between these models