Incremental construction of LSTM recurrent neural network

Long Short--Term Memory (LSTM) is a recurrent neural network that uses structures called memory blocks to allow the net remember significant events distant in the past input sequence in order to solve long time lag tasks, where other RNN approaches fail. Throughout this work we have performed experiments using LSTM networks extended with growing abilities, which we call GLSTM. Four methods of training growing LSTM has been compared. These methods include cascade and fully connected hidden layers as well as two different levels of freezing previous weights in the cascade case. GLSTM has been applied to a forecasting problem in a biomedical domain, where the input/output behavior of five controllers of the Central Nervous System control has to be modelled. We have compared growing LSTM results against other neural networks approaches, and our work applying conventional LSTM to the task at hand.Postprint (published version

A VHDL Core for Intrinsic Evolution of Discrete Time Filters with Signal Feedback

The design of an Evolvable Machine VHDL Core is presented, representing a discrete-time processing structure capable of supporting control system applications. This VHDL Core is implemented in an FPGA and is interfaced with an evolutionary algorithm implemented in firmware on a Digital Signal Processor (DSP) to create an evolvable system platform. The salient features of this architecture are presented. The capability to implement IIR filter structures is presented along with the results of the intrinsic evolution of a filter. The robustness of the evolved filter design is tested and its unique characteristics are described

Derivation of Eurocode 8 spectrum-compatible time-histories from recorded seismic accelerograms via harmonic wavelets

A computationally efficient harmonic wavelet-based iterative procedure is proposed to modify suites of recorded accelerograms to be used in the aseismic design of critical structures regulated by the European code provisions (EC8). Special attention is focused on assessing the potential of appropriately defined orthogonal harmonic wavelet basis functions to derive design spectrum compatible time-histories which preserve the non-stationary characteristics of the original recorded signals. This is a quite desirable attribute in the practice of the aseismic design of yielding structures. In this regard, seven recorded accelerograms recommended for the design of base-isolated structures are modified via the proposed procedure and base-line adjusted to meet the pertinent EC8 compatibility criteria. The instantaneous energy (IE) and the mean instantaneous frequency (MIF) of the modified EC8 compatible time-histories extracted from appropriate wavelet-based signal time-frequency analyses are compared vis-à-vis the IE and MIF of the corresponding original accelerograms. Examining these numerical results, it is established that the herein proposed procedure is a useful tool for processing recorded accelerograms in cases where accounting for the time-varying energy content and frequency composition of strong ground motions associated with historic seismic events is deemed essential in aseismic design

Hybrid Dy-NFIS & RLS equalization for ZCC code in optical-CDMA over multi-mode optical fiber

For long haul coherent optical fiber communication systems, it is significant to precisely monitor the quality of transmission links and optical signals. The channel capacity beyond Shannon limit of Single-mode optical fiber (SMOF) is achieved with the help of Multi-mode optical fiber (MMOF), where the signal is multiplexed in different spatial modes. To increase single-mode transmission capacity and to avoid a foreseen “capacity crunch”, researchers have been motivated to employ MMOF as an alternative. Furthermore, different multiplexing techniques could be applied in MMOF to improve the communication system. One of these techniques is the Optical Code Division Multiple Access (Optical-CDMA), which simplifies and decentralizes network controls to improve spectral efficiency and information security increasing flexibility in bandwidth granularity. This technique also allows synchronous and simultaneous transmission medium to be shared by many users. However, during the propagation of the data over the MMOF based on Optical-CDMA, an inevitable encountered issue is pulse dispersion, nonlinearity and MAI due to mode coupling. Moreover, pulse dispersion, nonlinearity and MAI are significant aspects for the evaluation of the performance of high-speed MMOF communication systems based on Optical-CDMA. This work suggests a hybrid algorithm based on nonlinear algorithm (Dynamic evolving neural fuzzy inference (Dy-NFIS)) and linear algorithm (Recursive least squares (RLS)) equalization for ZCC code in Optical-CDMA over MMOF. Root mean squared error (RMSE), mean squared error (MSE) and Structural Similarity index (SSIM) are used to measure performance results

Particle Swarm Optimization with Quantum Infusion for the Design of Digital Filters

In this paper, particle swarm optimization with quantum infusion (PSO-QI) has been applied for the design of digital filters. In PSO-QI, Global best (gbest) particle (in PSO star topology) obtained from particle swarm optimization is enhanced by doing a tournament with an offspring produced by quantum behaved PSO, and selecting the winner as the new gbest. Filters are designed based on the best approximation to the ideal response by minimizing the maximum ripples in passband and stopband of the filter response. PSO-QI, as is shown in the paper, converges to a better fitness. This new algorithm is implemented in the design of finite impulse response (FIR) and infinite impulse response (IIR) filter

Biologically inspired evolutionary temporal neural circuits

Biological neural networks have always motivated creation of new artificial neural networks, and in this case a new autonomous temporal neural network system. Among the more challenging problems of temporal neural networks are the design and incorporation of short and long-term memories as well as the choice of network topology and training mechanism. In general, delayed copies of network signals can form short-term memory (STM), providing a limited temporal history of events similar to FIR filters, whereas the synaptic connection strengths as well as delayed feedback loops (ER circuits) can constitute longer-term memories (LTM). This dissertation introduces a new general evolutionary temporal neural network framework (GETnet) through automatic design of arbitrary neural networks with STM and LTM. GETnet is a step towards realization of general intelligent systems that need minimum or no human intervention and can be applied to a broad range of problems. GETnet utilizes nonlinear moving average/autoregressive nodes and sub-circuits that are trained by enhanced gradient descent and evolutionary search in terms of architecture, synaptic delay, and synaptic weight spaces. The mixture of Lamarckian and Darwinian evolutionary mechanisms facilitates the Baldwin effect and speeds up the hybrid training. The ability to evolve arbitrary adaptive time-delay connections enables GETnet to find novel answers to many classification and system identification tasks expressed in the general form of desired multidimensional input and output signals. Simulations using Mackey-Glass chaotic time series and fingerprint perspiration-induced temporal variations are given to demonstrate the above stated capabilities of GETnet

Evolving Optimal IIR and Adaptive Filters

In this thesis, current digital filter design techniques are critically reviewed and problems associated with computational cost, complexity, frequency response and speed of convergence, identified. Based on this, a globally optimal, fine- tuned and efficient evolutionary hybrid technique has been developed to automate and optimise infinite impulse response (HR) and adaptive filter design. The proposed hybrid design approach employs an evolutionary algorithm (EA) as a global search tool and a least mean square (LMS) algorithm, whenever appropriate, as a fine-tuner. This permits optimal and real-time tracking of time varying changes in nonstationary environments as widely encountered in telecommunications. In the development, various improvements on existing algorithms are made, including those on components of EAs, LMS algorithm and the filter structures. The aims are to be able to evolve direct form HR structures using simple stability monitoring techniques, to improve local hue-tuning performance and to avoid premature convergence. To evolve complex phenotype chromosomes that are needed by complex HR. filters, a novel method of crossover operation is developed. This is a variation of the standard uniform crossover in which the split points are considered to combine uniquely as indivisible floating-point complex valued genes. The split-point crossover operation produces more new members than the standard crossover operation, and hence provides a faster rate of convergence and avoids premature convergence. The EAs have been particularly designed for small population sizes and to reduce premature convergence, a new operator is designed to introduce new members into the population during evolution. Two techniques are investigated in the design of linear adaptive HR digital filters, namely, the pole design method and the coefficient design method. The pole design method provides filter stability throughout the genetic search without requiring a variety of stability monitoring techniques. The coefficient design method uses simple stability guaranteeing techniques, which also improves the rate of convergence of the EAs. With the hybrid technique, complex-coefficient filters have been designed successfully and globally optimal and adaptive filters have been achieved. The developed methodologies and designs are verified using higher order complex HR systems and, for adaptation, inverse system modelling that is synonymous with channel equalising filters operating in multipath environments. Here adaptive complex parameters become possible to equalise amplitude and phase distortions of the received signals. Various stability-ensuring techniques are investigated extensively and their convergence performances are compared with the proposed method. The proposed hybrid, global and fine design technique is applied to solve adaptive channel equalisation and noise cancellation problems commonly existing in telecommunications