The combination of circle topology and leaky integrator neurons remarkably improves the performance of echo state network on time series prediction.

Recently, echo state network (ESN) has attracted a great deal of attention due to its high accuracy and efficient learning performance. Compared with the traditional random structure and classical sigmoid units, simple circle topology and leaky integrator neurons have more advantages on reservoir computing of ESN. In this paper, we propose a new model of ESN with both circle reservoir structure and leaky integrator units. By comparing the prediction capability on Mackey-Glass chaotic time series of four ESN models: classical ESN, circle ESN, traditional leaky integrator ESN, circle leaky integrator ESN, we find that our circle leaky integrator ESN shows significantly better performance than other ESNs with roughly 2 orders of magnitude reduction of the predictive error. Moreover, this model has stronger ability to approximate nonlinear dynamics and resist noise than conventional ESN and ESN with only simple circle structure or leaky integrator neurons. Our results show that the combination of circle topology and leaky integrator neurons can remarkably increase dynamical diversity and meanwhile decrease the correlation of reservoir states, which contribute to the significant improvement of computational performance of Echo state network on time series prediction

A priori data-driven multi-clustered reservoir generation algorithm for echo state network.

Echo state networks (ESNs) with multi-clustered reservoir topology perform better in reservoir computing and robustness than those with random reservoir topology. However, these ESNs have a complex reservoir topology, which leads to difficulties in reservoir generation. This study focuses on the reservoir generation problem when ESN is used in environments with sufficient priori data available. Accordingly, a priori data-driven multi-cluster reservoir generation algorithm is proposed. The priori data in the proposed algorithm are used to evaluate reservoirs by calculating the precision and standard deviation of ESNs. The reservoirs are produced using the clustering method; only the reservoir with a better evaluation performance takes the place of a previous one. The final reservoir is obtained when its evaluation score reaches the preset requirement. The prediction experiment results obtained using the Mackey-Glass chaotic time series show that the proposed reservoir generation algorithm provides ESNs with extra prediction precision and increases the structure complexity of the network. Further experiments also reveal the appropriate values of the number of clusters and time window size to obtain optimal performance. The information entropy of the reservoir reaches the maximum when ESN gains the greatest precision

The <i>NRMSEs</i>₈₄ comparisons of prediction performance for four networks.

<p>The <i>NRMSEs</i>₈₄ comparisons of prediction performance for four networks.</p

The regular echo state network model with random reservoir topology.

<p>The regular echo state network model with random reservoir topology.</p

Correlation coefficient and the responding statistical distribution of reservoir states for four networks.

<p>(a) ESN (b) C-ESN (c) LI-ESN (d) C-LI-ESN.</p

Principal component energy analysis of reservoir states for four different networks. Log PC energy: log10 of reservoir signal energies in the principal component directions. Leading PC energy: The top ten signal energies in linear scale.

<p>(a) ESN (b) C-ESN (c) LI-ESN (d) C-LI-ESN.</p

Network output (dashed) diverges from the teaching signal (solid) when internal states become unstable (a) or divergent (b).

<p>Network output (dashed) diverges from the teaching signal (solid) when internal states become unstable (a) or divergent (b).</p

Capability of nonlinear time series prediction.

<p>The log10 testMSE of four networks vs the time delay <i>τ</i> in the Mackey-Glass system.</p