32,153 research outputs found
Exploring Transfer Function Nonlinearity in Echo State Networks
Supralinear and sublinear pre-synaptic and dendritic integration is
considered to be responsible for nonlinear computation power of biological
neurons, emphasizing the role of nonlinear integration as opposed to nonlinear
output thresholding. How, why, and to what degree the transfer function
nonlinearity helps biologically inspired neural network models is not fully
understood. Here, we study these questions in the context of echo state
networks (ESN). ESN is a simple neural network architecture in which a fixed
recurrent network is driven with an input signal, and the output is generated
by a readout layer from the measurements of the network states. ESN
architecture enjoys efficient training and good performance on certain
signal-processing tasks, such as system identification and time series
prediction. ESN performance has been analyzed with respect to the connectivity
pattern in the network structure and the input bias. However, the effects of
the transfer function in the network have not been studied systematically.
Here, we use an approach tanh on the Taylor expansion of a frequently used
transfer function, the hyperbolic tangent function, to systematically study the
effect of increasing nonlinearity of the transfer function on the memory,
nonlinear capacity, and signal processing performance of ESN. Interestingly, we
find that a quadratic approximation is enough to capture the computational
power of ESN with tanh function. The results of this study apply to both
software and hardware implementation of ESN.Comment: arXiv admin note: text overlap with arXiv:1502.0071
Product Reservoir Computing: Time-Series Computation with Multiplicative Neurons
Echo state networks (ESN), a type of reservoir computing (RC) architecture,
are efficient and accurate artificial neural systems for time series processing
and learning. An ESN consists of a core of recurrent neural networks, called a
reservoir, with a small number of tunable parameters to generate a
high-dimensional representation of an input, and a readout layer which is
easily trained using regression to produce a desired output from the reservoir
states. Certain computational tasks involve real-time calculation of high-order
time correlations, which requires nonlinear transformation either in the
reservoir or the readout layer. Traditional ESN employs a reservoir with
sigmoid or tanh function neurons. In contrast, some types of biological neurons
obey response curves that can be described as a product unit rather than a sum
and threshold. Inspired by this class of neurons, we introduce a RC
architecture with a reservoir of product nodes for time series computation. We
find that the product RC shows many properties of standard ESN such as
short-term memory and nonlinear capacity. On standard benchmarks for chaotic
prediction tasks, the product RC maintains the performance of a standard
nonlinear ESN while being more amenable to mathematical analysis. Our study
provides evidence that such networks are powerful in highly nonlinear tasks
owing to high-order statistics generated by the recurrent product node
reservoir
A critical evaluation of deterministic methods in size optimisation of reliable and cost effective standalone Hybrid renewable energy systems
Reliability of a hybrid renewable energy system (HRES) strongly depends on various uncertainties affecting the amount of power produced by the system. In the design of systems subject to uncertainties, both deterministic and nondeterministic design approaches can be adopted. In a deterministic design approach, the designer considers the presence of uncertainties and incorporates them indirectly into the design by applying safety factors. It is assumed that, by employing suitable safety factors and considering worst-case-scenarios, reliable systems can be designed. In fact, the multi-objective optimisation problem with two objectives of reliability and cost is reduced to a single-objective optimisation problem with the objective of cost only. In this paper the competence of deterministic design methods in size optimisation of reliable standalone wind-PV-battery, wind-PV-diesel and wind-PV-battery-diesel configurations is examined. For each configuration, first, using different values of safety factors, the optimal size of the system components which minimises the system cost is found deterministically. Then, for each case, using a Monte Carlo simulation, the effect of safety factors on the reliability and the cost are investigated. In performing reliability analysis, several reliability measures, namely, unmet load, blackout durations (total, maximum and average) and mean time between failures are considered. It is shown that the traditional methods of considering the effect of uncertainties in deterministic designs such as design for an autonomy period and employing safety factors have either little or unpredictable impact on the actual reliability of the designed wind-PV-battery configuration. In the case of wind-PV-diesel and wind-PV-battery-diesel configurations it is shown that, while using a high-enough margin of safety in sizing diesel generator leads to reliable systems, the optimum value for this margin of safety leading to a cost-effective system cannot be quantified without employing probabilistic methods of analysis. It is also shown that deterministic cost analysis yields inaccurate results for all of the investigated configurations
- …