Neural-Based Ensembles and Unorganized Machines to Predict Streamflow Series from Hydroelectric Plants

Estimating future streamflows is a key step in producing electricity for countries with hydroelectric plants. Accurate predictions are particularly important due to environmental and economic impact they lead. In order to analyze the forecasting capability of models regarding monthly seasonal streamflow series, we realized an extensive investigation considering: six versions of unorganized machines—extreme learning machines (ELM) with and without regularization coefficient (RC), and echo state network (ESN) using the reservoirs from Jaeger’s and Ozturk et al., with and without RC. Additionally, we addressed the ELM as the combiner of a neural-based ensemble, an investigation not yet accomplished in such context. A comparative analysis was performed utilizing two linear approaches (autoregressive model (AR) and autoregressive and moving average model (ARMA)), four artificial neural networks (multilayer perceptron, radial basis function, Elman network, and Jordan network), and four ensembles. The tests were conducted at five hydroelectric plants, using horizons of 1, 3, 6, and 12 steps ahead. The results indicated that the unorganized machines and the ELM ensembles performed better than the linear models in all simulations. Moreover, the errors showed that the unorganized machines and the ELM-based ensembles reached the best general performances

Wind Power Forecasting Based on Echo State Networks and Long Short-Term Memory

Wind power generation has presented an important development around the world. However, its integration into electrical systems presents numerous challenges due to the variable nature of the wind. Therefore, to maintain an economical and reliable electricity supply, it is necessary to accurately predict wind generation. The Wind Power Prediction Tool (WPPT) has been proposed to solve this task using the power curve associated with a wind farm. Recurrent Neural Networks (RNNs) model complex non-linear relationships without requiring explicit mathematical expressions that relate the variables involved. In particular, two types of RNN, Long Short-Term Memory (LSTM) and Echo State Network (ESN), have shown good results in time series forecasting. In this work, we present an LSTM+ESN architecture that combines the characteristics of both networks. An architecture similar to an ESN is proposed, but using LSTM blocks as units in the hidden layer. The training process of this network has two key stages: (i) the hidden layer is trained with a descending gradient method online using one epoch; (ii) the output layer is adjusted with a regularized regression. In particular, the case is proposed where Step (i) is used as a target for the input signal, in order to extract characteristics automatically as the autoencoder approach; and in the second stage (ii), a quantile regression is used in order to obtain a robust estimate of the expected target. The experimental results show that LSTM+ESN using the autoencoder and quantile regression outperforms the WPPT model in all global metrics used

Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era

Relative entropy minimizing noisy non-linear neural network to approximate stochastic processes

A method is provided for designing and training noise-driven recurrent neural networks as models of stochastic processes. The method unifies and generalizes two known separate modeling approaches, Echo State Networks (ESN) and Linear Inverse Modeling (LIM), under the common principle of relative entropy minimization. The power of the new method is demonstrated on a stochastic approximation of the El Nino phenomenon studied in climate research

Dynamic neural networks for real-time water level predictions of sewerage systems-covering gauged and ungauged sites

[[abstract]]In this research, we propose recurrent neural networks (RNNs) to build a relationship between rainfalls and water level patterns of an urban sewerage system based on historical torrential rain/storm events. The RNN allows signals to propagate in both forward and backward directions, which offers the network dynamic memories. Besides, the information at the current time-step with a feedback operation can yield a time-delay unit that provides internal input information at the next time-step to effectively deal with time-varying systems. The RNN is implemented at both gauged and ungauged sites for 5-, 10-, 15-, and 20-min-ahead water level predictions. The results show that the RNN is capable of learning the nonlinear sewerage system and producing satisfactory predictions at the gauged sites. Concerning the ungauged sites, there are no historical data of water level to support prediction. In order to overcome such problem, a set of synthetic data, generated from a storm water management model (SWMM) under cautious verification process of applicability based on the data from nearby gauging stations, are introduced as the learning target to the training procedure of the RNN and moreover evaluating the performance of the RNN at the ungauged sites. The results demonstrate that the potential role of the SWMM coupled with nearby rainfall and water level information can be of great use in enhancing the capability of the RNN at the ungauged sites. Hence we can conclude that the RNN is an effective and suitable model for successfully predicting the water levels at both gauged and ungauged sites in urban sewerage systems.[[incitationindex]]SCI[[booktype]]紙

Forecasting Particulate Matter Concentrations: Use of Unorganized Machines

Air pollution is an environmental issue studied worldwide, as it has serious impacts on human health. Therefore, forecasting its concentration is of great importance. Then, this study presents an analysis comprising the appliance of Unorganized Machines – Extreme Learning Machines (ELM) and Echo State Networks (ESN) aiming to predict particulate matter with aerodynamic diameter less than 2.5 m (PM2.5) and less than 10 m (PM10). The databases were from Kallio and Vallilla stations in Helsinki, Finland. The computational results showed that the ELM presented best results to PM2.5, while the ESN achieved the best performance to PM10

Data-driven discovery of coordinates and governing equations

The discovery of governing equations from scientific data has the potential to transform data-rich fields that lack well-characterized quantitative descriptions. Advances in sparse regression are currently enabling the tractable identification of both the structure and parameters of a nonlinear dynamical system from data. The resulting models have the fewest terms necessary to describe the dynamics, balancing model complexity with descriptive ability, and thus promoting interpretability and generalizability. This provides an algorithmic approach to Occam's razor for model discovery. However, this approach fundamentally relies on an effective coordinate system in which the dynamics have a simple representation. In this work, we design a custom autoencoder to discover a coordinate transformation into a reduced space where the dynamics may be sparsely represented. Thus, we simultaneously learn the governing equations and the associated coordinate system. We demonstrate this approach on several example high-dimensional dynamical systems with low-dimensional behavior. The resulting modeling framework combines the strengths of deep neural networks for flexible representation and sparse identification of nonlinear dynamics (SINDy) for parsimonious models. It is the first method of its kind to place the discovery of coordinates and models on an equal footing.Comment: 25 pages, 6 figures; added acknowledgment