Training Echo State Networks with Regularization through Dimensionality Reduction

In this paper we introduce a new framework to train an Echo State Network to predict real valued time-series. The method consists in projecting the output of the internal layer of the network on a space with lower dimensionality, before training the output layer to learn the target task. Notably, we enforce a regularization constraint that leads to better generalization capabilities. We evaluate the performances of our approach on several benchmark tests, using different techniques to train the readout of the network, achieving superior predictive performance when using the proposed framework. Finally, we provide an insight on the effectiveness of the implemented mechanics through a visualization of the trajectory in the phase space and relying on the methodologies of nonlinear time-series analysis. By applying our method on well known chaotic systems, we provide evidence that the lower dimensional embedding retains the dynamical properties of the underlying system better than the full-dimensional internal states of the network

Machine Learning for Fluid Mechanics

The field of fluid mechanics is rapidly advancing, driven by unprecedented volumes of data from field measurements, experiments and large-scale simulations at multiple spatiotemporal scales. Machine learning offers a wealth of techniques to extract information from data that could be translated into knowledge about the underlying fluid mechanics. Moreover, machine learning algorithms can augment domain knowledge and automate tasks related to flow control and optimization. This article presents an overview of past history, current developments, and emerging opportunities of machine learning for fluid mechanics. It outlines fundamental machine learning methodologies and discusses their uses for understanding, modeling, optimizing, and controlling fluid flows. The strengths and limitations of these methods are addressed from the perspective of scientific inquiry that considers data as an inherent part of modeling, experimentation, and simulation. Machine learning provides a powerful information processing framework that can enrich, and possibly even transform, current lines of fluid mechanics research and industrial applications.Comment: To appear in the Annual Reviews of Fluid Mechanics, 202

Probabilistic Wind Power and Electricity Load Forecasting with Echo State Networks

With the introduction of distributed generation and the establishment of smart grids, several new challenges in energy analytics arose. These challenges can be solved with a specific type of recurrent neural networks called echo state networks, which can handle the combination of both weather and power consumption or production depending on the dataset to make predictions. Echo state networks are particularly suitable for time series forecasting tasks. Having accurate energy forecasts is paramount to assure grid operation and power provision remains reliable during peak hours when the consumption is high. The majority of load forecasting algorithms do not produce prediction intervals with coverage guarantees but rather produce simple point estimates. Information about uncer- tainty and prediction intervals is rarely useless. It helps grid operators change strategies for configuring the grid from conservative to risk-based ones and assess the reliability of operations. A popular way of producing prediction intervals in regression tasks is by applying Bayesian regression as the regression algorithm. As Bayesian regression is done by sampling, it nat- urally lends itself to generating intervals. However, Bayesian regression is not guaranteed to satisfy the designed coverage level for finite samples. This thesis aims to modify the traditional echo state network model to produce marginally valid and calibrated prediction intervals. This is done by replacing the standard linear regression method with Bayesian linear regression while simultaneously reducing the di- mensions to speed up the computation times. Afterward, a novel calibration technique for time series forecasting is applied in order to obtain said valid prediction intervals. The experiments are conducted using three different time series, two of them being a time series of electricity load. One is univariate, and the other is bivariate. The third time series is a wind power production time series. The proposed method showed promising results for all three datasets while significantly reducing computation times in the sampling ste

Structural health monitoring of a footbridge using Echo State Networks and NARMAX

Echo State Networks (ESNs) and a Nonlinear Auto-Regressive Moving Average model with eXogenous inputs (NARMAX) have been applied to multi-sensor time-series data arising from a test footbridge which has been subjected to multiple potentially damaging interventions. The aim of the work was to automatically classify known potentially damaging events, while also allowing engineers to observe and localise any long term damage trends. The techniques reported here used data from ten temperature sensors as inputs and were tasked with predicting the output signal from eight tilt sensors embedded at various points over the bridge. Initially, interventions were identified by both ESNs and NARMAX. In addition, training ESNs using data up to the first event, and determining the ESNs’ subsequent predictions, allowed inferences to be made not only about when and where the interventions occurred, but also the level of damage caused, without requiring any prior data pre-processing or extrapolation. Finally, ESNs were successfully used as classifiers to characterise various different types of intervention that had taken place

Nonlinear Dynamic System Identification in the Spectral Domain Using Particle-Bernstein Polynomials

System identification (SI) is the discipline of inferring mathematical models from unknown dynamic systems using the input/output observations of such systems with or without prior knowledge of some of the system parameters. Many valid algorithms are available in the literature, including Volterra series expansion, Hammerstein–Wiener models, nonlinear auto-regressive moving average model with exogenous inputs (NARMAX) and its derivatives (NARX, NARMA). Different nonlinear estimators can be used for those algorithms, such as polynomials, neural networks or wavelet networks. This paper uses a different approach, named particle-Bernstein polynomials, as an estimator for SI. Moreover, unlike the mentioned algorithms, this approach does not operate in the time domain but rather in the spectral components of the signals through the use of the discrete Karhunen–Loève transform (DKLT). Some experiments are performed to validate this approach using a publicly available dataset based on ground vibration tests recorded from a real F-16 aircraft. The experiments show better results when compared with some of the traditional algorithms, especially for large, heterogeneous datasets such as the one used. In particular, the absolute error obtained with the prosed method is 63% smaller with respect to NARX and from 42% to 62% smaller with respect to various artificial neural network-based approaches