Inferring unknown unknowns: Regularized bias-aware ensemble Kalman filter

Because of physical assumptions and numerical approximations, low-order models are affected by uncertainties in the state and parameters, and by model biases. Model biases, also known as model errors or systematic errors, are difficult to infer because they are `unknown unknowns', i.e., we do not necessarily know their functional form a priori. With biased models, data assimilation methods may be ill-posed because either (i) they are 'bias-unaware' because the estimators are assumed unbiased, (ii) they rely on an a priori parametric model for the bias, or (iii) they can infer model biases that are not unique for the same model and data. First, we design a data assimilation framework to perform combined state, parameter, and bias estimation. Second, we propose a mathematical solution with a sequential method, i.e., the regularized bias-aware ensemble Kalman Filter (r-EnKF), which requires a model of the bias and its gradient (i.e., the Jacobian). Third, we propose an echo state network as the model bias estimator. We derive the Jacobian of the network, and design a robust training strategy with data augmentation to accurately infer the bias in different scenarios. Fourth, we apply the r-EnKF to nonlinearly coupled oscillators (with and without time-delay) affected by different forms of bias. The r-EnKF infers in real-time parameters and states, and a unique bias. The applications that we showcase are relevant to acoustics, thermoacoustics, and vibrations; however, the r-EnKF opens new opportunities for combined state, parameter and bias estimation for real-time and on-the-fly prediction in nonlinear systems.Comment: 22 Figure

Relative entropy minimizing noisy non-linear neural network to approximate stochastic processes

A method is provided for designing and training noise-driven recurrent neural networks as models of stochastic processes. The method unifies and generalizes two known separate modeling approaches, Echo State Networks (ESN) and Linear Inverse Modeling (LIM), under the common principle of relative entropy minimization. The power of the new method is demonstrated on a stochastic approximation of the El Nino phenomenon studied in climate research

Machine Learning Approaches for Data-Driven Analysis and Forecasting of High-Dimensional Chaotic Dynamical Systems

We consider problems in the forecasting of large, complex, spatiotemporal chaotic systems and the possibility that machine learning might be a useful tool for significant improvement of such forecasts. Focusing on weather forecasting as perhaps the most important example of such systems, we note that physics-based weather models have substantial error due to various factors including imperfect modeling of subgrid-scale dynamics and incomplete knowledge of physical processes. In this thesis, we ask if machine learning can potentially correct for such knowledge deficits. First, we demonstrate the effectiveness of using machine learning for model- free prediction of spatiotemporally chaotic systems of arbitrarily large spatial extent and attractor dimension purely from observations of the system’s past evolution. We present a parallel scheme with an example implementation based on the reservoir computing paradigm and demonstrate the scalability of our scheme using the Kuramoto-Sivashinsky equation as an example of a spatiotemporally chaotic system. We then demonstrate the use of machine learning for inferring fundamental properties of dynamical systems, namely the Lyapunov exponents, purely from observed data. We obtain results of unprecedented fidelity with our novel technique, making it possible to find the Lyapunov exponents of large systems where previously known techniques have failed. Next, we propose a general method that combines a physics-informed knowledge-based model and a machine learning technique to build a hybrid forecasting scheme. We further extend our hybrid forecasting approach to the difficult case where only partial measurements of the state of the dynamical system are available. For this purpose, we propose a novel technique that combines machine learning with a data assimilation method called an Ensemble Transform Kalman Filter (ETKF)

short term electric load forecasting using echo state networks and pca decomposition

In this paper, we approach the problem of forecasting a time series (TS) of an electrical load measured on the Azienda Comunale Energia e Ambiente (ACEA) power grid, the company managing the electricity distribution in Rome, Italy, with an echo state network (ESN) considering two different leading times of 10 min and 1 day. We use a standard approach for predicting the load in the next 10 min, while, for a forecast horizon of one day, we represent the data with a high-dimensional multi-variate TS, where the number of variables is equivalent to the quantity of measurements registered in a day. Through the orthogonal transformation returned by PCA decomposition, we reduce the dimensionality of the TS to a lower number $k$ of distinct variables; this allows us to cast the original prediction problem in $k$ different one-step ahead predictions. The overall forecast can be effectively managed by $k$ distinct prediction models, whose outputs are combined together to obtain the final result. We employ a genetic algorithm for tuning the parameters of the ESN and compare its prediction accuracy with a standard autoregressive integrated moving average model