A discrete approach to stochastic parametrization and dimensional reduction in nonlinear dynamics

Many physical systems are described by nonlinear differential equations that are too complicated to solve in full. A natural way to proceed is to divide the variables into those that are of direct interest and those that are not, formulate solvable approximate equations for the variables of greater interest, and use data and statistical methods to account for the impact of the other variables. In the present paper the problem is considered in a fully discrete-time setting, which simplifies both the analysis of the data and the numerical algorithms. The resulting time series are identified by a NARMAX (nonlinear autoregression moving average with exogenous input) representation familiar from engineering practice. The connections with the Mori-Zwanzig formalism of statistical physics are discussed, as well as an application to the Lorenz 96 system.Comment: 12 page, includes 2 figure

Variational message passing for online polynomial NARMAX identification

We propose a variational Bayesian inference procedure for online nonlinear system identification. For each output observation, a set of parameter posterior distributions is updated, which is then used to form a posterior predictive distribution for future outputs. We focus on the class of polynomial NARMAX models, which we cast into probabilistic form and represent in terms of a Forney-style factor graph. Inference in this graph is efficiently performed by a variational message passing algorithm. We show empirically that our variational Bayesian estimator outperforms an online recursive least-squares estimator, most notably in small sample size settings and low noise regimes, and performs on par with an iterative least-squares estimator trained offline.Comment: 6 pages, 4 figures. Accepted to the American Control Conference 202

ROBUST RECURSIVE IDENTIFICATION OF HAMMERSTEIN MODELS BASED ON WEISZFALD ALGORITHM

The Hammerstein models can accurately describe a wide variety of nonlinear systems (chemical process, power electronics, electrical drives, sticky control valves). Algorithms of identification depend, among other, on the assumption about the nature of stochastic disturbance. Practical research shows that disturbances, owing the presence of outliers, have a non-Gaussian distribution. In such case it is a common practice to use the robust statistics. In the paper, by analysis of the least favourable probability density, it is shown that the robust (Huber`s) estimation criterion can be presented as a sum of non-overlapping - norm and - norm criteria. By using a Weiszfald algorithm - norm criterion is converted to - norm criterion. So, the weighted - norm criterion is obtained for the identification. The main contributions of the paper are: (i) Presentation of the Huber`s criterion as a sum of - norm and - norm criteria; (ii) Using the Weiszfald algorithm  – norm criterion is converted to a weighted - norm criterion; (iii) Weighted extended least squares in which robustness is included through weighting coefficients are derived for NARMAX (nonlinear autoregressive moving average with exogenous variable) . The illustration of the behaviour of the proposed algorithm is presented through simulations

An extended orthogonal forward regression algorithm for system identification using entropy

In this paper, a fast identification algorithm for nonlinear dynamic stochastic system identification is presented. The algorithm extends the classical Orthogonal Forward Regression (OFR) algorithm so that instead of using the Error Reduction Ratio (ERR) for term selection, a new optimality criterion —Shannon’s Entropy Power Reduction Ratio(EPRR) is introduced to deal with both Gaussian and non-Gaussian signals. It is shown that the new algorithm is both fast and reliable and examples are provided to illustrate the effectiveness of the new approach

NARMAX Model Identification Using Multi-Objective Optimization Differential Evolution

Multi-objective optimization differential evolution (MOODE) algorithm has demonstrated to be an effective algorithm for selecting the structure of nonlinear auto-regressive with exogeneous input (NARX) model in dynamic system modeling. This paper presents the expansion of the MOODE algorithm to obtain an adequate and parsimonious nonlinear auto-regressive moving average with exogenous input (NARMAX) model. A simple methodology for developing the MOODE-NARMAX model is proposed. Two objective functions were considered in the algorithm for optimization; minimizing the number of term of a model structure and minimizing the mean square error between actual and predicted outputs. Two simulated systems and two real systems data were considered for testing the effectiveness of the algorithm. Model validity tests were applied to the set of solutions called the Pareto-optimal set that was generated from the MOODE algorithm in order to select an optimal model. The results show that the MOODE-NARMAX algorithm is able to correctly identify the simulated examples and adequately model real data structures

Data-driven model reduction, Wiener projections, and the Koopman-Mori-Zwanzig formalism

Model reduction methods aim to describe complex dynamic phenomena using only relevant dynamical variables, decreasing computational cost, and potentially highlighting key dynamical mechanisms. In the absence of special dynamical features such as scale separation or symmetries, the time evolution of these variables typically exhibits memory effects. Recent work has found a variety of data-driven model reduction methods to be effective for representing such non-Markovian dynamics, but their scope and dynamical underpinning remain incompletely understood. Here, we study data-driven model reduction from a dynamical systems perspective. For both chaotic and randomly-forced systems, we show the problem can be naturally formulated within the framework of Koopman operators and the Mori-Zwanzig projection operator formalism. We give a heuristic derivation of a NARMAX (Nonlinear Auto-Regressive Moving Average with eXogenous input) model from an underlying dynamical model. The derivation is based on a simple construction we call Wiener projection, which links Mori-Zwanzig theory to both NARMAX and to classical Wiener filtering. We apply these ideas to the Kuramoto-Sivashinsky model of spatiotemporal chaos and a viscous Burgers equation with stochastic forcing.Comment: Substantial revisions, including additional references. To appear in Journal of Computational Physic

The Challenge of Machine Learning in Space Weather Nowcasting and Forecasting

The numerous recent breakthroughs in machine learning (ML) make imperative to carefully ponder how the scientific community can benefit from a technology that, although not necessarily new, is today living its golden age. This Grand Challenge review paper is focused on the present and future role of machine learning in space weather. The purpose is twofold. On one hand, we will discuss previous works that use ML for space weather forecasting, focusing in particular on the few areas that have seen most activity: the forecasting of geomagnetic indices, of relativistic electrons at geosynchronous orbits, of solar flares occurrence, of coronal mass ejection propagation time, and of solar wind speed. On the other hand, this paper serves as a gentle introduction to the field of machine learning tailored to the space weather community and as a pointer to a number of open challenges that we believe the community should undertake in the next decade. The recurring themes throughout the review are the need to shift our forecasting paradigm to a probabilistic approach focused on the reliable assessment of uncertainties, and the combination of physics-based and machine learning approaches, known as gray-box.Comment: under revie