Integrated Pre-Processing for Bayesian Nonlinear System Identification with Gaussian Processes

We introduce GP-FNARX: a new model for nonlinear system identification based on a nonlinear autoregressive exogenous model (NARX) with filtered regressors (F) where the nonlinear regression problem is tackled using sparse Gaussian processes (GP). We integrate data pre-processing with system identification into a fully automated procedure that goes from raw data to an identified model. Both pre-processing parameters and GP hyper-parameters are tuned by maximizing the marginal likelihood of the probabilistic model. We obtain a Bayesian model of the system's dynamics which is able to report its uncertainty in regions where the data is scarce. The automated approach, the modeling of uncertainty and its relatively low computational cost make of GP-FNARX a good candidate for applications in robotics and adaptive control.Comment: Proceedings of the 52th IEEE International Conference on Decision and Control (CDC), Firenze, Italy, December 201

Sparse Bayesian Nonlinear System Identification using Variational Inference

IEEE Bayesian nonlinear system identification for one of the major classes of dynamic model, the nonlinear autoregressive with exogenous input (NARX) model, has not been widely studied to date. Markov chain Monte Carlo (MCMC) methods have been developed, which tend to be accurate but can also be slow to converge. In this contribution, we present a novel, computationally efficient solution to sparse Bayesian identification of the NARX model using variational inference, which is orders of magnitude faster than MCMC methods. A sparsity-inducing hyper-prior is used to solve the structure detection problem. Key results include: 1. successful demonstration of the method on low signal-to-noise ratio signals (down to 2dB); 2. successful benchmarking in terms of speed and accuracy against a number of other algorithms: Bayesian LASSO, reversible jump MCMC, forward regression orthogonalisation, LASSO and simulation error minimisation with pruning; 3. accurate identification of a real world system, an electroactive polymer; and 4. demonstration for the first time of numerically propagating the estimated nonlinear time-domain model parameter uncertainty into the frequency-domain

Sparse Bayesian Nonlinear System Identification using Variational Inference

IEEE Bayesian nonlinear system identification for one of the major classes of dynamic model, the nonlinear autoregressive with exogenous input (NARX) model, has not been widely studied to date. Markov chain Monte Carlo (MCMC) methods have been developed, which tend to be accurate but can also be slow to converge. In this contribution, we present a novel, computationally efficient solution to sparse Bayesian identification of the NARX model using variational inference, which is orders of magnitude faster than MCMC methods. A sparsity-inducing hyper-prior is used to solve the structure detection problem. Key results include: 1. successful demonstration of the method on low signal-to-noise ratio signals (down to 2dB); 2. successful benchmarking in terms of speed and accuracy against a number of other algorithms: Bayesian LASSO, reversible jump MCMC, forward regression orthogonalisation, LASSO and simulation error minimisation with pruning; 3. accurate identification of a real world system, an electroactive polymer; and 4. demonstration for the first time of numerically propagating the estimated nonlinear time-domain model parameter uncertainty into the frequency-domain

A Magnetorheological Damper with Embedded Piezoelectric Force Sensor: Experiment and Modeling

This chapter describes configuration, fabrication, calibration and performance tests of the devised self-sensing MR damper firstly. Then, a black-box identification approach for modeling the forward and inverse dynamics of the self-sensing MR damper is presented, which is developed with the synthesis of NARX model and neural network within a Bayesian inference framework to have the ability of enhancing generalization.Department of Civil and Environmental Engineerin

Towards Bayesian System Identification: With Application to SHM of Offshore Structures

Within the offshore industry Structural Health Monitoring remains a growing area of interest. The oil and gas sectors are faced with ageing infrastructure and are driven by the desire for reliable lifetime extension, whereas the wind energy sector is investing heavily in a large number of structures. This leads to a number of distinct challenges for Structural Health Monitoring which are brought together by one unifying theme --- uncertainty. The offshore environment is highly uncertain, existing structures have not been monitored from construction and the loading and operational conditions they have experienced (among other factors) are not known. For the wind energy sector, high numbers of structures make traditional inspection methods costly and in some cases dangerous due to the inaccessibility of many wind farms. Structural Health Monitoring attempts to address these issues by providing tools to allow automated online assessment of the condition of structures to aid decision making. The work of this thesis presents a number of Bayesian methods which allow system identification, for Structural Health Monitoring, under uncertainty. The Bayesian approach explicitly incorporates prior knowledge that is available and combines this with evidence from observed data to allow the formation of updated beliefs. This is a natural way to approach Structural Health Monitoring, or indeed, many engineering problems. It is reasonable to assume that there is some knowledge available to the engineer before attempting to detect, locate, classify, or model damage on a structure. Having a framework where this knowledge can be exploited, and the uncertainty in that knowledge can be handled rigorously, is a powerful methodology. The problem being that the actual computation of Bayesian results can pose a significant challenge both computationally and in terms of specifying appropriate models. This thesis aims to present a number of Bayesian tools, each of which leverages the power of the Bayesian paradigm to address a different Structural Health Monitoring challenge. Within this work the use of Gaussian Process models is presented as a flexible nonparametric Bayesian approach to regression, which is extended to handle dynamic models within the Gaussian Process NARX framework. The challenge in training Gaussian Process models is seldom discussed and the work shown here aims to offer a quantitative assessment of different learning techniques including discussions on the choice of cost function for optimisation of hyperparameters and the choice of the optimisation algorithm itself. Although rarely considered, the effects of these choices are demonstrated to be important and to inform the use of a Gaussian Process NARX model for wave load identification on offshore structures. The work is not restricted to only Gaussian Process models, but Bayesian state-space models are also used. The novel use of Particle Gibbs for identification of nonlinear oscillators is shown and modifications to this algorithm are applied to handle its specific use in Structural Health Monitoring. Alongside this, the Bayesian state-space model is used to perform joint input-state-parameter inference for Operational Modal Analysis where the use of priors over the parameters and the forcing function (in the form of a Gaussian Process transformed into a state-space representation) provides a methodology for this output-only identification under parameter uncertainty. Interestingly, this method is shown to recover the parameter distributions of the model without compromising the recovery of the loading time-series signal when compared to the case where the parameters are known. Finally, a novel use of an online Bayesian clustering method is presented for performing Structural Health Monitoring in the absence of any available training data. This online method does not require a pre-collected training dataset, nor a model of the structure, and is capable of detecting and classifying a range of operational and damage conditions while in service. This leaves the reader with a toolbox of methods which can be applied, where appropriate, to identification of dynamic systems with a view to Structural Health Monitoring problems within the offshore industry and across engineering

The Challenge of Machine Learning in Space Weather Nowcasting and Forecasting

The numerous recent breakthroughs in machine learning (ML) make imperative to carefully ponder how the scientific community can benefit from a technology that, although not necessarily new, is today living its golden age. This Grand Challenge review paper is focused on the present and future role of machine learning in space weather. The purpose is twofold. On one hand, we will discuss previous works that use ML for space weather forecasting, focusing in particular on the few areas that have seen most activity: the forecasting of geomagnetic indices, of relativistic electrons at geosynchronous orbits, of solar flares occurrence, of coronal mass ejection propagation time, and of solar wind speed. On the other hand, this paper serves as a gentle introduction to the field of machine learning tailored to the space weather community and as a pointer to a number of open challenges that we believe the community should undertake in the next decade. The recurring themes throughout the review are the need to shift our forecasting paradigm to a probabilistic approach focused on the reliable assessment of uncertainties, and the combination of physics-based and machine learning approaches, known as gray-box.Comment: under revie

Artificial Neuron-Based Model for a Hybrid Real-Time System: Induction Motor Case Study

Automatic Machine Learning (AML) methods are currently considered of great interest for use in the development of cyber-physical systems. However, in practice, they present serious application problems with respect to fitness computation, overfitting, lack of scalability, and the need for an enormous amount of time for the computation of neural network hyperparameters. In this work, we have experimentally investigated the impact of continuous updating and validation of the hyperparameters, on the performance of a cyber-physical model, with four estimators based on feedforward and narx ANNs, all with the gradient descent-based optimization technique. The main objective is to demonstrate that the optimized values of the hyperparameters can be validated by simulation with MATLAB/Simulink following a mixed approach based on interleaving the updates of their values with a classical training of the ANNs without affecting their efficiency and automaticity of the proposed method. For the two relevant variables of an Induction Motor (IM), two sets of estimators have been trained from the input current and voltage data. In contrast, the training data for the speed and output electromagnetic torque of the IM have been established with the help of a new Simulink model developed entirely. The results have demonstrated the effectiveness of ANN estimators obtained with the Deep Learning Toolbox (DLT) that we used to transform the trained ANNs into blocks that can be directly used in cyber-physical models designed with Simulink.Junta de Andalucia B-TIC-42-UGR20European CommissionSpanish Science Ministry (Ministerio de Ciencia e Innovacion) PID2020-112495RB-C2

On evolutionary system identification with applications to nonlinear benchmarks

This paper presents a record of the participation of the authors in a workshop on nonlinear system identification held in 2016. It provides a summary of a keynote lecture by one of the authors and also gives an account of how the authors developed identification strategies and methods for a number of benchmark nonlinear systems presented as challenges, before and during the workshop. It is argued here that more general frameworks are now emerging for nonlinear system identification, which are capable of addressing substantial ranges of problems. One of these frameworks is based on evolutionary optimisation (EO); it is a framework developed by the authors in previous papers and extended here. As one might expect from the ‘no-free-lunch’ theorem for optimisation, the methodology is not particularly sensitive to the particular (EO) algorithm used, and a number of different variants are presented in this paper, some used for the first time in system identification problems, which show equal capability. In fact, the EO approach advocated in this paper succeeded in finding the best solutions to two of the three benchmark problems which motivated the workshop. The paper provides considerable discussion on the approaches used and makes a number of suggestions regarding best practice; one of the major new opportunities identified here concerns the application of grey-box models which combine the insight of any prior physical-law based models (white box) with the power of machine learners with universal approximation properties (black box)