System identification for complex financial system.

The mam purpose of this thesis focuses on the investigation of major financial volatility models including the relevant mean model used in the context of volatility estimation, and the development of a systematic nonlinear identification methodology for these problems. Financial volatility is one of the key aspects in financial economics and volatility modelling involves both the mean process modelling, and the volatility process modelling. Although many volatility models have been derived to approximate the volatility process, linear mean models are almost always used and to the best of our knowledge there is no application of fitting the mean process using a nonlinear model with selected structure. Based on the fact that nonlinearity has been observed in many financial market return data sets, the Non linear AutoRegression Moving Average with eXogenous input (NARMAX) modelling methodology with the term selection algorithm Orthogonal Forward Regression (OFR) is proposed to approximate the nonlinear mean process during volatility modelling. However, the assumption of a constant variance is usually violated in financial market return data. A new Weighted OFR algorithm is therefore proposed to correct for the impact of heteroskedastic noise on the term selection of the nonlinear mean model based on the assumption that the variance process is modelled by a Generalized AutoRegressive Conditional Heteroskedastic (GARCH) model. Because the weights to use are unknown, an iterative refined procedure is developed to learn the weights and to simultaneously improve the parameter estimates of both the mean and the volatility models. New validation methods are proposed to validate the nonlinear selected mean model and the volatility model. During the validation, the assumptions associated with the mean model are tested using a correlation method and the assumptions of the volatility model are tested using a Brock-Dechert-Scheinkrnan (80S) independent and identically distributed (i.i.d.) testing method. The prediction performance of the mean and volatility models is evaluated using a hold out Cross Validation (CV)method. A departure in the prediction of the volatility for the linear mean model, when using nonlinear simulated data, is successfully identified by the new validation methods and the nonlinear selected mean model passes the test. Another application of the NARAMX model, in the very new field of modelling mortality rate, is introduced. A quadratic polynomial mortality rate model selected by the OFR algorithm is developed based on the LifeMetrics male deaths and exposures data for England & Wales from the Office of National Statistics. Comparing the long term prediction of the new model with the Cairns-Blake-Dowd (CSO) statistical mortality rate model indicates the better prediction performance of the quadratic polynomial models. A back-testing method is applied to indicate the robustness of the selected NARMAX type mortality rate models. The term selection, parameter estimation, validation methods and new identification procedures proposed in this thesis open a new gateway to apply the NARMAX modelling technique in the financial area, and for mortality rate modelling to provide a new empirical practice of the NARMAX modelling method

Identification and machine learning prediction of knee-point and knee-onset in capacity degradation curves of lithium-ion cells

ABSTRACT: High-performance batteries greatly benefit from accurate, early predictions of future capacity loss, to advance the management of the battery and sustain desirable application-specific performance characteristics for as long as possible. Li-ion cells exhibit a slow capacity degradation up to a knee-point, after which the degradation accelerates rapidly until the cell’s End-of-Life. Using capacity degradation data, we propose a robust method to identify the knee-point within capacity fade curves. In a new approach to knee research, we propose the concept ‘knee-onset’, marking the beginning of the nonlinear degradation, and provide a simple and robust identification mechanism for it. We link cycle life, knee-point and knee-onset, where predicting/identifying one promptly reveals the others. On data featuring continuous high C-rate cycling (1C–8C), we show that, on average, the knee-point occurs at 95% capacity under these conditions and the knee-onset at 97.1% capacity, with knee and its onset on average 108 cycles apart. After the critical identification step, we employ machine learning (ML) techniques for early prediction of the knee-point and knee-onset. Our models predict knee-point and knee-onset quantitatively with 9.4% error using only information from the first 50 cycles of the cells’ life. Our models use the knee-point predictions to classify the cells’ expected cycle lives as short, medium or long with 88–90% accuracy using only information from the first 3–5 cycles. Our accuracy levels are on par with existing literature for End-of-Life prediction (requiring information from 100-cycles), nonetheless, we address the more complex problem of knee prediction. All estimations are enriched with confidence/credibility metrics. The uncertainty regarding the ML model’s estimations is quantified through prediction intervals. These yield risk-criteria insurers and manufacturers of energy storage applications can use for battery warranties. Our classification model provides a tool for cell manufacturers to speed up the validation of cell production techniques

Data-driven adaptive model-based predictive control with application in wastewater systems

This study is concerned with the development of a new data-driven adaptive model-based predictive controller (MBPC) with input constraints. The proposed methods employ subspace identification technique and a singular value decomposition (SVD)-based optimisation strategy to formulate the control algorithm and incorporate the input constraints. Both direct adaptive model-based predictive controller (DAMBPC) and indirect adaptive model-based predictive controller (IAMBPC) are considered. In DAMBPC, the direct identification of controller parameters is desired to reduce the design effort and computational load while the IAMBPC involves a two-stage process of model identification and controller design. The former method only requires a single QR decomposition for obtaining the controller parameters and uses a receding horizon approach to process input/output data for the identification. A suboptimal SVD-based optimisation technique is proposed to incorporate the input constraints. The proposed techniques are implemented and tested on a fourth order non-linear model of a wastewater system. Simulation results are presented to compare the direct and indirect adaptive methods and to demonstrate the performance of the proposed algorithms

On the smoothness of nonlinear system identification

We shed new light on the \textit{smoothness} of optimization problems arising in prediction error parameter estimation of linear and nonlinear systems. We show that for regions of the parameter space where the model is not contractive, the Lipschitz constant and $\beta$-smoothness of the objective function might blow up exponentially with the simulation length, making it hard to numerically find minima within those regions or, even, to escape from them. In addition to providing theoretical understanding of this problem, this paper also proposes the use of multiple shooting as a viable solution. The proposed method minimizes the error between a prediction model and the observed values. Rather than running the prediction model over the entire dataset, multiple shooting splits the data into smaller subsets and runs the prediction model over each subset, making the simulation length a design parameter and making it possible to solve problems that would be infeasible using a standard approach. The equivalence to the original problem is obtained by including constraints in the optimization. The new method is illustrated by estimating the parameters of nonlinear systems with chaotic or unstable behavior, as well as neural networks. We also present a comparative analysis of the proposed method with multi-step-ahead prediction error minimization

The identification of coupled map lattice models for autonomous cellular neural network patterns

The identification problem for spatiotemporal patterns which are generated by autonomous Cellular Neural Networks (CNN) is investigated in this paper. The application of traditional identification algorithms to these special spatiotemporal systems can produce poor models due to the inherent piecewise nonlinear structure of CNN. To solve this problem, a new type of Coupled Map Lattice model with output constraints and corresponding identification algorithms are proposed in the present study. Numerical examples show that the identified CML models have good prediction capabilities even over the long term and the main dynamics of the original patterns appears to be well represented

Dynamic Estimation of Rigid Motion from Perspective Views via Recursive Identification of Exterior Differential Systems with Parameters on a Topological Manifold

We formulate the problem of estimating the motion of a rigid object viewed under perspective projection as the identification of a dynamic model in Exterior Differential form with parameters on a topological manifold. We first describe a general method for recursive identification of nonlinear implicit systems using prediction error criteria. The parameters are allowed to move slowly on some topological (not necessarily smooth) manifold. The basic recursion is solved in two different ways: one is based on a simple extension of the traditional Kalman Filter to nonlinear and implicit measurement constraints, the other may be regarded as a generalized "Gauss-Newton" iteration, akin to traditional Recursive Prediction Error Method techniques in linear identification. A derivation of the "Implicit Extended Kalman Filter" (IEKF) is reported in the appendix. The ID framework is then applied to solving the visual motion problem: it indeed is possible to characterize it in terms of identification of an Exterior Differential System with parameters living on a C0 topological manifold, called the "essential manifold". We consider two alternative estimation paradigms. The first is in the local coordinates of the essential manifold: we estimate the state of a nonlinear implicit model on a linear space. The second is obtained by a linear update on the (linear) embedding space followed by a projection onto the essential manifold. These schemes proved successful in performing the motion estimation task, as we show in experiments on real and noisy synthetic image sequences

Integrated Pre-Processing for Bayesian Nonlinear System Identification with Gaussian Processes

We introduce GP-FNARX: a new model for nonlinear system identification based on a nonlinear autoregressive exogenous model (NARX) with filtered regressors (F) where the nonlinear regression problem is tackled using sparse Gaussian processes (GP). We integrate data pre-processing with system identification into a fully automated procedure that goes from raw data to an identified model. Both pre-processing parameters and GP hyper-parameters are tuned by maximizing the marginal likelihood of the probabilistic model. We obtain a Bayesian model of the system's dynamics which is able to report its uncertainty in regions where the data is scarce. The automated approach, the modeling of uncertainty and its relatively low computational cost make of GP-FNARX a good candidate for applications in robotics and adaptive control.Comment: Proceedings of the 52th IEEE International Conference on Decision and Control (CDC), Firenze, Italy, December 201