Study of the best linear approximation of nonlinear systems with arbitrary inputs

System identification is the art of modelling of a process (physical, biological, etc.) or to predict its behaviour or output when the environment condition or parameter changes. One is modelling the input-output relationship of a system, for example, linking temperature of a greenhouse (output) to the sunlight intensity (input), power of a car engine (output) with fuel injection rate (input). In linear systems, changing an input parameter will result in a proportional increase in the system output. This is not the case in a nonlinear system. Linear system identification has been extensively studied, more so than nonlinear system identification. Since most systems are nonlinear to some extent, there is significant interest in this topic as industrial processes become more and more complex. In a linear dynamical system, knowing the impulse response function of a system will allow one to predict the output given any input. For nonlinear systems this is not the case. If advanced theory is not available, it is possible to approximate a nonlinear system by a linear one. One tool is the Best Linear Approximation (Bla), which is an impulse response function of a linear system that minimises the output differences between its nonlinear counterparts for a given class of input. The Bla is often the starting point for modelling a nonlinear system. There is extensive literature on the Bla obtained from input signals with a Gaussian probability density function (p.d.f.), but there has been very little for other kinds of inputs. A Bla estimated from Gaussian inputs is useful in decoupling the linear dynamics from the nonlinearity, and in initialisation of parameterised models. As Gaussian inputs are not always practical to be introduced as excitations, it is important to investigate the dependence of the Bla on the amplitude distribution in more detail. This thesis studies the behaviour of the Bla with regards to other types of signals, and in particular, binary sequences where a signal takes only two levels. Such an input is valuable in many practical situations, for example where the input actuator is a switch or a valve and hence can only be turned either on or off. While it is known in the literature that the Bla depends on the amplitude distribution of the input, as far as the author is aware, there is a lack of comprehensive theoretical study on this topic. In this thesis, the Blas of discrete-time time-invariant nonlinear systems are studied theoretically for white inputs with an arbitrary amplitude distribution, including Gaussian and binary sequences. In doing so, the thesis offers answers to fundamental questions of interest to system engineers, for example: 1) How the amplitude distribution of the input and the system dynamics affect the Bla? 2) How does one quantify the difference between the Bla obtained from a Gaussian input and that obtained from an arbitrary input? 3) Is the difference (if any) negligible? 4) What can be done in terms of experiment design to minimise such difference? To answer these questions, the theoretical expressions for the Bla have been developed for both Wiener-Hammerstein (Wh) systems and the more general Volterra systems. The theory for the Wh case has been verified by simulation and physical experiments in Chapter 3 and Chapter 6 respectively. It is shown in Chapter 3 that the difference between the Gaussian and non-Gaussian Bla’s depends on the system memory as well as the higher order moments of the non-Gaussian input. To quantify this difference, a measure called the Discrepancy Factor—a measure of relative error, was developed. It has been shown that when the system memory is short, the discrepancy can be as high as 44.4%, which is not negligible. This justifies the need for a method to decrease such discrepancy. One method is to design a random multilevel sequence for Gaussianity with respect to its higher order moments, and this is discussed in Chapter 5. When estimating the Bla even in the absence of environment and measurement noise, the nonlinearity inevitably introduces nonlinear distortions—deviations from the Bla specific to the realisation of input used. This also explains why more than one realisation of input and averaging is required to obtain a good estimate of the Bla. It is observed that with a specific class of pseudorandom binary sequence (Prbs), called the maximum length binary sequence (Mlbs or the m-sequence), the nonlinear distortions appear structured in the time domain. Chapter 4 illustrates a simple and computationally inexpensive method to take advantage this structure to obtain better estimates of the Bla—by replacing mean averaging by median averaging. Lastly, Chapters 7 and 8 document two independent benchmark studies separate from the main theoretical work of the thesis. The benchmark in Chapter 7 is concerned with the modelling of an electrical Wh system proposed in a special session of the 15th International Federation of Automatic Control (Ifac) Symposium on System Identification (Sysid) 2009 (Schoukens, Suykens & Ljung, 2009). Chapter 8 is concerned with the modelling of a ‘hyperfast’ Peltier cooling system first proposed in the U.K. Automatic Control Council (Ukacc) International Conference on Control, 2010 (Control 2010)

FAST SOLUTION METHODS FOR CONVEX QUADRATIC OPTIMIZATION OF FRACTIONAL DIFFERENTIAL EQUATIONS

In this paper, we present numerical methods suitable for solving convex quadratic Fractional Differential Equation (FDE) constrained optimization problems, with box constraints on the state and/or control variables. We develop an Alternating Direction Method of Multipliers (ADMM) framework, which uses preconditioned Krylov subspace solvers for the resulting sub-problems. The latter allows us to tackle a range of Partial Differential Equation (PDE) optimization problems with box constraints, posed on space-time domains, that were previously out of the reach of state-of-the-art preconditioners. In particular, by making use of the powerful Generalized Locally Toeplitz (GLT) sequences theory, we show that any existing GLT structure present in the problem matrices is preserved by ADMM, and we propose some preconditioning methodologies that could be used within the solver, to demonstrate the generality of the approach. Focusing on convex quadratic programs with time-dependent 2-dimensional FDE constraints, we derive multilevel circulant preconditioners, which may be embedded within Krylov subspace methods, for solving the ADMM sub-problems. Discretized versions of FDEs involve large dense linear systems. In order to overcome this difficulty, we design a recursive linear algebra, which is based on the Fast Fourier Transform (FFT). We manage to keep the storage requirements linear, with respect to the grid size $N$, while ensuring an order $N \log N$ computational complexity per iteration of the Krylov solver. We implement the proposed method, and demonstrate its scalability, generality, and efficiency, through a series of experiments over different setups of the FDE optimization problem

Bayesian Item Response Modeling in R with brms and Stan

Item Response Theory (IRT) is widely applied in the human sciences to model persons' responses on a set of items measuring one or more latent constructs. While several R packages have been developed that implement IRT models, they tend to be restricted to respective prespecified classes of models. Further, most implementations are frequentist while the availability of Bayesian methods remains comparably limited. We demonstrate how to use the R package brms together with the probabilistic programming language Stan to specify and fit a wide range of Bayesian IRT models using flexible and intuitive multilevel formula syntax. Further, item and person parameters can be related in both a linear or non-linear manner. Various distributions for categorical, ordinal, and continuous responses are supported. Users may even define their own custom response distribution for use in the presented framework. Common IRT model classes that can be specified natively in the presented framework include 1PL and 2PL logistic models optionally also containing guessing parameters, graded response and partial credit ordinal models, as well as drift diffusion models of response times coupled with binary decisions. Posterior distributions of item and person parameters can be conveniently extracted and post-processed. Model fit can be evaluated and compared using Bayes factors and efficient cross-validation procedures.Comment: 54 pages, 16 figures, 3 table

The design of periodic excitations for dynamic system identification

System identification techniques are developed for modelling linear and nonlinear systems. The main results of the work are concerned with the design and utilisation of periodic perturbation signals in general areas of time- and frequency-domain system identification. A design strategy is given for a new class of perturbation signals, together with examples of their use in system identification applications. Signal processing procedures are developed for the practical treatment of drift disturbances and transient effects, and also for the detection of nonlinear contributions to the measurement data. The techniques rely completely on the periodicity of the excitation, and so the advantageous properties of periodic input signals are considered in detail. The use of periodic excitations in discrete- and continuous-time nonlinear system identification is also reported, with the identification methods illustrating the worth of frequency-domain measurements in this area. An automatic tuning procedure for PID controllers is also developed, which illustrates an application of system identification techniques to control problems