Regularized Nonparametric Volterra Kernel Estimation

In this paper, the regularization approach introduced recently for nonparametric estimation of linear systems is extended to the estimation of nonlinear systems modelled as Volterra series. The kernels of order higher than one, representing higher dimensional impulse responses in the series, are considered to be realizations of multidimensional Gaussian processes. Based on this, prior information about the structure of the Volterra kernel is introduced via an appropriate penalization term in the least squares cost function. It is shown that the proposed method is able to deliver accurate estimates of the Volterra kernels even in the case of a small amount of data points

Maximum entropy properties of discrete-time first-order stable spline kernel

The first order stable spline (SS-1) kernel is used extensively in regularized system identification. In particular, the stable spline estimator models the impulse response as a zero-mean Gaussian process whose covariance is given by the SS-1 kernel. In this paper, we discuss the maximum entropy properties of this prior. In particular, we formulate the exact maximum entropy problem solved by the SS-1 kernel without Gaussian and uniform sampling assumptions. Under general sampling schemes, we also explicitly derive the special structure underlying the SS-1 kernel (e.g. characterizing the tridiagonal nature of its inverse), also giving to it a maximum entropy covariance completion interpretation. Along the way similar maximum entropy properties of the Wiener kernel are also given

Regularized parametric system identification: a decision-theoretic formulation

Parametric prediction error methods constitute a classical approach to the identification of linear dynamic systems with excellent large-sample properties. A more recent regularized approach, inspired by machine learning and Bayesian methods, has also gained attention. Methods based on this approach estimate the system impulse response with excellent small-sample properties. In several applications, however, it is desirable to obtain a compact representation of the system in the form of a parametric model. By viewing the identification of such models as a decision, we develop a decision-theoretic formulation of the parametric system identification problem that bridges the gap between the classical and regularized approaches above. Using the output-error model class as an illustration, we show that this decision-theoretic approach leads to a regularized method that is robust to small sample-sizes as well as overparameterization.Comment: 10 pages, 8 figure

Reweighted nuclear norm regularization: A SPARSEVA approach

The aim of this paper is to develop a method to estimate high order FIR and ARX models using least squares with re-weighted nuclear norm regularization. Typically, the choice of the tuning parameter in the reweighting scheme is computationally expensive, hence we propose the use of the SPARSEVA (SPARSe Estimation based on a VAlidation criterion) framework to overcome this problem. Furthermore, we suggest the use of the prediction error criterion (PEC) to select the tuning parameter in the SPARSEVA algorithm. Numerical examples demonstrate the veracity of this method which has close ties with the traditional technique of cross validation, but using much less computations.Comment: This paper is accepted and will be published in The Proceedings of the 17th IFAC Symposium on System Identification (SYSID 2015), Beijing, China, 201

Outlier robust system identification: a Bayesian kernel-based approach

In this paper, we propose an outlier-robust regularized kernel-based method for linear system identification. The unknown impulse response is modeled as a zero-mean Gaussian process whose covariance (kernel) is given by the recently proposed stable spline kernel, which encodes information on regularity and exponential stability. To build robustness to outliers, we model the measurement noise as realizations of independent Laplacian random variables. The identification problem is cast in a Bayesian framework, and solved by a new Markov Chain Monte Carlo (MCMC) scheme. In particular, exploiting the representation of the Laplacian random variables as scale mixtures of Gaussians, we design a Gibbs sampler which quickly converges to the target distribution. Numerical simulations show a substantial improvement in the accuracy of the estimates over state-of-the-art kernel-based methods.Comment: 5 figure

A nonparametric approach for model individualization in an artificial pancreas

The identification of patient-tailored linear time invariant glucose-insulin models is investigated for type 1 diabetic patients, that are characterized by a substantial inter-subject variability. The individualized linear models are identified by considering a novel kernel-based nonparametric approach and are compared with a linear time invariant average model in terms of prediction performance by means of the coefficient of determination, fit, positive and negative max errors, and root mean squared error. Model identification and validation are based on in-silico data collected from the adult virtual population of the UVA/Padova simulator. The data generation involves a protocol designed to produce a sufficient input excitation without compromising patient safety, compatible also with real life scenarios. The identified models are exploited to synthesize an individualized Model Predictive Controller (MPC) for each patient, which is used in an Artificial Pancreas to maintain the blood glucose concentration within an euglycemic range. The MPC used in several clinical studies, synthesized on the basis of a non-individualized average linear time invariant model, is also considered as reference. The closed-loop control performance is evaluated in an in-silico study on the adult virtual population of the UVA/Padova simulator in a perturbed scenario, in which the MPC is blind to random variations of insulin sensitivity in each virtual patient. © 2015, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved

Bayesian kernel-based system identification with quantized output data

In this paper we introduce a novel method for linear system identification with quantized output data. We model the impulse response as a zero-mean Gaussian process whose covariance (kernel) is given by the recently proposed stable spline kernel, which encodes information on regularity and exponential stability. This serves as a starting point to cast our system identification problem into a Bayesian framework. We employ Markov Chain Monte Carlo (MCMC) methods to provide an estimate of the system. In particular, we show how to design a Gibbs sampler which quickly converges to the target distribution. Numerical simulations show a substantial improvement in the accuracy of the estimates over state-of-the-art kernel-based methods when employed in identification of systems with quantized data.Comment: Submitted to IFAC SysId 201

Regularized system identification using orthonormal basis functions

Most of existing results on regularized system identification focus on regularized impulse response estimation. Since the impulse response model is a special case of orthonormal basis functions, it is interesting to consider if it is possible to tackle the regularized system identification using more compact orthonormal basis functions. In this paper, we explore two possibilities. First, we construct reproducing kernel Hilbert space of impulse responses by orthonormal basis functions and then use the induced reproducing kernel for the regularized impulse response estimation. Second, we extend the regularization method from impulse response estimation to the more general orthonormal basis functions estimation. For both cases, the poles of the basis functions are treated as hyperparameters and estimated by empirical Bayes method. Then we further show that the former is a special case of the latter, and more specifically, the former is equivalent to ridge regression of the coefficients of the orthonormal basis functions.Comment: 6 pages, final submission of an contribution for European Control Conference 2015, uploaded on March 20, 201