Undermodelling Detection with Sign-Perturbed Sums

Sign-Perturbed Sums (SPS) is a finite sample system identification method that can build exact confidence regions for the unknown parameters of linear systems under mild statistical assumptions. Theoretical studies of the SPS method have assumed so far that the order of the system model is known to the user. In this paper we discuss the implications of this assumption for the applicability of the SPS method, and we propose an extension that, under mild assumptions, i) still delivers guaranteed confidence regions when the model order is correct, and ii) it is guaranteed to detect, in the long run, if the model order is wrong

Sign-Perturbed Sums (SPS) with Asymmetric Noise: Robustness Analysis and Robustification Techniques

Sign-Perturbed Sums (SPS) is a recently developed finite sample system identification method that can build exact confidence regions for linear regression problems under mild statistical assumptions. The regions are well-shaped, e.g., they are centred around the least-squares (LS) estimate, star-convex and strongly consistent. One of the main assumptions of SPS is that the distribution of the noise terms are symmetric about zero. This paper analyses how robust SPS is with respect to the violation of this assumption and how it could be robustified with respect to non-symmetric noises. First, some alternative solutions are overviewed, then a robustness analysis is performed resulting in a robustified version of SPS. We also suggest a modification of SPS, called LAD-SPS, which builds exact confidence regions around the least-absolute deviation (LAD) estimate instead of the LS estimate. LAD-SPS requires less assumptions as the noise needs only to have a conditionally zero median (w.r.t. the past). Furthermore, that approach can also be robustified using similar ideas as in the LS-SPS case. Finally, some numerical experiments are presented

Asymptotic properties of SPS confidence regions

Sign-Perturbed Sums (SPS) is a system identification method that constructs non-asymptotic confidence regions for the parameters of linear regression models under mild statistical assumptions. One of its main features is that, for any finite number of data points and any user-specified probability, the constructed confidence region contains the true system parameter with exactly the user-chosen probability. In this paper we examine the size and the shape of the confidence regions, and we show that the regions are strongly consistent, i.e., they almost surely shrink around the true parameter as the number of data points increases. Furthermore, the confidence region is contained in a marginally inflated version of the confidence ellipsoid obtained from the asymptotic system identification theory. The results are also illustrated by a simulation example

Asymptotic properties of SPS confidence regions

Sign-Perturbed Sums (SPS) is a system identification method that constructs non-asymptotic confidence regions for the parameters of linear regression models under mild statistical assumptions. One of its main features is that, for any finite number of data points and any user-specified probability, the constructed confidence region contains the true system parameter with exactly the user-chosen probability. In this paper we examine the size and the shape of the confidence regions, and we show that the regions are strongly consistent, i.e., they almost surely shrink around the true parameter as the number of data points increases. Furthermore, the confidence region is contained in a marginally inflated version of the confidence ellipsoid obtained from the asymptotic system identification theory. The results are also illustrated by a simulation example

Sign-perturbed sums: A new system identification approach for constructing exact non-asymptotic confidence regions in linear regression models

We propose a new system identification method, called Sign - Perturbed Sums (SPS), for constructing nonasymptotic confidence regions under mild statistical assumptions. SPS is introduced for linear regression models, including but not limited to FIR systems, and we show that the SPS confidence regions have exact confidence probabilities, i.e., they contain the true parameter with a user-chosen exact probability for any finite data set. Moreover, we also prove that the SPS regions are star convex with the Least-Squares (LS) estimate as a star center. The main assumptions of SPS are that the noise terms are independent and symmetrically distributed about zero, but they can be nonstationary, and their distributions need not be known. The paper also proposes a computationally efficient ellipsoidal outer approximation algorithm for SPS. Finally, SPS is demonstrated through a number of simulation experiments

Guaranteed Non-Asymptotic Confidence Ellipsoids for FIR Systems

Recently, a new finite-sample system identification algorithm, called Sign-Perturbed Sums (SPS), was introduced in [2]. SPS constructs finite-sample confidence regions that are centered around the least squares estimate, and are guaranteed to contain the true system parameters with a user-chosen exact probability for any finite number of data points. The main assumption of SPS is that the noise terms are independent and symmetrically distributed about zero, but they do not have to be stationary, nor do their variances and distributions have to be known. Although it is easy to determine if a particular parameter belongs to the confidence region, it is not easy to describe the boundary of the region, and hence to compactly represent the exact confidence region. In this paper we show that an ellipsoidal outer-approximation of the SPS confidence region can be found by solving a convex optimization problem, and we illustrate the properties of the SPS region and the ellipsoidal outer-approximation in simulation examples

Finite-sample system identification: An overview and a new correlation method

Finite-sample system identification algorithms can be used to build guaranteed confidence regions for unknown model parameters under mild statistical assumptions. It has been shown that in many circumstances these rigorously built regions are comparable in size and shape to those that could be built by resorting to the asymptotic theory. The latter sets are, however, not guaranteed for finite samples and can sometimes lead to misleading results. The general principles behind finite-sample methods make them virtually applicable to a large variety of even nonlinear systems. While these principles are simple enough, a rigorous treatment of the attendant technical issues makes the corresponding theory complex and not easy to access. This is believed to be one of the reasons why these methods have not yet received widespread acceptance by the identification community and this letter is meant to provide an easy access point to finite-sample system identification by presenting the fundamental ideas underlying these methods in a simplified manner. We then review three (classes of) methods that have been proposed so far-1) Leave-out Sign-dominant Correlation Regions (LSCR); 2) Sign-Perturbed Sums (SPS); 3) Perturbed Dataset Methods (PDMs). By identifying some difficulties inherent in these methods, we also propose in this letter a new sign-perturbation method based on correlation which overcome some of these difficulties

Towards D-Optimal Input Design for Finite-Sample System Identification

Finite-sample system identification methods provide statistical inference, typically in the form of confidence regions, with rigorous non-asymptotic guarantees under minimal distributional assumptions. Data Perturbation (DP) methods constitute an important class of such algorithms, which includes, for example, Sign-Perturbed Sums (SPS) as a special case. Here we study a natural input design problem for DP methods in linear regression models, where we want to select the regressors in a way that the expected volume of the resulting confidence regions are minimized. We suggest a general approach to this problem and analyze it for the fundamental building blocks of all DP confidence regions, namely, for ellipsoids having confidence probability exactly 1/2. We also present experiments supporting that minimizing the expected volumes of such ellipsoids significantly reduces the average sizes of the constructed DP confidence regions