Asymptotic properties of SPS confidence regions

Sign-Perturbed Sums (SPS) is a system identification method that constructs non-asymptotic confidence regions for the parameters of linear regression models under mild statistical assumptions. One of its main features is that, for any finite number of data points and any user-specified probability, the constructed confidence region contains the true system parameter with exactly the user-chosen probability. In this paper we examine the size and the shape of the confidence regions, and we show that the regions are strongly consistent, i.e., they almost surely shrink around the true parameter as the number of data points increases. Furthermore, the confidence region is contained in a marginally inflated version of the confidence ellipsoid obtained from the asymptotic system identification theory. The results are also illustrated by a simulation example

Asymptotic properties of SPS confidence regions

Sign-Perturbed Sums (SPS) is a system identification method that constructs non-asymptotic confidence regions for the parameters of linear regression models under mild statistical assumptions. One of its main features is that, for any finite number of data points and any user-specified probability, the constructed confidence region contains the true system parameter with exactly the user-chosen probability. In this paper we examine the size and the shape of the confidence regions, and we show that the regions are strongly consistent, i.e., they almost surely shrink around the true parameter as the number of data points increases. Furthermore, the confidence region is contained in a marginally inflated version of the confidence ellipsoid obtained from the asymptotic system identification theory. The results are also illustrated by a simulation example

Perturbed Datasets Methods for Hypothesis Testing and Structure of Corresponding Confidence Sets

Hypothesis testing methods that do not rely on exact distribution assumptions have been emerging lately. The method of sign-perturbed sums (SPS) is capable of characterizing confidence regions with exact confidence levels for linear regression and linear dynamical systems parameter estimation problems if the noise distribution is symmetric. This paper describes a general family of hypothesis testing methods that have an exact user chosen confidence level based on finite sample count and without relying on an assumed noise distribution. It is shown that the SPS method belongs to this family and we provide another hypothesis test for the case where the symmetry assumption is replaced with exchangeability. In the case of linear regression problems it is shown that the confidence regions are connected, bounded and possibly non-convex sets in both cases. To highlight the importance of understanding the structure of confidence regions corresponding to such hypothesis tests it is shown that confidence sets for linear dynamical systems parameter estimates generated using the SPS method can have non-connected parts, which have far reaching consequences

Sign-perturbed sums: A new system identification approach for constructing exact non-asymptotic confidence regions in linear regression models

We propose a new system identification method, called Sign - Perturbed Sums (SPS), for constructing nonasymptotic confidence regions under mild statistical assumptions. SPS is introduced for linear regression models, including but not limited to FIR systems, and we show that the SPS confidence regions have exact confidence probabilities, i.e., they contain the true parameter with a user-chosen exact probability for any finite data set. Moreover, we also prove that the SPS regions are star convex with the Least-Squares (LS) estimate as a star center. The main assumptions of SPS are that the noise terms are independent and symmetrically distributed about zero, but they can be nonstationary, and their distributions need not be known. The paper also proposes a computationally efficient ellipsoidal outer approximation algorithm for SPS. Finally, SPS is demonstrated through a number of simulation experiments

Finite-sample system identification: An overview and a new correlation method

Finite-sample system identification algorithms can be used to build guaranteed confidence regions for unknown model parameters under mild statistical assumptions. It has been shown that in many circumstances these rigorously built regions are comparable in size and shape to those that could be built by resorting to the asymptotic theory. The latter sets are, however, not guaranteed for finite samples and can sometimes lead to misleading results. The general principles behind finite-sample methods make them virtually applicable to a large variety of even nonlinear systems. While these principles are simple enough, a rigorous treatment of the attendant technical issues makes the corresponding theory complex and not easy to access. This is believed to be one of the reasons why these methods have not yet received widespread acceptance by the identification community and this letter is meant to provide an easy access point to finite-sample system identification by presenting the fundamental ideas underlying these methods in a simplified manner. We then review three (classes of) methods that have been proposed so far-1) Leave-out Sign-dominant Correlation Regions (LSCR); 2) Sign-Perturbed Sums (SPS); 3) Perturbed Dataset Methods (PDMs). By identifying some difficulties inherent in these methods, we also propose in this letter a new sign-perturbation method based on correlation which overcome some of these difficulties

Sign-Perturbed Sums (SPS) with Asymmetric Noise: Robustness Analysis and Robustification Techniques

Sign-Perturbed Sums (SPS) is a recently developed finite sample system identification method that can build exact confidence regions for linear regression problems under mild statistical assumptions. The regions are well-shaped, e.g., they are centred around the least-squares (LS) estimate, star-convex and strongly consistent. One of the main assumptions of SPS is that the distribution of the noise terms are symmetric about zero. This paper analyses how robust SPS is with respect to the violation of this assumption and how it could be robustified with respect to non-symmetric noises. First, some alternative solutions are overviewed, then a robustness analysis is performed resulting in a robustified version of SPS. We also suggest a modification of SPS, called LAD-SPS, which builds exact confidence regions around the least-absolute deviation (LAD) estimate instead of the LS estimate. LAD-SPS requires less assumptions as the noise needs only to have a conditionally zero median (w.r.t. the past). Furthermore, that approach can also be robustified using similar ideas as in the LS-SPS case. Finally, some numerical experiments are presented

Guaranteed Non-Asymptotic Confidence Ellipsoids for FIR Systems

Recently, a new finite-sample system identification algorithm, called Sign-Perturbed Sums (SPS), was introduced in [2]. SPS constructs finite-sample confidence regions that are centered around the least squares estimate, and are guaranteed to contain the true system parameters with a user-chosen exact probability for any finite number of data points. The main assumption of SPS is that the noise terms are independent and symmetrically distributed about zero, but they do not have to be stationary, nor do their variances and distributions have to be known. Although it is easy to determine if a particular parameter belongs to the confidence region, it is not easy to describe the boundary of the region, and hence to compactly represent the exact confidence region. In this paper we show that an ellipsoidal outer-approximation of the SPS confidence region can be found by solving a convex optimization problem, and we illustrate the properties of the SPS region and the ellipsoidal outer-approximation in simulation examples

Mapping the QCD Phase Diagram

I review recent theoretical developments which show how a key qualitative feature of the QCD phase diagram, namely a critical point which in a sense defines the landscape which heavy ion collision experiments are seeking to map, can be discovered. The map of the phase diagram which I sketch is based on reasonable inference from universality, lattice gauge theory and models; the discovery of the critical point would provide an experimental foundation for the central qualitative feature of the landscape. I also review recent progress in our understanding of cold, dense quark matter, as may occur in the cores of neutron stars. In this regime, quarks form Cooper pairs. The formation of such superconducting phases requires only weak attractive interactions, as provided by one-gluon exchange at asymptotically high density; these phases may nevertheless break chiral symmetry (by locking flavor symmetries to color symmetry) and may have excitations which are indistinguishable from those in a confined phase. Mapping this part of the phase diagram will require a better understanding of how the presence of color superconductivity and color-flavor locking affects neutron star phenomenology.Comment: Contribution to proceedings of Quark Matter '99, Torino, Italy. 12 pages. 4 figure

Undermodelling Detection with Sign-Perturbed Sums

Sign-Perturbed Sums (SPS) is a finite sample system identification method that can build exact confidence regions for the unknown parameters of linear systems under mild statistical assumptions. Theoretical studies of the SPS method have assumed so far that the order of the system model is known to the user. In this paper we discuss the implications of this assumption for the applicability of the SPS method, and we propose an extension that, under mild assumptions, i) still delivers guaranteed confidence regions when the model order is correct, and ii) it is guaranteed to detect, in the long run, if the model order is wrong