Hypothesis testing methods that do not rely on exact distribution assumptions
have been emerging lately. The method of sign-perturbed sums (SPS) is capable
of characterizing confidence regions with exact confidence levels for linear
regression and linear dynamical systems parameter estimation problems if the
noise distribution is symmetric. This paper describes a general family of
hypothesis testing methods that have an exact user chosen confidence level
based on finite sample count and without relying on an assumed noise
distribution. It is shown that the SPS method belongs to this family and we
provide another hypothesis test for the case where the symmetry assumption is
replaced with exchangeability. In the case of linear regression problems it is
shown that the confidence regions are connected, bounded and possibly
non-convex sets in both cases. To highlight the importance of understanding the
structure of confidence regions corresponding to such hypothesis tests it is
shown that confidence sets for linear dynamical systems parameter estimates
generated using the SPS method can have non-connected parts, which have far
reaching consequences