21,703 research outputs found
Some conservative stopping rules for the operational testing of safety-critical software
Operational testing, which aims to generate sequences of test cases with the same statistical properties as those that would be experienced in real operational use, can be used to obtain quantitative measures of the reliability of software. In the case of safety critical software it is common to demand that all known faults are removed. This means that if there is a failure during the operational testing, the offending fault must be identified and removed. Thus an operational test for safety critical software takes the form of a specified number of test cases (or a specified period of working) that must be executed failure-free. This paper addresses the problem of specifying the numbers of test cases (or time periods) required for a test, when the previous test has terminated as a result of a failure. It has been proposed that, after the obligatory fix of the offending fault, the software should be treated as if it were completely novel, and be required to pass exactly the same test as originally specified. The reasoning here claims to be conservative, inasmuch as no credit is given for any previous failure-free operation prior to the failure that terminated the test. We show that, in fact, this is not a conservative approach in all cases, and propose instead some new Bayesian stopping rules. We show that the degree of conservatism in stopping rules depends upon the precise way in which the reliability requirement is expressed. We define a particular form of conservatism that seems desirable on intuitive grounds, and show that the stopping rules that exhibit this conservatism are also precisely the ones that seem preferable on other grounds
Sparse Conformal Predictors
Conformal predictors, introduced by Vovk et al. (2005), serve to build
prediction intervals by exploiting a notion of conformity of the new data point
with previously observed data. In the present paper, we propose a novel method
for constructing prediction intervals for the response variable in multivariate
linear models. The main emphasis is on sparse linear models, where only few of
the covariates have significant influence on the response variable even if
their number is very large. Our approach is based on combining the principle of
conformal prediction with the penalized least squares estimator
(LASSO). The resulting confidence set depends on a parameter and
has a coverage probability larger than or equal to . The numerical
experiments reported in the paper show that the length of the confidence set is
small. Furthermore, as a by-product of the proposed approach, we provide a
data-driven procedure for choosing the LASSO penalty. The selection power of
the method is illustrated on simulated data
Selecting the number of principal components: estimation of the true rank of a noisy matrix
Principal component analysis (PCA) is a well-known tool in multivariate
statistics. One significant challenge in using PCA is the choice of the number
of components. In order to address this challenge, we propose an exact
distribution-based method for hypothesis testing and construction of confidence
intervals for signals in a noisy matrix. Assuming Gaussian noise, we use the
conditional distribution of the singular values of a Wishart matrix and derive
exact hypothesis tests and confidence intervals for the true signals. Our paper
is based on the approach of Taylor, Loftus and Tibshirani (2013) for testing
the global null: we generalize it to test for any number of principal
components, and derive an integrated version with greater power. In simulation
studies we find that our proposed methods compare well to existing approaches.Comment: 29 pages, 9 figures, 4 table
Fixed Size Confidence Regions for Parameters of Stationary Processes Based on a Minimum Contrast Estimator
For parameters of stationary processes with zero mean and spectral density, sequential procedures are proposed for constructing fixed size confidence ellipsoidal regions for unknown parameters using a minimum contrast estimator. The confidence ellipsoids are shown to be asymptotically consistent and the associated stopping rules are shown to be asymptotically efficient as the size of the region becomes small when the assumed parametric model is correct. Monte Carlo simulations are given to investigate the performance of our proposed sequential procedures.
Recommended from our members
Some conservative stopping rules for the operational testing of saftey-critical software
Operational testing, which aims to generate sequences of test cases with the same statistical properties as those that would be experienced in real operational use, can be used to obtain quantitative measures of the reliability of software. In the case of safety critical software it is common to demand that all known faults are removed. This means that if there is a failure during the operational testing, the offending fault must be identified and removed. Thus an operational test for safety critical software takes the form of a specified number of test cases (or a specified period of working) that must be executed failure-free. This paper addresses the problem of specifiying the number of test cases (or time periods) required for a test, when the previous test has terminated as a result of a failue. It has been proposed that, after the obligatory fix of the offending fault, the software should be treated as if it were completely novel, and be required to pass exactly the same test as originally specified. The reasoning here claims to be conservative, inasmuch as no credit is given for any previous failure-free operation prior to the failure that terminated the test. We show that, in fact, this is not a conservative approach in all cases, and propose instead some new Bayesian stopping rules. We show that the degree of conservatism in stopping rules depends upon the precise way in which the reliability requirements is expressed. We show that some rules are 'completely' conservative and argue that these are also precisely the ones that should be preferred on intuitive grounds
Validation procedures in radiological diagnostic models. Neural network and logistic regression
The objective of this paper is to compare the performance of two predictive radiological models, logistic regression (LR) and neural network (NN), with five different resampling methods. One hundred and sixty-seven patients with proven calvarial lesions as the only known disease were enrolled. Clinical and CT data were used for LR and NN models. Both models were developed with cross validation, leave-one-out and three different bootstrap algorithms. The final results of each model were compared with error rate and the area under receiver operating characteristic curves (Az). The neural network obtained statistically higher Az than LR with cross validation. The remaining resampling validation methods did not reveal statistically significant differences between LR and NN rules. The neural network classifier performs better than the one based on logistic regression. This advantage is well detected by three-fold cross-validation, but remains unnoticed when leave-one-out or bootstrap algorithms are used.Skull, neoplasms, logistic regression, neural networks, receiver operating characteristic curve, statistics, resampling
Multi-path Probabilistic Available Bandwidth Estimation through Bayesian Active Learning
Knowing the largest rate at which data can be sent on an end-to-end path such
that the egress rate is equal to the ingress rate with high probability can be
very practical when choosing transmission rates in video streaming or selecting
peers in peer-to-peer applications. We introduce probabilistic available
bandwidth, which is defined in terms of ingress rates and egress rates of
traffic on a path, rather than in terms of capacity and utilization of the
constituent links of the path like the standard available bandwidth metric. In
this paper, we describe a distributed algorithm, based on a probabilistic
graphical model and Bayesian active learning, for simultaneously estimating the
probabilistic available bandwidth of multiple paths through a network. Our
procedure exploits the fact that each packet train provides information not
only about the path it traverses, but also about any path that shares a link
with the monitored path. Simulations and PlanetLab experiments indicate that
this process can dramatically reduce the number of probes required to generate
accurate estimates
- …