441,392 research outputs found
Finding a boundary between valid and invalid regions of the input space
In the context of robustness testing, the boundary between the valid and
invalid regions of the input space can be an interesting source of erroneous
inputs. Knowing where a specific software under test (SUT) has a boundary is
essential for validation in relation to requirements. However, finding where a
SUT actually implements the boundary is a non-trivial problem that has not
gotten much attention. This paper proposes a method of finding the boundary
between the valid and invalid regions of the input space. The proposed method
consists of two steps. First, test data generators, directed by a search
algorithm to maximise distance to known, valid test cases, generate valid test
cases that are closer to the boundary. Second, these valid test cases undergo
mutations to try to push them over the boundary and into the invalid part of
the input space. This results in a pair of test sets, one consisting of test
cases on the valid side of the boundary and a matched set on the outer side,
with only a small distance between the two sets. The method is evaluated on a
number of examples from the standard library of a modern programming language.
We propose a method of determining the boundary between valid and invalid
regions of the input space and apply it on a SUT that has a non-contiguous
valid region of the input space. From the small distance between the developed
pairs of test sets, and the fact that one test set contains valid test cases
and the other invalid test cases, we conclude that the pair of test sets
described the boundary between the valid and invalid regions of that input
space. Differences of behaviour can be observed between different distances and
sets of mutation operators, but all show that the method is able to identify
the boundary between the valid and invalid regions of the input space. This is
an important step towards more automated robustness testing.Comment: 10 pages, conferenc
Recommended from our members
Scientists and software engineers: A tale of two cultures
The two cultures of the title are those observed in my field studies: the culture of scientists (financial
mathematicians, earth and planetary scientists, and molecular biologists) developing their own software, and the culture of software engineers developing scientific software. In this paper, I shall describe some problems arising when scientists and software engineers come together to develop scientific software and discuss how these problems may be ascribed to their two different cultures
Interactively Test Driving an Object Detector: Estimating Performance on Unlabeled Data
In this paper, we study the problem of `test-driving' a detector, i.e.
allowing a human user to get a quick sense of how well the detector generalizes
to their specific requirement. To this end, we present the first system that
estimates detector performance interactively without extensive ground truthing
using a human in the loop. We approach this as a problem of estimating
proportions and show that it is possible to make accurate inferences on the
proportion of classes or groups within a large data collection by observing
only of samples from the data. In estimating the false detections (for
precision), the samples are chosen carefully such that the overall
characteristics of the data collection are preserved. Next, inspired by its use
in estimating disease propagation we apply pooled testing approaches to
estimate missed detections (for recall) from the dataset. The estimates thus
obtained are close to the ones obtained using ground truth, thus reducing the
need for extensive labeling which is expensive and time consuming.Comment: Published at Winter Conference on Applications of Computer Vision,
201
- β¦