A case study in statistical testing of reusable concurrent objects

Abstract. A test strategy is presented which makes use of the information got from OO analysis and design documents to determine the testing levels (unit, integration) and the associated test objectives. It defines solutions for some of the OO testing issues: here, emphasis is put on applications which consist of concurrent objects linked by client-server relationships. Two major concerns have guided the choice of the proposed techniques: component reusability, and nondeterminism induced by asynchronous communication between objects. The strategy is illustrated on a control program for an existing production cell taken from a metal-processing plant in Karlsruhe. The program was developed using the Fusion method and implemented in Ada 95. We used a probabilistic method for generating test inputs, called statistical testing. Test experiments were conducted from the unit to the system levels, and a few errors were detected

The optimisation of stochastic grammars to enable cost-effective probabilistic structural testing

Abstract The effectiveness of statistical testing, a probabilistic structural testing strategy, depends on the characteristics of the probability distribution from which test inputs are sampled. Metaheuristic search has been shown to be a practical method of optimising the characteristics of such distributions. However, the applicability of the existing search-based algorithm is limited by the requirement that the software’s inputs must be a fixed number of ordinal values. In this paper we propose a new algorithm that relaxes this limitation and so permits the derivation of probability distributions for a much wider range of software. The representation used by the new algorithm is based on a stochastic grammar supplemented with two novel features: conditional production weights and the dynamic partitioning of ordinal ranges. We demonstrate empirically that a search algorithm using this representation can optimise probability distributions over complex input domains and thereby enable cost-effective statistical testing, and that the use of both conditional production weights and dynamic partitioning can be beneficial to the search process

An Experimentation with Statistical Testing

Abstract Statistical testing is based on a probabilistic generation of test data: classical structural or functional criteria serve as guides for defining an input profile and a test size. The method is intended to compensate for the imperfect connection of current criteria with software faults, and should not be confused with random testing, a "blind" approach that uses a uniform profile over the input domain. This paper reports on experimental results obtained on a software component from the nuclear field: • unit testing of four functions -statistical input sets were designed according to structural criteria; their efficiency was compared to the one of 1) deterministic sets derived from the same criteria and 2) uniform random sets; the comparison involved 2816 faults of mutation type seeded one by one in the source codes. • whole component testing -statistical functional testing was designed from behaviour models of the component: finite state machines, decision tables, Statecharts; its efficiency was compared to the one of random testing, using two versions of the component: the real one, in which a minor fault was found, and a student version with 12 revealed faults. The results showed the high fault revealing power of statistical testing, and its best efficiency in comparison to deterministic and random testing

Empirical Studies of Object-Oriented Artifacts, Methods, and Processes: State of The Art and Future Directions

This paper summarizes the results of a working group at the Empirical Studies of Softwar

Testing levels for object-oriented software

One of the characteristics of object-oriented software is the complex dependency that may exist between classes due to inheritance, association and aggregation relationships. Hence, where to start testing and how to define an integration strategy are issues that require further investigation. This paper presents an approach to define a test order by exploiting a model produced during design stages (e.g., using OMT, UML), namely the class diagram. Our goal is to minimize the number of stubs to be cons