Quality Week Europe ’98 Evolutionary Testing of Temporal Correctness

Abstract

The development of real-time systems is an essential industrial activity. Dynamic testing is the most important analytical method to assure the quality of real-time systems. It is the only method that examines the run-time behavior, based on an execution in the application environment. An investigation of existing software test methods shows that they mostly concentrate on testing for functional correctness. They are not specialized in the examination of temporal correctness that is essential to real-time systems. Therefore, existing test procedures must be supplemented by new methods that concentrate on determining whether the system violates its specified timing constraints. In general, a violation means that outputs are produced too early or their computation takes too long. The task of the tester is to find the inputs with the longest or shortest execution times to check whether they produce a temporal error. If the search for such inputs is interpreted as a problem of optimization, genetic algorithms can be used to find the inputs with the longest or shortest execution times automatically. The fitness function is the measured execution time. The use of genetic algorithms for testing is called evolutionary testing. Experiments using evolutionary testing on a number of programs with up to 1511 LOC and 843 input parameters have successfully identified new longer and shorter execution times than those that had been found using random testing and systematic testing. Evolutionary testing therefore seems to be well-suited for checking the temporal correctness of real-time software. A combination of evolutionary testing with systematic testing offers further opportunities to improve the test quality and is suggested in this paper as an effective test strategy for real-time systems.