Experience With The Cost Of Different Coverage Goals For Testing

Abstract

this paper. What percentage of branch, loop, multi-condition, and weak mutation coverage can be expected from thorough unit testing? Seven units from application programs were tested using an extension of traditional black box testing. Nearly 100% feasible coverage was consistently achieved. Except for weak mutation, the additional cost of reaching 100% feasible coverage required only a few percent of the total time. The high cost for weak mutation was due to the time spent identifying impossible coverage conditions. Because the incremental cost of coverage is low, it is reasonable to set a unit testing goal of 100% for branch, loop, multi-condition, and a subset of weak mutation coverage. However, reaching that goal after measuring coverage is less important than nearly reaching it with the initial black box test suite. A low initial coverage signals a problem in the testing process. BIOGRAPHICAL SKETCH Brian Marick graduated from the University of Illinois in 1981 with a BA in English Literature and a BS in Mathematics and Computer Science. Until 1989, he worked in product development as a programmer, tester, and line manager, while attending graduate school as a hobby. The work reported here was a result of a research project funded by a Motorola Partnerships in Research grant through Motorola's Software Research Division. He is currently applying the techniques and tools reported here to Motorola products, training others in their use, and planning further investigations of cost-effective and predictable testing techniques. Author's address: Motorola, 1101 E. University, Urbana, IL 61801. Phone: (217) 384-8500 Email: [email protected] or [email protected]. 1. Introductio