On the selection of strength for fixed-strength interaction coverage based prioritization

Abstract

Abstract test cases are derived by modeling the system under test, and have been widely applied in practice, such as for software product line testing and combinatorial testing. Abstract test case prioritization (ATCP) is used to prioritize abstract test cases and aims at achieving higher rates of fault detection. Many ATCP algorithms have been proposed, using different prioritization criteria and information. One ATCP approach makes use of fixed-strength level-combinations information covered by abstract test cases, and is called fixed-strength interaction coverage based prioritization (FICBP). Before using FICBP, the prioritization strength λ needs to be decided. Previous studies have generally focused on λ values ranging between 1 and 6. However, no study has investigated the appropriateness of such a range, nor how to assign the prioritization strength for FICBP. To answer these questions, this paper reports on an empirical study involving four real-life programs (each of which with six versions). The experimental results indicate that λ should be set approximately equal to a value corresponding to half of the number of parameters, when testing resources are sufficient. Our results also show that when testing resources are limited or insufficient, either small or large λ values are suggested for FICBP