A Survey of Constrained Combinatorial Testing

Combinatorial Testing (CT) is a potentially powerful testing technique, whereas its failure revealing ability might be dramatically reduced if it fails to handle constraints in an adequate and efficient manner. To ensure the wider applicability of CT in the presence of constrained problem domains, large and diverse efforts have been invested towards the techniques and applications of constrained combinatorial testing. In this paper, we provide a comprehensive survey of representations, influences, and techniques that pertain to constraints in CT, covering 129 papers published between 1987 and 2018. This survey not only categorises the various constraint handling techniques, but also reviews comparatively less well-studied, yet potentially important, constraint identification and maintenance techniques. Since real-world programs are usually constrained, this survey can be of interest to researchers and practitioners who are looking to use and study constrained combinatorial testing techniques

Priority Integration for Weighted Combinatorial Testing

Priorities (weights) for parameter values can improve the effectiveness of combinatorial testing. Previous approaches have employed weights to derive high-priority test cases either earlier or more frequently. Our approach integrates these order-focused and frequency-focused prioritizations. We show that our priority integration realizes a small test suite providing high-priority test cases early and frequently in a good balance. We also propose two algorithms that apply our priority integration to existing combinatorial test generation algorithms. Experimental results using numerous test models show that our approach improves the existing approaches w.r.t. Order-focused and frequency-focused metrics, while overheads in the size and generation time of test suites are small.QC 20170109</p