Test Case Prioritization for Software Product Line: A Systematic Mapping Study

Combinatorial explosion remains a common issue in testing. Due to the vast number of product variants, the number of test cases required for comprehensive coverage has significantly increased. One of the techniques to efficiently tackle this problem is prioritizing the test suites using a regression testing method. However, there is a lack of comprehensive reviews focusing on test case prioritization in SPLs. To address this research gap, this paper proposed a systematic mapping study to observe the extent of test case prioritization usage in Software Product Line Testing. The study aims to classify various aspects of SPL-TCP (Software Product Line – Test Case Prioritization), including methods, criteria, measurements, constraints, empirical studies, and domains. Over the last ten years, a thorough investigation uncovered twenty-four primary studies, consisting of 12 journal articles and 12 conference papers, all related to Test Case Prioritization for SPLs. This systematic mapping study presents a comprehensive classification of the different approaches to test case prioritization for Software Product Lines. This classification can be valuable in identifying the most suitable strategies to address specific challenges and serves as a guide for future research works. In conclusion, this mapping study systematically classifies different approaches to test case prioritization in Software Product Lines. The results of this study can serve as a valuable resource for addressing challenges in SPL testing and provide insights for future research

Enhancing similarity distances using mandatory and optional forearly fault detection

Software Product Line (SPL) describes procedures, techniques, and tools in software engineering by using a common method of production for producing a group of software systems that identical from a shared set of software assets. In SPL, the similarity-based prioritization can resemble combinatorial interaction testing in scalable and efficient way by choosing and prioritize configurations that most dissimilar. However, the similarity distances in SPL still not so much cover the basic detail of feature models which are the notations. Plus, the configurations always have been prioritized based on domain knowledge but not much attention has been paid to feature model notations. In this paper, we proposed the usage of mandatory and optional notations for similarity distances. The objective is to improve the average percentage of faults detected (APFD). We investigate four different distances and make modifications on the distances to increase APFD value. These modifications are the inclusion of mandatory and optional notations with the similarity distances. The results are the APFD values for all the similarity distances including the original and modified similarity distances. Overall, the results shown that by subtracting the optional notation value can increase the APFD by 3.71% from the original similarity distance

A Comparison of Test Case Prioritization Criteria for Software Product Lines

Software Product Line (SPL) testing is challenging due to the potentially huge number of derivable products. To alleviate this problem, numerous contributions have been proposed to reduce the number of products to be tested while still having a good coverage. However, not much attention has been paid to the order in which the products are tested. Test case prioritization techniques reorder test cases to meet a certain performance goal. For instance, testers may wish to order their test cases in order to detect faults as soon as possible, which would translate in faster feedback and earlier fault correction. in this paper, we explore the applicability of test case prioritization techniques to SPL testing. We propose five different prioritization criteria based on common metrics of feature models and we compare their effectiveness in increasing the rate of early fault detection, i.e. a measure of how quickly faults are detected. The results show that different orderings of the same SPL suite may lead to significant differences in the rate of early fault detection. They also show that our approach may contribute to accelerate the detection of faults of SPL test suites based on combinatorial testingMinisterio de Ciencia e Innovación TIN2009-07366 (SETI)Ministerio de Economía y Competitividad TIN2012-32273Junta de Andalucía P10-TIC-590

Variability testing in the wild: the Drupal case study

Variability testing techniques search for effective and manageable test suites that lead to the rapid detection of faults in systems with high variability. Evaluating the effectiveness of these techniques in realistic settings is a must, but challenging due to the lack of variability intensive systems with available code, automated tests and fault reports. In this article, we propose using the Drupal framework as a case study to evaluate variability testing techniques. First, we represent the framework variability using a feature model. Then, we report on extensive non–functional data extracted from the Drupal Git repository and the Drupal issue tracking system. Among other results, we identified 3,392 faults in single features and 160 faults triggered by the interaction of up to 4 features in Drupal v7.23. We also found positive correlations relating the number of bugs in Drupal features to their size, cyclomatic complexity, number of changes and fault history. To show the feasibility of our work, we evaluated the effectiveness of non–functional data for test case prioritization in Drupal. Results show that non–functional attributes are effective at accelerating the detection of faults, outperforming related prioritization criteria as test case similarity.Ministerio de Economía y Competitividad IPT-2012-0890-390000Ministerio de Economía y Competitividad TIN2012-3227