Test Case Selection Using CBIR and Clustering

Choosing test cases for the optimization process of information systems testing is crucial, because it helps to eliminate unnecessary and redundant testing data. However, its use in systems that address complex domains (e.g. images) is still underexplored. This paper presents a new approach that uses Content-Based Image Retrieval (CBIR), similarity functions and clustering techniques to select test cases from an image-based test suite. Two experiments performed on an image processing system show that our approach, when compared with random tests, can significantly enhance the performance of tests execution by reducing the test cases required to find a fault. The results also show the potential use of CBIR for information abstraction, as well as the effectiveness of similarity functions and clustering for test case selection

Enhancing similarity distances using mandatory and optional forearly fault detection

Software Product Line (SPL) describes procedures, techniques, and tools in software engineering by using a common method of production for producing a group of software systems that identical from a shared set of software assets. In SPL, the similarity-based prioritization can resemble combinatorial interaction testing in scalable and efficient way by choosing and prioritize configurations that most dissimilar. However, the similarity distances in SPL still not so much cover the basic detail of feature models which are the notations. Plus, the configurations always have been prioritized based on domain knowledge but not much attention has been paid to feature model notations. In this paper, we proposed the usage of mandatory and optional notations for similarity distances. The objective is to improve the average percentage of faults detected (APFD). We investigate four different distances and make modifications on the distances to increase APFD value. These modifications are the inclusion of mandatory and optional notations with the similarity distances. The results are the APFD values for all the similarity distances including the original and modified similarity distances. Overall, the results shown that by subtracting the optional notation value can increase the APFD by 3.71% from the original similarity distance

A Comparison on Similarity Distances and Prioritization Techniques for Early Fault Detection Rate

Nowadays, the Software Product Line (SPL) had replaced the conventional product development system. Many researches have been carried out to ensure the SPL usage prune the benefits toward the recent technologies. However, there are still some problems exist within the concept itself, such as variability and commonality. Due to its variability, exhaustive testing is not possible. Various solutions have been proposed to lessen this problem. One of them is prioritization technique, in which it is used to arrange back the test cases to achieve a specific performance goal. In this paper, the early fault detection is selected as the performance goal. Similarity function is used within our prioritization approach. Five different types of prioritization techniques are used in the experiment. The experiment results indicate that the greed-aided-clustering ordered sequence (GOS) shows the highest rate of early fault detection

Search-based Similarity-driven Behavioural SPL Testing

International audienceDissimilar test cases have been proven to be effective to reveal faults in software systems. In the Software Product Line (SPL) context, this criterion has been applied successfully to mimic combinatorial interaction testing in an efficient and scalable manner by selecting and prioritising most dissimilar configurations of feature models using evolutionary algorithms. In this paper, we extend dissimilarity to behavioural SPL models (FTS) in a search-based approach, and evaluate its effectiveness in terms of product and fault coverage. We investigate different distances as well as as single-objective algorithms, (dissimilarity on actions, random , all-actions). Our results on four case studies show the relevance of dissimilarity-based test generation for be-havioural SPL models, especially on the largest case-study where no other approach can match it

Redefining and Evaluating Coverage Criteria Based on the Testing Scope

Test coverage information can help testers in deciding when to stop testing and in augmenting their test suites when the measured coverage is not deemed sufficient. Since the notion of a test criterion was introduced in the 70’s, research on coverage testing has been very active with much effort dedicated to the definition of new, more cost-effective, coverage criteria or to the adaptation of existing ones to a different domain. All these studies share the premise that after defining the entity to be covered (e.g., branches), one cannot consider a program to be adequately tested if some of its entities have never been exercised by any input data. However, it is not the case that all entities are of interest in every context. This is particularly true for several paradigms that emerged in the last decade (e.g., component-based development, service-oriented architecture). In such cases, traditional coverage metrics might not always provide meaningful information. In this thesis we address such situation and we redefine coverage criteria so to focus on the program parts that are relevant to the testing scope. We instantiate this general notion of scope-based coverage by introducing three coverage criteria and we demonstrate how they could be applied to different testing contexts. When applied to the context of software reuse, our approach proved to be useful for supporting test case prioritization, selection and minimization. Our studies showed that for prioritization we can improve the average rate of faults detected. For test case selection and minimization, we can considerably reduce the test suite size with small to no extra impact on fault detection effectiveness. When the source code is not available, such as in the service-oriented architecture paradigm, we propose an approach that customizes coverage, measured on invocations at service interface, based on data from similar users. We applied this approach to a real world application and, in our study, we were able to predict the entities that would be of interest for a given user with high precision. Finally, we introduce the first of its kind coverage criterion for operational profile based testing that exploits program spectra obtained from usage traces. Our study showed that it is better correlated than traditional coverage with the probability that the next test input will fail, which implies that our approach can provide a better stopping rule. Promising results were also observed for test case selection. Our redefinition of coverage criteria approaches the topic of coverage testing from a completely different angle. Such a novel perspective paves the way for new avenues of research towards improving the cost-effectiveness of testing, yet all to be explored