Assessing the influence of multiple test case selection on mutation experiments

Mutation testing is widely used in experiments.\ud Some papers experiment with mutation directly, while others\ud use it to introduce faults to measure the effectiveness of tests\ud created by other methods. There is some random variation in the\ud mutation score depending on the specific test values used. When\ud generating tests to use in experiments, a common, although not\ud universal practice, is to generate multiple sets of tests to satisfy\ud the same criterion or according to the same procedure, and then\ud to compute their average performance. Averaging over multiple\ud test sets is thought to reduce the variation in the mutation score.\ud This practice is extremely expensive when tests are generated by\ud hand (as is common) and as the number of programs increase (a\ud current positive trend in software engineering experimentation).\ud The research reported in this short paper asks a simple\ud and direct question: do we need to generate multiple sets of\ud test cases? That is, how do different test sets influence the\ud cost and effectiveness results? In a controlled experiment, we\ud generated 10 different test cases to be adequate for the Statement\ud Deletion (SSDL) mutation operator for 39 small programs and\ud functions, and then evaluated how they differ in terms of cost and\ud effectiveness. We found that averaging over multiple programs\ud was effective in reducing the variance in the mutation scores\ud introduced by specific testsFAPESP (número processo 2012/16950-5

The Role of Public Policies in the Development of the Brazilian Automotive Industry

Since the 1950s, the Brazilian automotive sector has been supported by different public policies. Despite the importance of the industry, there are no recent studies that analyze how the different public policies have shaped the progress of this sector since the beginning of the public policy phases. This paper evaluates how public policies contributed to the development of the Brazilian automotive sector. To capture those impacts, it is necessary to develop a methodological approach supported by a systematic review of the literature and research specific indicators that will allow measuring the results. The results indicate that each of the public policies contributed different mechanisms for the redefinition of local strategies for the development of products, investments in technologies and innovation, and the installation of new automakers in Brazil. The main limitations of the research are a large volume of fragmented information and the results of public policies are not always easily measurable and available in order to allow a more concrete analysis of their effectiveness or provide elements that support the reformulation of new public strategies

Test Case Selection Using CBIR and Clustering

Choosing test cases for the optimization process of information systems testing is crucial, because it helps to eliminate unnecessary and redundant testing data. However, its use in systems that address complex domains (e.g. images) is still underexplored. This paper presents a new approach that uses Content-Based Image Retrieval (CBIR), similarity functions and clustering techniques to select test cases from an image-based test suite. Two experiments performed on an image processing system show that our approach, when compared with random tests, can significantly enhance the performance of tests execution by reducing the test cases required to find a fault. The results also show the potential use of CBIR for information abstraction, as well as the effectiveness of similarity functions and clustering for test case selection

Establishing theoretical minimal sets of mutants

Mutation analysis generates tests that distinguish\ud variations, or mutants, of an artifact from the original. Mutation\ud analysis is widely considered to be a powerful approach to testing,\ud and hence is often used to evaluate other test criteria in terms of\ud mutation score, which is the fraction of mutants that are killed\ud by a test set. But mutation analysis is also known to provide\ud large numbers of redundant mutants, and these mutants can\ud inflate the mutation score. While mutation approaches broadly\ud characterized as reduced mutation try to eliminate redundant\ud mutants, the literature lacks a theoretical result that articulates\ud just how many mutants are needed in any given situation. Hence,\ud there is, at present, no way to characterize the contribution\ud of, for example, a particular approach to reduced mutation\ud with respect to any theoretical minimal set of mutants. This\ud paper’s contribution is to provide such a theoretical foundation\ud for mutant set minimization. The central theoretical result of the\ud paper shows how to minimize efficiently mutant sets with respect\ud to a set of test cases. We evaluate our method with a widely-used\ud benchmark.FAPESP (número processo 2012/16950-5

Turismo e Sustentabilidade

A alardeada convicção de que o turismo pode dar um impulso extraordinário à economia de um lugar mostra-se com frequência uma ilusão: prevalecem o abandono de ocupações tradicionais e a migração para as cidades. Assim como as monoculturas exploraram e continuam explorando territórios e gentes, a centralidade dada ao turismo como fator de desenvolvimento regional suga recursos públicos e concentra benefícios, geralmente com destaque para agentes de outros locais

Designing deletion mutation operators

Mutation analysis modifies a program by applying\ud syntactic rules, called mutation operators, systematically to create\ud many versions of the program (mutants) that differ in small\ud ways. Testers then design tests to cause the mutants to behave\ud differently from the original program. Mutation testing is widely\ud considered to result in very effective tests, however, it is also quite\ud costly. Cost comes from the many mutants that are created, the\ud number of tests that are needed to kill the mutants, and the\ud difficulty of deciding whether mutants behave equivalently to\ud the original program. One-op mutation theorizes that cost can be\ud reduced by using a single, very powerful, mutation operator that\ud leads to tests that are almost as effective as if all operators are\ud used. Previous research proposed the statement deletion operator\ud (SDL) and found promising results. This paper investigates the\ud use of SDL-mutation in a new context, the language C, and poses\ud additional empirical questions, including whether other operators\ud can be used. We carried out a controlled experiment in which\ud cost and effectiveness of each individual C mutation operator\ud were collected for 39 different subject programs. Experimental\ud data are used to define a cost-effectiveness metric to choose the\ud best single operator for one-op mutation.FAPESP (número processo 2012/16950-5

Experimental evaluation of SDL and One-Op mutation for C

Mutation analysis modifies a program by applying\ud syntactic rules, called mutation operators, systematically to create\ud many versions of the program (mutants) that differ in small\ud ways. Testers then design tests to cause the mutants to behave\ud differently from the original program. Mutation testing is widely\ud considered to result in very effective tests, however, it is also quite\ud costly. Cost comes from the many mutants that are created, the\ud number of tests that are needed to kill the mutants, and the\ud difficulty of deciding whether mutants behave equivalently to\ud the original program. One-op mutation theorizes that cost can be\ud reduced by using a single, very powerful, mutation operator that\ud leads to tests that are almost as effective as if all operators are\ud used. Previous research proposed the statement deletion operator\ud (SDL) and found promising results. This paper investigates the\ud use of SDL-mutation in a new context, the language C, and poses\ud additional empirical questions, including whether other operators\ud can be used. We carried out a controlled experiment in which\ud cost and effectiveness of each individual C mutation operator\ud were collected for 39 different subject programs. Experimental\ud data are used to define a cost-effectiveness metric to choose the\ud best single operator for one-op mutation.FAPESP (número processo 2012/16950-5

Mutant subsumption graphs

Mutation testing researchers have long known that\ud many generated mutants are not needed. This paper develops a\ud graph model to describe redundancy among mutations. We define\ud “true” subsumption, a relation that practicing test engineers\ud would like to have, but cannot due to issues of computability.\ud We also define dynamic subsumption and static subsumption as\ud approximations of “true” subsumption. We explore the properties\ud of the approximate subsumption relations in the context of a small\ud example. We suggest possible uses for subsumption graphs.FAPESP (número processo 2012/16950-5