IRPS – An Efficient Test Data Generation Strategy For Pairwise Testing.

Software testing is an integral part of software engineering. Lack of testing often leads to disastrous consequences including loss of data, fortunes, and even lives

Development Of Interaction Test Data Generation Strategy With Input-Output Mapping Supports

Uniform strength t-way testing (where t represents interaction strength) forms the basis of interaction testing. However, t is rarely uniform in real world as not all interaction faults are solely constituted by these fixed t-interactions. Consequently, a general solution has been introduced: input-output based relationship interaction testing. Although useful, most existing strategy implementations are lacking in terms of the automated input-output mapping support (to translate the symbolic outputs back into actual data form) and test suite generation flexibility. In order to address these aforementioned issues, a non-deterministic input-output based relationship interaction testing strategy, AURA, has been developed. AURA strategy also integrated with post-processing automated input-output mapping support and flexible iteration control capability to support test suite generation flexibility. Experimental results indicated that AURA strategy is generating competitive test suite size against existing strategies (Density, ParaOrder, Union, TVG, PICT, AETG, ACA, GA-N, IPO-N, IPO, Jenny, SA and ACS). Specifically, this strategy is capable to generate the test suite size as optimized as other strategies for certain inputs. Lastly, the post-processing automated input-output mapping support and flexible iteration control capability are evaluated with experiments

The Effectiveness of <i>t</i>-Way Test Data Generation

Modern society is increasingly dependent on the correct functioning of software and increasingly so in areas that are considered safety related or safety critical. Therefore, there is an increasing need to be able to verify and validate that the software is in fact correct and will perform its intended function. Many approaches to this problem have been proposed; however, none seems likely to supplant the role of testing in the near future. If we accept that there is, and will be, a continuing need to be able to test software then the question becomes one of how can this be done effectively, both in terms of ability to detect errors and in terms of cost. One avenue of research that offers prospects of improving both of these aspects is the automatic generation of test data. There has recently been a large amount of work conducted in this area. One particularly promising direction has been the application of ideas from the field of experimental design and in particular, the field of t-way adequate factorial designs. The area however, is not without issues; there is evidence that the technique is capable of detecting errors but that evidence is not unequivocal. Moreover, as with almost all work in the area of automatic test generation, there has been very little comparative work comparing the technique with other test data generation techniques. Worse, there has been effectively no work done that compares any automatic test data generation technique with the effectiveness of tests generated by humans. Another major issue with the technique is the number of tests that applying the technique can result in. This implies that there is a need for an automated oracle if the technique is to be successfully applied. The flaw with this is of course that in most situations the oracle is the human that is conducting the tests, a point often ignored in testing research. The work presented here addresses both of these points. To do this I have used a code base taken from an industrial engine control system that has an existing set of high quality unit tests developed by hand. To complement this, several other techniques for automatically generating test data have been applied, namely random testing, random experimental designs and a technique for generating single factor experiments. To address the issue of being able to compare the error detection ability of all of the sets of test vectors, rather than the usual effectiveness surrogates of code coverage I have used mutation analysis on the code base to directly measure the ability of each set of test vectors to discover common coding errors. The results presented here show that test data generation techniques based on t-way factorial designs are at least as effective as handgenerated tests and superior to random testing and the factor experimental technique. The oracle problem associated with the factorial design techniques was addressed using a test set minimisation approach. The mutation tool monitored which vectors could “kill” which code mutants. After a subset of the test vectors had been run, the most effective vectors were retained and the rest discarded. Likewise, mutants that were killed were removed from further consideration and the process repeated. Experimental results show that this minimisation procedure is effective at reducing computational overhead and is capable of producing final sets of test vectors that are comparable in size with the sets of hand-generated tests and so amenable to final hand checking

An Enhanced Pairwise Search Approach for Generating Optimum Number of Test Data and Reduce Execution Time

In recent days testing considers the most important task for building software that is free from error. Since the resources and time is limited to produce software, hence, it is not possible of performing exhaustive tests (i.e. to test all possible combinations of input data.) An alternative to get ride from this type exhaustive numbers and as well to reduce cost, an approach called Pairwise (2 way) test data generation approach will be effective. Most of the software faults in pairwise approach caused by unusual combination of input data.  Hence, the demand for the optimization of number of generated test-cases and reducing the execution time is growing in software industries. This paper proposes an enhancement in pairwise search approach which generates optimum number of input values for testing purposes.  In this approach it searches the most coverable pairs by pairing parameters and adopts one-test-at-a-time strategy for constructing a final test-suite.  Compared to other existing strategies, Our proposed approach is effective in terms of number of generated test cases and of execution time. Keywords:, Software testing, Pairwise testing, Combinatorial interaction testing, Test case generation

PWiseHA: Application of Harmony Search Algorithm for Test Suites Generation using Pairwise Techniques

Pairwise testing is an approach that tests every possible combinations of values of parameters. In this approach, number of all combinations are selected to ensure all possible pairs of parameter values are included in the final test suite. Generating test cases is the most active research area in pairwise testing, but the generation process of the efficient test suite with minimum size can be considered as one of optimization problem. In this research paper we articulate the problem of finding a pairwise final test suite as a search problem and the application of harmony search algorithm to solve it. Also, in this research paper, we developed a pairwise software testing tool called PWiseHA that will generate test cases using harmony search algorithm and this PWiseHA is well optimized. Finally, the result obtained from PWiseHA shows a competitive results if matched with the result of existing pairwise testing tools. PWiseHA is still in prototype form, an obvious starting point for future work

Testing embedded system through optimal mining technique (OMT) based on multi-input domain

Testing embedded systems must be done carefully particularly in the significant regions of the embedded systems. Inputs from an embedded system can happen in multiple order and many relationships can exist among the input sequences. Consideration of the sequences and the relationships among the sequences is one of the most important considerations that must be tested to find the expected behavior of the embedded systems. On the other hand combinatorial approaches help determining fewer test cases that are quite enough to test the embedded systems exhaustively. In this paper, an Optimal Mining Technique that considers multi-input domain which is based on built-in combinatorial approaches has been presented. The method exploits multi-input sequences and the relationships that exist among multi-input vectors. The technique has been used for testing an embedded system that monitors and controls the temperature within the Nuclear reactors

A Requirements-Based Partition Testing Framework Using Particle Swarm Optimization Technique

Modern society is increasingly dependent on the quality of software systems. Software failure can cause severe consequences, including loss of human life. There are various ways of fault prevention and detection that can be deployed in different stages of software development. Testing is the most widely used approach for ensuring software quality. Requirements-Based Testing and Partition Testing are two of the widely used approaches for testing software systems. Although both of these techniques are mature and are addressed widely in the literature and despite the general agreement on both of these key techniques of functional testing, a combination of them lacks a systematic approach. In this thesis, we propose a framework along with a procedural process for testing a system using Requirements-Based Partition Testing (RBPT). This framework helps testers to start from the requirements documents and follow a straightforward step by step process to generate the required test cases without loosing any required data. Although many steps of the process are manual, the framework can be used as a foundation for automating the whole test case generation process. Another issue in testing a software product is the test case selection problem. Choosing appropriate test cases is an essential part of software testing that can lead to significant improvements in efficiency, as well as reduced costs of combinatorial testing. Unfortunately, the problem of finding minimum size test sets is NP-complete in general. Therefore, artificial intelligence-based search algorithms have been widely used for generating near-optimal solutions. In this thesis, we also propose a novel technique for test case generation using Particle Swarm Optimization (PSO), an effective optimization tool which has emerged in the last decade. Empirical studies show that in some domains particle swarm optimization is equally well-suited or even better than some other techniques. At the same time, a particle swarm algorithm is much simpler, easier to implement, and has just a few parameters that the user needs to adjust. These properties make PSO an ideal technique for test case generation. In order to have a fair comparison of our newly proposed algorithm against existing techniques, we have designed and implemented a framework for automatic evaluation of these methods. Through experiments using our evaluation framework, we illustrate how this new test case generation technique can outperform other existing methodologies

A Survey of Constrained Combinatorial Testing

Combinatorial Testing (CT) is a potentially powerful testing technique, whereas its failure revealing ability might be dramatically reduced if it fails to handle constraints in an adequate and efficient manner. To ensure the wider applicability of CT in the presence of constrained problem domains, large and diverse efforts have been invested towards the techniques and applications of constrained combinatorial testing. In this paper, we provide a comprehensive survey of representations, influences, and techniques that pertain to constraints in CT, covering 129 papers published between 1987 and 2018. This survey not only categorises the various constraint handling techniques, but also reviews comparatively less well-studied, yet potentially important, constraint identification and maintenance techniques. Since real-world programs are usually constrained, this survey can be of interest to researchers and practitioners who are looking to use and study constrained combinatorial testing techniques