A Context-Sensitive Coverage Criterion for Test Suite Reduction

Modern software is increasingly developed using multi-language implementations, large supporting libraries and frameworks, callbacks, virtual function calls, reflection, multithreading, and object- and aspect-oriented programming. The predominant example of such software is the graphical user interface (GUI), which is used as a front-end to most of today's software applications. The characteristics of GUIs and other modern software present new challenges to software testing. Because recently developed techniques for automated test case generation can generate more tests than are practical to regularly execute, one important challenge is test suite reduction. Test suite reduction seeks to decrease the size of a test suite without overly compromising its original fault detection ability. This research advances the state-of-the-art in test suite reduction by empirically studying a coverage criterion which considers the context in which program concepts are covered. Conventional approaches to test suite reduction were developed and evaluated on batch-style applications and, due to the aforementioned considerations, are not always easily applicable to modern software. Furthermore, many existing techniques fail to consider the context in which code executes inside an event-driven paradigm, where programs wait for and interactively respond to user- and system-generated events. Consequently, they yield reduced test suites with severely impaired fault detection ability. The novel feature of this research is a test suite reduction technique based on the call stack coverage criterion which addresses many of the challenges associated with coverage-based test suite reduction in modern applications. Results show that reducing test suites while maintaining call stack coverage yields good tradeoffs between size reduction and fault detection effectiveness compared to traditional techniques. The output of this research includes models, metrics, algorithms, and techniques based upon this approach

A Survey on Software Testing Techniques using Genetic Algorithm

The overall aim of the software industry is to ensure delivery of high quality software to the end user. To ensure high quality software, it is required to test software. Testing ensures that software meets user specifications and requirements. However, the field of software testing has a number of underlying issues like effective generation of test cases, prioritisation of test cases etc which need to be tackled. These issues demand on effort, time and cost of the testing. Different techniques and methodologies have been proposed for taking care of these issues. Use of evolutionary algorithms for automatic test generation has been an area of interest for many researchers. Genetic Algorithm (GA) is one such form of evolutionary algorithms. In this research paper, we present a survey of GA approach for addressing the various issues encountered during software testing.Comment: 13 Page

Defining and Evaluating Test Suite Consolidation for Event Sequence-based Test Cases

This research presents a new test suite consolidation technique, called CONTEST, for automated GUI testing. A new probabilistic model of the GUI is developed to allow direct application of CONTEST. Multiple existing test suites are used to populate the model and compute probabilities based on the observed event sequences. These probabilities are used to generate a new test suite that consolidates the original ones. A new test suite similarity metric, called CONTeSSi(n), is introduced which compares multiple event sequence-based test suites using relative event positions. Results of empirical studies showed that CONTEST yields a test suite that achieves better fault detection and code coverage than the original suites, and that the CONTeSSi(n) metric is a better indicator of the similarity between sequence-based test suites than existing metrics

Dynamically Testing Graphical User Interfaces

Software test generation for GUIs is a hard problem. The goal of this thesis is to investigate different methods for dynamically generating tests for GUIs. We introduce the concept of an event-pair graph, which is used to represent and measure test suites, and show how it can be used to generate tests and measure GUI coverage. Before we can begin generating tests, we first want to determine which is better: a small test suite with a few long tests or a large test suite with many short tests. Therefore, we designed and conducted a study to determine which is more effective. We found that moderate to long tests perform better than short tests. We then move on to discuss seven test generation algorithms. Two are based on random selection, two are based on greedy selection, one is based on Q-Learning, and the last two are based on ant colony optimization. We conducted a study in order to compare the performance of each algorithm. We measured code coverage, GUI coverage, time to run, and faults found. The results show that the greedy algorithms performed the best. Finally, we conducted a study in order to determine if any of the GUI coverage metrics can be used to predict code coverage, and we conducted a study to determine if any of the coverage metrics can be used to predict the faults found. The results show that event pairs are good at predicting code coverage, and that predicting faults is difficult

Towards a Regression Test Selection Technique for Message-Based Software Integration

Regression testing is essential to ensure software quality. Regression Test-case selection is another process wherein, the testers would like to ensure that test-cases which are obsolete due to the changes in the system should not be considered for further testing. This is the Regression Test-case Selection problem. Although existing research has addressed many related problems, most of the existing regression test-case selection techniques cater to procedural systems. Being academic, they lack the scalability and detail to cater to multi-tier applications. Such techniques can be employed for procedural systems, usually mathematical applications. Enterprise applications have become complex and distributed leading to component-based architectures. Thus, inter-process communication has become a very important activity of any such system. Messaging is the most widely employed intermodule interaction mechanism. Today\u27s systems, being heavily internet dependent, are Web-Services based which utilize XML for messaging. We propose an RTS technique which is specifically targeted at enterprise applications

Towards a Regression Test Selection Technique for Message-Based Software Integration

Regression testing is essential to ensure software quality. Regression Test-case selection is another process wherein, the testers would like to ensure that test-cases which are obsolete due to the changes in the system should not be considered for further testing. This is the Regression Test-case Selection problem. Although existing research has addressed many related problems, most of the existing regression test-case selection techniques cater to procedural systems. Being academic, they lack the scalability and detail to cater to multi-tier applications. Such techniques can be employed for procedural systems, usually mathematical applications. Enterprise applications have become complex and distributed leading to component-based architectures. Thus, inter-process communication has become a very important activity of any such system. Messaging is the most widely employed intermodule interaction mechanism. Today\u27s systems, being heavily internet dependent, are Web-Services based which utilize XML for messaging. We propose an RTS technique which is specifically targeted at enterprise applications

Erlang Code Evolution Control

During the software lifecycle, a program can evolve several times for different reasons such as the optimisation of a bottle-neck, the refactoring of an obscure function, etc. These code changes often involve several functions or modules, so it can be difficult to know whether the correct behaviour of the previous releases has been preserved in the new release. Most developers rely on a previously defined test suite to check this behaviour preservation. We propose here an alternative approach to automatically obtain a test suite that specifically focusses on comparing the old and new versions of the code. Our test case generation is directed by a sophisticated combination of several already existing tools such as TypEr, CutEr, and PropEr; and other ideas such as allowing the programmer to chose an expression of interest that must preserve the behaviour, or the recording of the sequences of values to which this expression is evaluated. All the presented work has been implemented in an open-source tool that is publicly available on GitHub.Comment: Pre-proceedings paper presented at the 27th International Symposium on Logic-Based Program Synthesis and Transformation (LOPSTR 2017), Namur, Belgium, 10-12 October 2017 (arXiv:1708.07854