Covering array sampling of input event sequences for automated gui testing

This paper describes a new automated technique to generate test cases for GUIs by using covering arrays (CAs). The key motivation is to generate long GUI event sequences that are systematically sampled at a particular coverage strength. CAs, to date, have not been effectively used in sampling event driven systems such as GUIs which maintain state. We leverage a “stateless ” abstraction of GUIs that allows us to use CAs. Once the CAs have been generated, we reuse the abstractions to reinsert ordering relationships between GUI events, thereby creating executable test cases. A feasi-bility study on a well-studied GUI-based application shows that the new technique is able to detect a large number of previously undetected faults

Automated GUI performance testing

A significant body of prior work has devised approaches for automating the functional testing of interactive applications. However, little work exists for automatically testing their performance. Performance testing imposes additional requirements upon GUI test automation tools: the tools have to be able to replay complex interactive sessions, and they have to avoid perturbing the application's performance. We study the feasibility of using five Java GUI capture and replay tools for GUI performance test automation. Besides confirming the severity of the previously known GUI element identification problem, we also describe a related problem, the temporal synchronization problem, which is of increasing importance for GUI applications that use timer-driven activity. We find that most of the tools we study have severe limitations when used for recording and replaying realistic sessions of real-world Java applications and that all of them suffer from the temporal synchronization problem. However, we find that the most reliable tool, Pounder, causes only limited perturbation and thus can be used to automate performance testing. Based on an investigation of Pounder's approach, we further improve its robustness and reduce its perturbation. Finally, we demonstrate in a set of case studies that the conclusions about perceptible performance drawn from manual tests still hold when using automated tests driven by Pounder. Besides the significance of our findings to GUI performance testing, the results are also relevant to capture and replay-based functional GUI test automation approache

Automated gui testing guided by usage profiles

Most software developed in recent years has a graphical user interface (GUI). The only way for the end-user to interact with the software application is through the GUI. Hence, acceptance and system testing of the software requires GUI testing. This paper presents a new technique for testing of GUI applications. Information on the actual usage of the application, in the form of “usage profiles, ” is used to ensure that a new version of the application will function correctly. Usage profiles, sequences of events that end-users execute on a GUI, are used to develop a probabilistic usage model of the application. An algorithm uses the model to generate test cases that represent events the user is most likely to execute. Reverse engineering methods are used to extract the underlying structure of the application. An em-pirical study on four open source GUI applications reveals that test suites generated from the probabilistic model are 0.2-22 % of the size of test suites produced directly from us-age profiles. Furthermore, the test suites generated from the model detect more faults per test case than those de-tected directly from the usage profiles, and detect faults not detected by the original profiles

Alternating GUI Test Generation and Execution

Users of today’s software perform tasks by interacting with a graphical user interface (GUI) front-end via se-quences of input events. Due to the flexibility offered by most GUIs, the number of event sequences grows exponen-tially with length. One ubiquitous challenge of GUI test-ing is to selectively generate those sequences that lead to potentially problematic states. This paper presents ALT, a new technique that generates GUI test cases in batches, by leveraging GUI run-time information from a previously run batch to obtain the next batch. Each successive batch con-sists of “longer ” test cases that expand the state space to be explored, yet prune the “unimportant ” states. The “alter-nating ” nature of ALT allows it to enhance the next batch by leveraging certain relationships between GUI events (e.g., one enables the other, one alters the other’s execution) that are revealed only at run-time and non-trivial to infer stati-cally. An empirical study on four fielded GUI-based appli-cations demonstrates that ALT is successful at identifying complex failure-causing interactions between GUI events.

Reducing GUI Test Suites via Program Slicing

ABSTRACT A crucial problem in GUI testing is the identification of accurate event sequences that encode corresponding user interactions with the GUI. Ultimately, event sequences should be both feasible (i. e., executable on the GUI) and relevant (i. e., cover as much of the code as possible). So far, most work on GUI testing focused on approaches to generate feasible event sequences. In addition, based on event dependency analyses, a recently proposed static analysis approach systematically aims at selecting both relevant and feasible event sequences. However, statically analyzing event dependencies can cause the generation of a huge number of event sequences, leading to unmanageable GUI test suites that are not executable within reasonable time. In this paper we propose a refined static analysis approach based on program slicing. On the theoretical side, our approach identifies and eliminates redundant event sequences in GUI test suites. Redundant event sequences have the property that they are guaranteed to not affect the test effectiveness. On the practical side, we have implemented a slicing-based test suite reduction algorithm that approximatively identifies redundant event sequences. Our experiments on six open source GUI applications show that our reduction algorithm significantly reduces the size of GUI test suites. As a result, the overall execution time could significantly be reduced without losing test effectiveness