GUI Testing Techniques Evaluation by Designed Experiments

Industry uses different testing techniques for test case generation and execution. But in general no systematic evaluation is performed to identify which technique is better (for instance, to find bugs faster). This paper presents a statistical assessment of two GUI testing techniques, BxT and DH, which are used on Motorola phone applications. These techniques test applications by pressing certain phone keys, from certain screens and during some amount of time. We consider three exploration parameters for each technique in our design and analysis of experiments: Driven determines whether a test case always starts from a single initial state (screen) or set of initial states; KeyProb associates an occurrence probability for each phone key; and SizeTC refers to the number of steps a test can have (a fourth parameter is the Technique itself). As conclusions, we show that BxT is better than DH and the SizeTC and the Technique parameters and the combination Driven*SizeTC have significant effects on the time to find a bug.