Comparing GUI Functional System Testing with Functional System Logic Testing - An Experiment

The practitioner interested in reducing software verification effort may found herself lost in the many alternative definitions of Graphical User Interface (GUI) testing that exist and their relation to the notion of system testing. One result of these many definitions is that one may end up testi ng twice the same parts of the Software Under Test (SUT), specifically the application logic code. To clarify two important testing activities for the avoidance of duplicate testing effort, this paper studies possible differences between GUI testing and system testing experimentally. Specifically, we selected a SUT equipped with system tests that directly exercise the application code; We used GUITAR, a well-known GUI testing software to GUI test this SUT. Experimental results show important differences between system testing and GUI testing in terms of structural coverage and test cost

Test Generation for Graphical User Interfaces Based on Symbolic Execution

While Graphical User Interfaces (GUIs) have become ubiquitous, testing them remains largely ad-hoc. Since the state of a GUI is defined by a sequence of events on the GUI's widgets, a test input for a GUI is such an event sequence. Due to the combinatorial nature of the sequences, testing a GUI thoroughly is problematic and time-consuming. Moreover, the wide range of possible values for certain GUI widgets, such as a textbox, compounds the problem. This paper presents a novel test generation approach based on symbolic execution to obtain data inputs and enumerate event sequences that are likely to maximize code coverage of a GUI application. Key contributions are introducing the technique of symbolic execution in GUI testing (addressing a common weakness of traditional GUI testing frameworks) and performing symbolic execution over strings (in addition to primitives). Doing so minimizes the number of event sequences that form the resulting test suite. To determine feasibility of path conditions that arise in symbolic execution, we implement a solver for constraints over strings (in addition to primitives). We evaluate our test generation approach

A Comparative Study of GUI Testing Approaches

A maioria das aplicações de software modernas apresentam uma Interface Gráfica de Utilizador (GUI), que torna a aplicação mais simples de usar, promovendo maior produtividade e melhor acessibilidade, e oferecendo flexibilidade na forma como os utilizadores podem executar tarefas. No entanto, devido à complexidade das GUIs, o processo de teste de GUI pode ser moroso e intensivo. Assim, automatizar o processo tanto quanto possível é indispensável para testar qualquer interface gráfica mais complexa e evoluída.Existem diferentes abordagens para testes de GUI automatizados, no entanto, a maioria delas requerer esforços manuais substanciais, outras apenas são capazes de encontrar erros específicos ou não permitem a reutilização de casos de teste após alterações de sistema ou GUI. Muitos investigadores afirmam que devem ser utilizadas diferentes técnicas/abordagens para um bom processo de teste.Uma nova abordagem baseada em modelos, denominada de Pattern-Based GUI Testing, foi implementada a fim de aumentar a sistematização, reutilização e diminuir o esforço da modelação e teste de GUIs. Baseia-se no conceito de Padrões de Teste de Interface de Utilizador (UITP), que contêm estratégias de teste genéricas para testar características recorrentes e comuns (UI Patterns) em GUIs. É apoiada pela ferramenta PBGT, que integra um ambiente de modelação e execução de testes de modo a suportar a criação de modelos de teste com base em UITPs, com recurso a uma linguagem específica de domínio (PARADIGM) para modelação da GUI.Como a abordagem é recente, é relevante submetê-la a experiências e testes sistematizados a fim de avaliar o seu bom desempenho/comportamento e compará-la com outras técnicas. Assim, este trabalho de dissertação baseia-se na avaliação e comparação da abordagem PBGT em relação a outras ferramentas e técnicas, no que diz respeito à detecção de falhas, facilidade de utilização, e aos esforços necessários para testar a aplicação.Para a realização de experiências, foram introduzidas mutações em três aplicações web diferentes - iAddressBook , TaskFreak e Tudu - de modo a abranger um maior número de casos de uso, e cada mutante foi, por sua vez, testado por cada uma das ferramentas selecionadas ou desenvolvidas e que implementam as diferentes abordagens de teste consideradas.Most of the modern software applications feature a Graphical User Interface (GUI), which turns the application easier to use, promoting higher productivity and better accessibility, and offering flexibility in how users perform tasks. However, due to GUI's complexity, the GUI testing process can be a time-consuming and intensive process. Therefore, automate the process as much as possible is indispensable to test any more evolved graphic user interface.There are some common automated GUI testing approaches, but while most of them require substantial manual efforts, others lack reusability or are only able to find specific types of errors. Many researchers state that a variety of techniques should be used.A new model-based testing approach, called Pattern- Based GUI Testing, was implemented in order to increase systematization, reusability and diminish the effort in modelling and testing. It is based on the concept of User Interface Test Patterns (UITP), which contain generic test strategies for testing common recurrent behavior (UI Patterns) on GUIs, and supported by the PBGT Tool which provides an integrated modeling and testing environment that supports the crafting of test models based on UI Test Patterns, using a GUI modeling DSL (PARADIGM).As a novel proposal, it is entirely relevant to submit it to systematized experiments and tests in order to assess its good performance/behavior and compare it with other techniques. Thus, this dissertation work mainly addresses PBGT's approach, aiming to compare it with different testing approaches/tools in what concerns to errors/fault detection, ease of use, and overall efforts required to test the application.To perform the experiments, mutations were introduced in each of three different web applications - iAddressBook, TaskFreak and Tudu - to cover a greater number of use cases, and each mutant was tested by each of the selected or developed testing tools which implement the considered approaches. Those approaches' benefits and problems are then conveniently described