Systematic automation of scenario-based testing of user interfaces

Ensuring the effectiveness factor of usability consists in ensuring that the application allows users to reach their goals and perform their tasks. One of the few means for reaching this goal relies on task analysis and proving the compatibility between the interactive application and its task models. Synergistic execution enables the validation of a system against its task model by co-executing the system and the task model and comparing the behavior of the system against what is prescribed in the model. This allows a tester to explore scenarios in order to detect deviations between the two behaviors. Manual exploration of scenarios does not guarantee a good coverage of the analysis. To address this, we resort to model-based testing (MBT) techniques to automatically generate scenarios for automated synergistic execution. To achieve this, we generate, from the task model, scenarios to be co-executed over the task model and the system. During this generation step we explore the possibility of including considerations about user error in the analysis. The automation of the execution of the scenarios closes the process. We illustrate the approach with an example.José Campos acknowledges support from project "NORTE01-0145-FEDER-000016", financed by the North Portugal Regional Operational Programme (NORTE 2020), under the PORTUGAL 2020 Partnership Agreement, and through the European Regional Development Fund (ERDF

Smells in system user interactive tests

peer reviewedTest smells are known as bad development practices that reflect poor design and implementation choices in software tests. Over the last decade, there are few attempts to study test smells in the context of system tests that interact with the System Under Test through a Graphical User Interface. To fill the gap, we conduct an exploratory analysis of test smells occurring in System User Interactive Tests (SUIT). We thus, compose a catalog of 35 SUIT-specific smells, identified through a multi-vocal literature review, and show how they differ from smells encountered in unit tests. We also conduct an empirical analysis to assess the diffuseness and removal of these smells in 48 industrial repositories and 12 open-source projects. Our results show that the same type of smells tends to appear in both industrial and open-source projects, but they are not addressed in the same way. We also find that smells originating from a combination of multiple code locations appear more often than those that are localized on a single line. This happens because of the difficulty to observe non-local smells without tool support. Furthermore, we find that smell-removing actions are not frequent with less than 50% of the affected tests ever undergoing a smell removal. Interestingly, while smell-removing actions are rare, some smells disappear while discarding tests, i.e., these smells do not appear in follow-up tests that replace the discarded ones

A Comparative Study of GUI Testing Approaches

A maioria das aplicações de software modernas apresentam uma Interface Gráfica de Utilizador (GUI), que torna a aplicação mais simples de usar, promovendo maior produtividade e melhor acessibilidade, e oferecendo flexibilidade na forma como os utilizadores podem executar tarefas. No entanto, devido à complexidade das GUIs, o processo de teste de GUI pode ser moroso e intensivo. Assim, automatizar o processo tanto quanto possível é indispensável para testar qualquer interface gráfica mais complexa e evoluída.Existem diferentes abordagens para testes de GUI automatizados, no entanto, a maioria delas requerer esforços manuais substanciais, outras apenas são capazes de encontrar erros específicos ou não permitem a reutilização de casos de teste após alterações de sistema ou GUI. Muitos investigadores afirmam que devem ser utilizadas diferentes técnicas/abordagens para um bom processo de teste.Uma nova abordagem baseada em modelos, denominada de Pattern-Based GUI Testing, foi implementada a fim de aumentar a sistematização, reutilização e diminuir o esforço da modelação e teste de GUIs. Baseia-se no conceito de Padrões de Teste de Interface de Utilizador (UITP), que contêm estratégias de teste genéricas para testar características recorrentes e comuns (UI Patterns) em GUIs. É apoiada pela ferramenta PBGT, que integra um ambiente de modelação e execução de testes de modo a suportar a criação de modelos de teste com base em UITPs, com recurso a uma linguagem específica de domínio (PARADIGM) para modelação da GUI.Como a abordagem é recente, é relevante submetê-la a experiências e testes sistematizados a fim de avaliar o seu bom desempenho/comportamento e compará-la com outras técnicas. Assim, este trabalho de dissertação baseia-se na avaliação e comparação da abordagem PBGT em relação a outras ferramentas e técnicas, no que diz respeito à detecção de falhas, facilidade de utilização, e aos esforços necessários para testar a aplicação.Para a realização de experiências, foram introduzidas mutações em três aplicações web diferentes - iAddressBook , TaskFreak e Tudu - de modo a abranger um maior número de casos de uso, e cada mutante foi, por sua vez, testado por cada uma das ferramentas selecionadas ou desenvolvidas e que implementam as diferentes abordagens de teste consideradas.Most of the modern software applications feature a Graphical User Interface (GUI), which turns the application easier to use, promoting higher productivity and better accessibility, and offering flexibility in how users perform tasks. However, due to GUI's complexity, the GUI testing process can be a time-consuming and intensive process. Therefore, automate the process as much as possible is indispensable to test any more evolved graphic user interface.There are some common automated GUI testing approaches, but while most of them require substantial manual efforts, others lack reusability or are only able to find specific types of errors. Many researchers state that a variety of techniques should be used.A new model-based testing approach, called Pattern- Based GUI Testing, was implemented in order to increase systematization, reusability and diminish the effort in modelling and testing. It is based on the concept of User Interface Test Patterns (UITP), which contain generic test strategies for testing common recurrent behavior (UI Patterns) on GUIs, and supported by the PBGT Tool which provides an integrated modeling and testing environment that supports the crafting of test models based on UI Test Patterns, using a GUI modeling DSL (PARADIGM).As a novel proposal, it is entirely relevant to submit it to systematized experiments and tests in order to assess its good performance/behavior and compare it with other techniques. Thus, this dissertation work mainly addresses PBGT's approach, aiming to compare it with different testing approaches/tools in what concerns to errors/fault detection, ease of use, and overall efforts required to test the application.To perform the experiments, mutations were introduced in each of three different web applications - iAddressBook, TaskFreak and Tudu - to cover a greater number of use cases, and each mutant was tested by each of the selected or developed testing tools which implement the considered approaches. Those approaches' benefits and problems are then conveniently described

Model-based testing of user interfaces

Dissertação de mestrado em Computer ScienceThe Graphical User Interface (GUI) is crucial for the success of an interactive system. Incorrect operation of the GUI may inhibit the proper functioning of the system. Ensuring the quality of a system is essential to its success. A practical way to ensure it is through software testing. Model- Based Testing (MBT) is a black-box technique that compares the behaviour of the system under test against the behaviour of an oracle (the system's model). These tests are intended to verify that the implementation of a software meets its speci cation. Applying MBT tools to GUIs enables the systematic testing of the system by automatically simulating user actions on the user interface. However, applying the MBT process to GUIs creates several challenges such as the mapping between the actions in the model and the actions in the application interface. This dissertation will focus on the model-based testing of graphical user interfaces. The main objective is to further the development of the TOM Framework. The TOM Framework supports the process of MBT applied to GUIs, in particular, web-based GUIs, enabling the creation and execution of user interfaces tests and thus increasing the probability of error detection in the tested interfaces.A interface gráfica do utilizador é imprescindível para o sucesso de um sistema interativo. O incorreto funcionamento da interface gráfica pode inibir o bom funcionamento da aplicação e, consequentemente, do sistema de software. Assegurar a qualidade de um sistema é essencial para o seu sucesso. Uma maneira prática de o garantir é através dos testes de software. Os testes baseados em modelos são uma técnica de caixa-preta que compara o comportamento do sistema sob teste com o comportamento de um oráculo (o modelo do sistema). Estes testes destinam-se a verificar se a implementação do software cumpre as suas especificações. A aplicação de ferramentas que suportem testes baseados em modelos em interfaces gráficas do utilizador permite o teste sistemático do sistema, simulando automaticamente ações realizadas pelo utilizador na interface disponibilizada. No entanto, a aplicação do processo dos testes baseados em modelos em interfaces gráficas cria vários desafios, como por exemplo, o mapeamento entre as ações no modelo e as ações na interface da aplicação. Esta dissertação incidirá sobre os testes baseados em modelos e a sua aplicação a interfaces gráficas do utilizador. O principal objectivo é continuar o desenvolvimento da Framework TOM. A Framework TOM suporta a aplicação do processo de testes baseados em modelos a interfaces gráficas do utilizador, permitindo a criação e execução de testes em interfaces e aumentando a probabilidade de detecção de erros nas interfaces testadas.This dissertation was supported by Project ”NORTE-01-0145-FEDER-000016”, financed by the North Portugal Regional Operational Programme (NORTE 2020), under the PORTUGAL 2020 Partnership Agreement, and through the European Regional Development Fund (ERDF

On Model-Based Testing Advanced GUIs

International audienceGraphical User Interface (GUI) design is currently shifting from designing GUIs composed of standard widgets to designing GUIs relying on more natural interactions and ad hoc widgets. This shift is meant to support the advent of GUIs providing users with more adapted and natural interactions, and the support of new input devices such as multi-touch screens. Standard widgets (e.g. buttons) are more and more replaced by ad hoc ones (e.g. the drawing area of graphical editors), and interactions are shifting from mono-event (e.g. button pressures) to multi-event interactions (e.g. multi-touch and gesture-based interactions). As a consequence, the current GUI model-based testing approaches, which target event-based systems, show their limits when applied to test such new advanced GUIs. The work introduced in this paper establishes three contributions: a precise analysis of the reasons of these current limits; a proposition to tackle the identified limits by leveraging the Malai GUI specification language and by proposing the concept of interaction-action-flow graph; feedback from two use cases, an industrial project and an open-source application, where the proposed approach has been applied