Systematic automation of scenario-based testing of user interfaces

Ensuring the effectiveness factor of usability consists in ensuring that the application allows users to reach their goals and perform their tasks. One of the few means for reaching this goal relies on task analysis and proving the compatibility between the interactive application and its task models. Synergistic execution enables the validation of a system against its task model by co-executing the system and the task model and comparing the behavior of the system against what is prescribed in the model. This allows a tester to explore scenarios in order to detect deviations between the two behaviors. Manual exploration of scenarios does not guarantee a good coverage of the analysis. To address this, we resort to model-based testing (MBT) techniques to automatically generate scenarios for automated synergistic execution. To achieve this, we generate, from the task model, scenarios to be co-executed over the task model and the system. During this generation step we explore the possibility of including considerations about user error in the analysis. The automation of the execution of the scenarios closes the process. We illustrate the approach with an example.José Campos acknowledges support from project "NORTE01-0145-FEDER-000016", financed by the North Portugal Regional Operational Programme (NORTE 2020), under the PORTUGAL 2020 Partnership Agreement, and through the European Regional Development Fund (ERDF

Model-based testing of user interfaces

Dissertação de mestrado em Computer ScienceThe Graphical User Interface (GUI) is crucial for the success of an interactive system. Incorrect operation of the GUI may inhibit the proper functioning of the system. Ensuring the quality of a system is essential to its success. A practical way to ensure it is through software testing. Model- Based Testing (MBT) is a black-box technique that compares the behaviour of the system under test against the behaviour of an oracle (the system's model). These tests are intended to verify that the implementation of a software meets its speci cation. Applying MBT tools to GUIs enables the systematic testing of the system by automatically simulating user actions on the user interface. However, applying the MBT process to GUIs creates several challenges such as the mapping between the actions in the model and the actions in the application interface. This dissertation will focus on the model-based testing of graphical user interfaces. The main objective is to further the development of the TOM Framework. The TOM Framework supports the process of MBT applied to GUIs, in particular, web-based GUIs, enabling the creation and execution of user interfaces tests and thus increasing the probability of error detection in the tested interfaces.A interface gráfica do utilizador é imprescindível para o sucesso de um sistema interativo. O incorreto funcionamento da interface gráfica pode inibir o bom funcionamento da aplicação e, consequentemente, do sistema de software. Assegurar a qualidade de um sistema é essencial para o seu sucesso. Uma maneira prática de o garantir é através dos testes de software. Os testes baseados em modelos são uma técnica de caixa-preta que compara o comportamento do sistema sob teste com o comportamento de um oráculo (o modelo do sistema). Estes testes destinam-se a verificar se a implementação do software cumpre as suas especificações. A aplicação de ferramentas que suportem testes baseados em modelos em interfaces gráficas do utilizador permite o teste sistemático do sistema, simulando automaticamente ações realizadas pelo utilizador na interface disponibilizada. No entanto, a aplicação do processo dos testes baseados em modelos em interfaces gráficas cria vários desafios, como por exemplo, o mapeamento entre as ações no modelo e as ações na interface da aplicação. Esta dissertação incidirá sobre os testes baseados em modelos e a sua aplicação a interfaces gráficas do utilizador. O principal objectivo é continuar o desenvolvimento da Framework TOM. A Framework TOM suporta a aplicação do processo de testes baseados em modelos a interfaces gráficas do utilizador, permitindo a criação e execução de testes em interfaces e aumentando a probabilidade de detecção de erros nas interfaces testadas.This dissertation was supported by Project ”NORTE-01-0145-FEDER-000016”, financed by the North Portugal Regional Operational Programme (NORTE 2020), under the PORTUGAL 2020 Partnership Agreement, and through the European Regional Development Fund (ERDF

UI-driven test-first development of interactive systems

Test-driven development (TDD) is a software development approach, which has grown out of the Extreme Programming and Agile movements, whereby tests are written prior to the implementation code which is then developed and refactored so that it passes the tests. Test-first development (TFD) takes a similar approach, but rather than relying on the testers to infer the correct tests from the requirements (often expressed via use cases) they use models of the requirements as the basis for the tests (and as such have a more formal approach). One of the problems with both TDD and TFD is that is has proven hard to adapt it for interactive systems as it is not always clear how to develop tests to also support user interfaces (UIs). In this paper we propose a method which uses both formal models of informal UI design artefacts and formal specifications to derive abstract tests which then form the basis of a test-first development process

A behavior-driven approach for specifying and testing user requirements in interactive systems

Dans un processus de conception centré sur l'utilisateur, les artefacts évoluent par cycles itératifs jusqu'à ce qu'ils répondent aux exigences des utilisateurs et deviennent ensuite le produit final. Chaque cycle donne l'occasion de réviser la conception et d'introduire de nouvelles exigences qui pourraient affecter les artefacts qui ont été définis dans les phases de développement précédentes. Garder la cohérence des exigences dans tels artefacts tout au long du processus de développement est une activité lourde et longue, surtout si elle est faite manuellement. Actuellement, certains cadres d'applications implémentent le BDD (Développement dirigé par le comportement) et les récits utilisateur comme un moyen d'automatiser le test des systèmes interactifs en construction. Les tests automatisés permettent de simuler les actions de l'utilisateur sur l'interface et, par conséquent, de vérifier si le système se comporte correctement et conformément aux exigences de l'utilisateur. Cependant, les outils actuels supportant BDD requièrent que les tests soient écrits en utilisant des événements de bas niveau et des composants qui n'existent que lorsque le système est déjà implémenté. En conséquence d'un tel bas niveau d'abstraction, les tests BDD peuvent difficilement être réutilisés avec des artefacts plus abstraits. Afin d'éviter que les tests doivent être écrits sur chaque type d'artefact, nous avons étudié l'utilisation des ontologies pour spécifier à la fois les exigences et les tests, puis exécuter des tests dans tous les artefacts partageant les concepts ontologiques. L'ontologie fondée sur le comportement que nous proposons ici vise alors à élever le niveau d'abstraction tout en supportant l'automatisation de tests dans des multiples artefacts. Cette thèse présente tel ontologie et une approche fondée sur BDD et les récits utilisateur pour soutenir la spécification et l'évaluation automatisée des exigences des utilisateurs dans les artefacts logiciels tout au long du processus de développement des systèmes interactifs. Deux études de cas sont également présentées pour valider notre approche. La première étude de cas évalue la compréhensibilité des spécifications des récits utilisateur par une équipe de propriétaires de produit (POs) du département en charge des voyages d'affaires dans notre institut. À l'aide de cette première étude de cas, nous avons conçu une deuxième étude pour démontrer comment les récits utilisateur rédigés à l'aide de notre ontologie peuvent être utilisées pour évaluer les exigences fonctionnelles exprimées dans des différents artefacts, tels que les modèles de tâche, les prototypes d'interface utilisateur et les interfaces utilisateur à part entière. Les résultats ont montré que notre approche est capable d'identifier même des incohérences à grain fin dans les artefacts mentionnés, permettant d'établir une compatibilité fiable entre les différents artefacts de conception de l'interface utilisateur.In a user-centered design process, artifacts evolve in iterative cycles until they meet user requirements and then become the final product. Every cycle gives the opportunity to revise the design and to introduce new requirements which might affect the artifacts that have been set in former development phases. Keeping the consistency of requirements in such artifacts along the development process is a cumbersome and time-consuming activity, especially if it is done manually. Nowadays, some software development frameworks implement Behavior-Driven Development (BDD) and User Stories as a means of automating the test of interactive systems under construction. Automated testing helps to simulate user's actions on the user interface and therefore check if the system behaves properly and in accordance with the user requirements. However, current tools supporting BDD requires that tests should be written using low-level events and components that only exist when the system is already implemented. As a consequence of such low-level of abstraction, BDD tests can hardly be reused with more abstract artifacts. In order to prevent that tests should be written to every type of artifact, we have investigated the use of ontologies for specifying both requirements and tests once, and then run tests on all artifacts sharing the ontological concepts. The resultant behavior-based ontology we propose herein is therefore aimed at raising the abstraction level while supporting test automation on multiple artifacts. This thesis presents this ontology and an approach based on BDD and User Stories to support the specification and the automated assessment of user requirements on software artifacts along the development process of interactive systems. Two case studies are also presented to validate our approach. The first case study evaluates the understandability of User Stories specifications by a team of Product Owners (POs) from the department in charge of business trips in our institute. With the help of this first case study, we designed a second one to demonstrate how User Stories written using our ontology can be used to assess functional requirements expressed in different artifacts, such as task models, user interface (UI) prototypes, and full-fledged UIs. The results have shown that our approach is able to identify even fine-grained inconsistencies in the mentioned artifacts, allowing establishing a reliable compatibility among different user interface design artifacts

INTERACT 2015 Adjunct Proceedings. 15th IFIP TC.13 International Conference on Human-Computer Interaction 14-18 September 2015, Bamberg, Germany

INTERACT is among the world’s top conferences in Human-Computer Interaction. Starting with the first INTERACT conference in 1990, this conference series has been organised under the aegis of the Technical Committee 13 on Human-Computer Interaction of the UNESCO International Federation for Information Processing (IFIP). This committee aims at developing the science and technology of the interaction between humans and computing devices. The 15th IFIP TC.13 International Conference on Human-Computer Interaction - INTERACT 2015 took place from 14 to 18 September 2015 in Bamberg, Germany. The theme of INTERACT 2015 was "Connection.Tradition.Innovation". This volume presents the Adjunct Proceedings - it contains the position papers for the students of the Doctoral Consortium as well as the position papers of the participants of the various workshops