Pattern based GUI testing for mobile applications

This paper presents a study aiming to assess the feasibility of using the Pattern Based GUI Testing approach, PBGT, to test mobile applications. PBGT is a new model based testing approach that aims to increase systematization, reusability and diminish the effort in modelling and testing. It is based on the concept of User Interface Test Patterns (UITP) that contain generic test strategies for testing common recurrent behaviour, the so-called UI Patterns, on GUIs through its possible different implementations after a configuration step. Although PBGT was developed having web applications in mind, it is possible to develop drivers for other platforms in order to test a wide set of applications. However, web and mobile applications are different and only the development of a new driver to execute test cases over mobile applications may not be enough. This paper describes a study aiming to identify the adaptations and updates the PBGT should undergo in order to test mobile applications. © 2014 IEEE

Geração e execução de casos de teste baseados em padrões de interação

Tese de Mestrado Integrado. Engenharia Informática e Computação. Universidade do Porto. Faculdade de Engenharia. 201

Android Testing

Nos dias de hoje, as aplicações móveis são essenciais no dia-a-dia das pessoas, especialmente no que toca às aplicações móveis para plataformas Android. Isto deve-se ao facto deste sistema deter uma larga maioria do mercado de aplicações móveis que são disponibilizadas através da Google Play Store e ao largo número de dispositivos que utilizem o Android como sistema operativo. Tendo isto em vista, bem como a influência que algumas destas aplicações têm tido na vida das pessoas, torna-se cada vez mais evidente a importância de garantir a qualidade das aplicações disponibilizadas aos utilizadores, nomeadamente na plataforma oficial, a Google Play Store. Uma forma de aumentar a qualidade destas aplicações é através do teste. Contudo, muitas vezes as empresas, por falta de tempo e recursos, não dão a devida atenção a esta componente do desenvolvimento e manutenção dos seus produtos/serviços. Este fator levou ao surgimento de diversas ferramentas que automatizam e facilitam o teste de aplicações. No entanto, as abordagens existentes ainda não são satisfatórias. Neste sentido, este projeto pretende continuar o desenvolvimento de uma nova abordagem iniciada num trabalho anterior (a ferramenta de teste de aplicações móveis baseada em padrões - iMPAcT tool), que testa se as boas práticas de programação em Android são cumpridas por parte dos programadores e empresas fornecedoras. A ferramenta iMPAct tool testa as aplicações Android num processo iterativo que combina reverse engineering, pattern matching e teste. O objetivo é testar comportamento recorrente que está definido num catálogo (UI patterns). Para cada comportamento existe uma estratégia de teste associada (UI Test Pattern) que verifica se o comportamento foi ou não bem implementado. O objetivo deste trabalho de investigação é estender a iMPAcT com mais comportamento a testar adicionando novas estratégias de teste (UI Test Patterns). Em suma, o desenvolvimento e melhoria da iMPAcT Tool permitirá aos programadores de Android e entidades envolvidas no processo de desenvolvimento destas aplicações aumentar a qualidade dos seus produtos e serviços, e melhorar a qualidade das aplicações que nos chegam todos os dias através do smartphone ou tablet e que influenciam cada vez mais o nosso dia-a-dia.Nowadays, mobile applications are essential in people's daily lives, especially in mobile applications for Android platforms. This is due to the fact that this system holds a large majority of the mobile applications market that are available through the Google Play Store and the large number of devices that use Android as an operating system. With this in mind, as well as the influence that some of these applications have had on people's lives, it becomes increasingly clear how important it is to ensure the quality of the applications available to users, including the official platform, the Google Play Store. One way to increase the quality of these applications is through testing. However, often companies, due to lack of time and resources, do not give due attention to this component of the development and maintenance of their products / services. This factor led to the emergence of several tools that automate and facilitate the testing of applications. However, the existing approaches are still not satisfactory. In this sense, this project intends to continue the development of a new approach initiated in a previous work (the pattern-based mobile application testing tool - iMPAcT tool), which tests whether good programming practices in Android are fulfilled by programmers and suppliers. The iMPAct tool tests Android applications in an iterative process that combines reverse engineering, pattern matching and testing. The purpose is to test recurring behavior that is defined in a catalog (UI patterns). For each behavior there is an associated test strategy (UI Test Pattern) that verifies whether or not the behavior was well implemented. The objective of this research is to extend iMPAcT with more behavior to be tested by adding new test strategies (UI Test Patterns). In short, the development and improvement of iMPAcT Tool will allow Android developers and entities involved in the development process of these applications to increase the quality of their products and services and improve the quality of the applications that arrive to us every day through the smartphone or tablet and which increasingly influence our daily lives

Testing GUI-based Software with Undetermined Input Spaces

Most software applications feature a Graphical User Interface (GUI) front-end as the main, and often the only, method for the user to interact with the software. System-testing a software application requires it to be tested as a whole through the GUI. Testers need to generate sequences of GUI events (e.g., mouse clicks and menu selections) to exercise various behaviors of the application. Because the input space of a typical GUI (i.e., the space of all possible GUI events and their interactions) is often enormous, manual GUI testing is impractical. Model-based testing is a new approach that automatically and systematically generates a large number of test cases by leveraging a formal model representing the GUI input space. Unfortunately, modern applications often have a ``context-sensitive reachability GUI,'' in which the GUI components are only reachable with some particular state or environment constraints. Thus, it is challenging to determine the GUI input space and and obtain a GUI model for automated GUI testing. This research proposes new testing techniques to tackle the challenges in model-based GUI testing. The central thesis is this: GUI-based applications can be effectively and efficiently tested by systematically and incrementally leveraging the application runtime execution observations. To explore the thesis, a novel model-based testing paradigm called Observer-Model-Exercise* (OME*) is developed. This paradigm relies on the opportunistic observations obtained during test case execution to incrementally explore the GUI input space and construct a GUI model for test case generation. To evaluate OME*, an open-source automated model-based GUI testing framework called GUITAR is developed. An empirical study with 8 widely-used open-source applications demonstrated that the OME* approach is feasible. Compared to previous model-based testing approaches, OME* was able to increase the GUI input space discovered by as much as 1,044%. As a result, 34 new faults were detected in the subject applications

Model-based testing of automotive HMIs with consideration for product variability

The human-machine interfaces (HMIs) of today’s premium automotive infotainment systems are complex embedded systems which have special characteristics in comparison to GUIs of standard PC applications, in particular regarding their variability. The variability of infotainment system HMIs results from different car models, product series, markets, equipment configuration possibilities, system types and languages and necessitates enormous testing efforts. The model-based testing approach is a promising solution for reducing testing efforts and increasing test coverage. However, while model-based testing has been widely used for function tests of subsystems in practice, HMI tests have remained manual or only semi-automated and are very time-consuming and work-intensive. Also, it is very difficult to achieve systematic or high test coverage via manual tests. A large amount of research work has addressed GUI testing in recent years. In addition, variability is becoming an ever more popular topic in the domain of software product line development. However, a model-based testing approach for complex HMIs which also considers variability is still lacking. This thesis presents a model-based testing approach for infotainment system HMIs with the particular aim of resolving the variability problem. Furthermore, the thesis provides a foundation for future standards of HMI testing in practice. The proposed approach is based on a model-based HMI testing framework which includes two essential components: a test-oriented HMI specification and a test generation component. The test-oriented HMI specification has a layered structure and is suited to specifying data which is required for testing different features of the HMI. Both the dynamic behavior and the representation of the HMI are the testing focuses of this thesis. The test generation component automatically generates tests from the test-oriented HMI specification. Furthermore, the framework can be extended in order to automatically execute the generated tests. Generated tests must first be initialized, which means that they are enhanced with concrete user input data. Afterwards, initialized tests can be automatically executed with the help of a test execution tool which must be extended into the testing framework. In this thesis, it is proposed to specify and test different HMI-variants which have a large set of commonalities based on the software product line approach. This means the test-oriented HMI specification is extended in order to describe the commonalities and variabilities between HMI variants of an HMI product line. In particular, strategies are developed in order to generate tests for different HMI products. One special feature is that redundancies are avoided both for the test generation and the execution processes. This is especially important for the industrial practice due to limited test resources. Modeling and testing variability of automotive HMIs make up the main research contributions of this thesis. We hope that the results presented in this thesis will offer GUI testing research a solution for model-based testing of multi-variant HMIs and provide the automotive industry with a foundation for future HMI testing standards

Model Checking and Model-Based Testing : Improving Their Feasibility by Lazy Techniques, Parallelization, and Other Optimizations

This thesis focuses on the lightweight formal method of model-based testing for checking safety properties, and derives a new and more feasible approach. For liveness properties, dynamic testing is impossible, so feasibility is increased by specializing on an important class of properties, livelock freedom, and deriving a more feasible model checking algorithm for it. All mentioned improvements are substantiated by experiments

Model-based testing through a GUI

Abstract. So far, model-based testing approaches have mostly been used in testing through various kinds of APIs. In practice, however, testing through a GUI is another equally important application area, which introduces new challenges. In this paper, we introduce a new methodology for model-based GUI testing. This includes using Labeled Transition Systems (LTSs) in conjunction with action word and keyword techniques for test modeling. We have also conducted an industrial case study where we tested a mobile device and were able to find previously unreported defects. The test environment included a standard MS Windows GUI testing tool as well as components implementing our approach. Assessment of the results from an industrial point of view suggests directions for future development.