Android Testing

Nos dias de hoje, as aplicações móveis são essenciais no dia-a-dia das pessoas, especialmente no que toca às aplicações móveis para plataformas Android. Isto deve-se ao facto deste sistema deter uma larga maioria do mercado de aplicações móveis que são disponibilizadas através da Google Play Store e ao largo número de dispositivos que utilizem o Android como sistema operativo. Tendo isto em vista, bem como a influência que algumas destas aplicações têm tido na vida das pessoas, torna-se cada vez mais evidente a importância de garantir a qualidade das aplicações disponibilizadas aos utilizadores, nomeadamente na plataforma oficial, a Google Play Store. Uma forma de aumentar a qualidade destas aplicações é através do teste. Contudo, muitas vezes as empresas, por falta de tempo e recursos, não dão a devida atenção a esta componente do desenvolvimento e manutenção dos seus produtos/serviços. Este fator levou ao surgimento de diversas ferramentas que automatizam e facilitam o teste de aplicações. No entanto, as abordagens existentes ainda não são satisfatórias. Neste sentido, este projeto pretende continuar o desenvolvimento de uma nova abordagem iniciada num trabalho anterior (a ferramenta de teste de aplicações móveis baseada em padrões - iMPAcT tool), que testa se as boas práticas de programação em Android são cumpridas por parte dos programadores e empresas fornecedoras. A ferramenta iMPAct tool testa as aplicações Android num processo iterativo que combina reverse engineering, pattern matching e teste. O objetivo é testar comportamento recorrente que está definido num catálogo (UI patterns). Para cada comportamento existe uma estratégia de teste associada (UI Test Pattern) que verifica se o comportamento foi ou não bem implementado. O objetivo deste trabalho de investigação é estender a iMPAcT com mais comportamento a testar adicionando novas estratégias de teste (UI Test Patterns). Em suma, o desenvolvimento e melhoria da iMPAcT Tool permitirá aos programadores de Android e entidades envolvidas no processo de desenvolvimento destas aplicações aumentar a qualidade dos seus produtos e serviços, e melhorar a qualidade das aplicações que nos chegam todos os dias através do smartphone ou tablet e que influenciam cada vez mais o nosso dia-a-dia.Nowadays, mobile applications are essential in people's daily lives, especially in mobile applications for Android platforms. This is due to the fact that this system holds a large majority of the mobile applications market that are available through the Google Play Store and the large number of devices that use Android as an operating system. With this in mind, as well as the influence that some of these applications have had on people's lives, it becomes increasingly clear how important it is to ensure the quality of the applications available to users, including the official platform, the Google Play Store. One way to increase the quality of these applications is through testing. However, often companies, due to lack of time and resources, do not give due attention to this component of the development and maintenance of their products / services. This factor led to the emergence of several tools that automate and facilitate the testing of applications. However, the existing approaches are still not satisfactory. In this sense, this project intends to continue the development of a new approach initiated in a previous work (the pattern-based mobile application testing tool - iMPAcT tool), which tests whether good programming practices in Android are fulfilled by programmers and suppliers. The iMPAct tool tests Android applications in an iterative process that combines reverse engineering, pattern matching and testing. The purpose is to test recurring behavior that is defined in a catalog (UI patterns). For each behavior there is an associated test strategy (UI Test Pattern) that verifies whether or not the behavior was well implemented. The objective of this research is to extend iMPAcT with more behavior to be tested by adding new test strategies (UI Test Patterns). In short, the development and improvement of iMPAcT Tool will allow Android developers and entities involved in the development process of these applications to increase the quality of their products and services and improve the quality of the applications that arrive to us every day through the smartphone or tablet and which increasingly influence our daily lives

Overcoming Language Dichotomies: Toward Effective Program Comprehension for Mobile App Development

Mobile devices and platforms have become an established target for modern software developers due to performant hardware and a large and growing user base numbering in the billions. Despite their popularity, the software development process for mobile apps comes with a set of unique, domain-specific challenges rooted in program comprehension. Many of these challenges stem from developer difficulties in reasoning about different representations of a program, a phenomenon we define as a "language dichotomy". In this paper, we reflect upon the various language dichotomies that contribute to open problems in program comprehension and development for mobile apps. Furthermore, to help guide the research community towards effective solutions for these problems, we provide a roadmap of directions for future work.Comment: Invited Keynote Paper for the 26th IEEE/ACM International Conference on Program Comprehension (ICPC'18

Automated Test Input Generation for Android: Are We There Yet?

Mobile applications, often simply called "apps", are increasingly widespread, and we use them daily to perform a number of activities. Like all software, apps must be adequately tested to gain confidence that they behave correctly. Therefore, in recent years, researchers and practitioners alike have begun to investigate ways to automate apps testing. In particular, because of Android's open source nature and its large share of the market, a great deal of research has been performed on input generation techniques for apps that run on the Android operating systems. At this point in time, there are in fact a number of such techniques in the literature, which differ in the way they generate inputs, the strategy they use to explore the behavior of the app under test, and the specific heuristics they use. To better understand the strengths and weaknesses of these existing approaches, and get general insight on ways they could be made more effective, in this paper we perform a thorough comparison of the main existing test input generation tools for Android. In our comparison, we evaluate the effectiveness of these tools, and their corresponding techniques, according to four metrics: code coverage, ability to detect faults, ability to work on multiple platforms, and ease of use. Our results provide a clear picture of the state of the art in input generation for Android apps and identify future research directions that, if suitably investigated, could lead to more effective and efficient testing tools for Android

Translating Video Recordings of Mobile App Usages into Replayable Scenarios

Screen recordings of mobile applications are easy to obtain and capture a wealth of information pertinent to software developers (e.g., bugs or feature requests), making them a popular mechanism for crowdsourced app feedback. Thus, these videos are becoming a common artifact that developers must manage. In light of unique mobile development constraints, including swift release cycles and rapidly evolving platforms, automated techniques for analyzing all types of rich software artifacts provide benefit to mobile developers. Unfortunately, automatically analyzing screen recordings presents serious challenges, due to their graphical nature, compared to other types of (textual) artifacts. To address these challenges, this paper introduces V2S, a lightweight, automated approach for translating video recordings of Android app usages into replayable scenarios. V2S is based primarily on computer vision techniques and adapts recent solutions for object detection and image classification to detect and classify user actions captured in a video, and convert these into a replayable test scenario. We performed an extensive evaluation of V2S involving 175 videos depicting 3,534 GUI-based actions collected from users exercising features and reproducing bugs from over 80 popular Android apps. Our results illustrate that V2S can accurately replay scenarios from screen recordings, and is capable of reproducing $\approx$ 89% of our collected videos with minimal overhead. A case study with three industrial partners illustrates the potential usefulness of V2S from the viewpoint of developers.Comment: In proceedings of the 42nd International Conference on Software Engineering (ICSE'20), 13 page

ChimpCheck: Property-Based Randomized Test Generation for Interactive Apps

We consider the problem of generating relevant execution traces to test rich interactive applications. Rich interactive applications, such as apps on mobile platforms, are complex stateful and often distributed systems where sufficiently exercising the app with user-interaction (UI) event sequences to expose defects is both hard and time-consuming. In particular, there is a fundamental tension between brute-force random UI exercising tools, which are fully-automated but offer low relevance, and UI test scripts, which are manual but offer high relevance. In this paper, we consider a middle way---enabling a seamless fusion of scripted and randomized UI testing. This fusion is prototyped in a testing tool called ChimpCheck for programming, generating, and executing property-based randomized test cases for Android apps. Our approach realizes this fusion by offering a high-level, embedded domain-specific language for defining custom generators of simulated user-interaction event sequences. What follows is a combinator library built on industrial strength frameworks for property-based testing (ScalaCheck) and Android testing (Android JUnit and Espresso) to implement property-based randomized testing for Android development. Driven by real, reported issues in open source Android apps, we show, through case studies, how ChimpCheck enables expressing effective testing patterns in a compact manner.Comment: 20 pages, 21 figures, Symposium on New ideas, New Paradigms, and Reflections on Programming and Software (Onward!2017

Pattern based GUI testing for mobile applications

This paper presents a study aiming to assess the feasibility of using the Pattern Based GUI Testing approach, PBGT, to test mobile applications. PBGT is a new model based testing approach that aims to increase systematization, reusability and diminish the effort in modelling and testing. It is based on the concept of User Interface Test Patterns (UITP) that contain generic test strategies for testing common recurrent behaviour, the so-called UI Patterns, on GUIs through its possible different implementations after a configuration step. Although PBGT was developed having web applications in mind, it is possible to develop drivers for other platforms in order to test a wide set of applications. However, web and mobile applications are different and only the development of a new driver to execute test cases over mobile applications may not be enough. This paper describes a study aiming to identify the adaptations and updates the PBGT should undergo in order to test mobile applications. © 2014 IEEE