2,894 research outputs found
ChimpCheck: Property-Based Randomized Test Generation for Interactive Apps
We consider the problem of generating relevant execution traces to test rich
interactive applications. Rich interactive applications, such as apps on mobile
platforms, are complex stateful and often distributed systems where
sufficiently exercising the app with user-interaction (UI) event sequences to
expose defects is both hard and time-consuming. In particular, there is a
fundamental tension between brute-force random UI exercising tools, which are
fully-automated but offer low relevance, and UI test scripts, which are manual
but offer high relevance. In this paper, we consider a middle way---enabling a
seamless fusion of scripted and randomized UI testing. This fusion is
prototyped in a testing tool called ChimpCheck for programming, generating, and
executing property-based randomized test cases for Android apps. Our approach
realizes this fusion by offering a high-level, embedded domain-specific
language for defining custom generators of simulated user-interaction event
sequences. What follows is a combinator library built on industrial strength
frameworks for property-based testing (ScalaCheck) and Android testing (Android
JUnit and Espresso) to implement property-based randomized testing for Android
development. Driven by real, reported issues in open source Android apps, we
show, through case studies, how ChimpCheck enables expressing effective testing
patterns in a compact manner.Comment: 20 pages, 21 figures, Symposium on New ideas, New Paradigms, and
Reflections on Programming and Software (Onward!2017
LLM for Test Script Generation and Migration: Challenges, Capabilities, and Opportunities
This paper investigates the application of large language models (LLM) in the
domain of mobile application test script generation. Test script generation is
a vital component of software testing, enabling efficient and reliable
automation of repetitive test tasks. However, existing generation approaches
often encounter limitations, such as difficulties in accurately capturing and
reproducing test scripts across diverse devices, platforms, and applications.
These challenges arise due to differences in screen sizes, input modalities,
platform behaviors, API inconsistencies, and application architectures.
Overcoming these limitations is crucial for achieving robust and comprehensive
test automation.
By leveraging the capabilities of LLMs, we aim to address these challenges
and explore its potential as a versatile tool for test automation. We
investigate how well LLMs can adapt to diverse devices and systems while
accurately capturing and generating test scripts. Additionally, we evaluate its
cross-platform generation capabilities by assessing its ability to handle
operating system variations and platform-specific behaviors. Furthermore, we
explore the application of LLMs in cross-app migration, where it generates test
scripts across different applications and software environments based on
existing scripts.
Throughout the investigation, we analyze its adaptability to various user
interfaces, app architectures, and interaction patterns, ensuring accurate
script generation and compatibility. The findings of this research contribute
to the understanding of LLMs' capabilities in test automation. Ultimately, this
research aims to enhance software testing practices, empowering app developers
to achieve higher levels of software quality and development efficiency.Comment: Accepted by the 23rd IEEE International Conference on Software
Quality, Reliability, and Security (QRS 2023
Translating Video Recordings of Mobile App Usages into Replayable Scenarios
Screen recordings of mobile applications are easy to obtain and capture a
wealth of information pertinent to software developers (e.g., bugs or feature
requests), making them a popular mechanism for crowdsourced app feedback. Thus,
these videos are becoming a common artifact that developers must manage. In
light of unique mobile development constraints, including swift release cycles
and rapidly evolving platforms, automated techniques for analyzing all types of
rich software artifacts provide benefit to mobile developers. Unfortunately,
automatically analyzing screen recordings presents serious challenges, due to
their graphical nature, compared to other types of (textual) artifacts. To
address these challenges, this paper introduces V2S, a lightweight, automated
approach for translating video recordings of Android app usages into replayable
scenarios. V2S is based primarily on computer vision techniques and adapts
recent solutions for object detection and image classification to detect and
classify user actions captured in a video, and convert these into a replayable
test scenario. We performed an extensive evaluation of V2S involving 175 videos
depicting 3,534 GUI-based actions collected from users exercising features and
reproducing bugs from over 80 popular Android apps. Our results illustrate that
V2S can accurately replay scenarios from screen recordings, and is capable of
reproducing 89% of our collected videos with minimal overhead. A case
study with three industrial partners illustrates the potential usefulness of
V2S from the viewpoint of developers.Comment: In proceedings of the 42nd International Conference on Software
Engineering (ICSE'20), 13 page
Mobile application testing matrix and challenges
The adoption of smart phones and the usages of mobile applications are increasing rapidly. Consequently, within limited time-range, mobile Internet usages have managed to take over the desktop usages particularly since the first smart phone-touched application released by iPhone in 2007. This paper is proposed to provide solution and answer the most demandable questions related to mobile application automated and manual testing limitations. Moreover, Mobile application testing requires agility and physically testing. Agile testing is to detect bugs through automated tools, whereas the compatibility testing is more to ensure that the apps operates on mobile OS (Operation Systems) as well as on the different real devices. Moreover, we have managed to answer automated or manual questions through two mobile application case studies MES (Mobile Exam System) and MLM (Mobile Lab Mate) by creating test scripts for both case studies and our experiment results have been discussed and evaluated on whether to adopt test on real devices or on emulators? In addition to this, we have introduced new mobile application testing matrix for the testers and some enterprises to obtain knowledge fro
Continuous, Evolutionary and Large-Scale: A New Perspective for Automated Mobile App Testing
Mobile app development involves a unique set of challenges including device
fragmentation and rapidly evolving platforms, making testing a difficult task.
The design space for a comprehensive mobile testing strategy includes features,
inputs, potential contextual app states, and large combinations of devices and
underlying platforms. Therefore, automated testing is an essential activity of
the development process. However, current state of the art of automated testing
tools for mobile apps poses limitations that has driven a preference for manual
testing in practice. As of today, there is no comprehensive automated solution
for mobile testing that overcomes fundamental issues such as automated oracles,
history awareness in test cases, or automated evolution of test cases.
In this perspective paper we survey the current state of the art in terms of
the frameworks, tools, and services available to developers to aid in mobile
testing, highlighting present shortcomings. Next, we provide commentary on
current key challenges that restrict the possibility of a comprehensive,
effective, and practical automated testing solution. Finally, we offer our
vision of a comprehensive mobile app testing framework, complete with research
agenda, that is succinctly summarized along three principles: Continuous,
Evolutionary and Large-scale (CEL).Comment: 12 pages, accepted to the Proceedings of 33rd IEEE International
Conference on Software Maintenance and Evolution (ICSME'17
Towards Automated Translation between Generations of GUI-based Tests for Mobile Devices
Market demands for faster delivery and higher software quality are progressively becoming more stringent.
A key hindrance for software companies to meet such demands is how to test the software due to to the intrinsic costs of development, maintenance and evolution of testware.
Especially since testware should be defined, and aligned, with all layers of system under test (SUT), including all graphical user interface (GUI) abstraction levels. These levels can be tested with different generations of GUI-based test approaches, where 2nd generation, or Layout-based, tests leverage GUI properties and 3rd generation, or Visual, tests make use of image recognition.
The two approaches provide different benefits and drawbacks and are seldom used together because of the aforementioned costs, despite growing academic evidence of the complementary benefits.
In this work we propose the proof of concept of a novel two-step translation approach for Android GUI testing, where a translator first creates a technology independent script with actions and elements of the GUI, and then translates it to a script with the syntax chosen by the user.
The approach enables users to translate Layout-based to Visual scripts and vice versa, to gain the benefits (e.g. robustness, speed and ability to emulate the user) of both generations, whilst minimizing the drawbacks (e.g. development and maintenance costs).
We outline our approach from a technical perspective, discuss some of the key challenges with the realization of our approach, evaluate the feasibility and the advantages provided by our approach on an open-source Android application, and discuss the potential industrial impact of this work
A Quantitative Study of Java Software Buildability
Researchers, students and practitioners often encounter a situation when the
build process of a third-party software system fails. In this paper, we aim to
confirm this observation present mainly as anecdotal evidence so far. Using a
virtual environment simulating a programmer's one, we try to fully
automatically build target archives from the source code of over 7,200 open
source Java projects. We found that more than 38% of builds ended in failure.
Build log analysis reveals the largest portion of errors are
dependency-related. We also conduct an association study of factors affecting
build success
Evolution and Fragility of Mobile Automated Test Suites
L'abstract è presente nell'allegato / the abstract is in the attachmen
- …