24 research outputs found
The perceived usability of automated testing tools for mobile applications
Mobile application development is a fast-emerging area in software development.The testing of mobile applications is very significant and there are a lot of tools available for testing such applications particularly for Android
and iOS.This paper presents the most frequently used automated testing tools for mobile applications.In this study, we found that Android app developers used automated testing tools such as JUnit, MonkeyTalk, Robotium, Appium, and Robolectric.However, they often prefer to test their apps manually, whereas Windows apps developers prefer to use in-house tools such as Visual Studio and Microsoft Test Manager.Both Android and Windows apps developers face many challenges such as time constraints, compatibility issues, lack of exposure, and cumbersome tools, etc. Software testing tools are the key assets of a project that can help improve productivity and software quality.A survey method was used to assess the perceived usability of automated testing tools using forty (40) respondents as participants.The result indicates that JUnit
has the highest perceived usability.The study’s result will profit practitioners and researchers in the research and development of usable automated testing tools for mobile applications
Target Directed Event Sequence Generation for Android Applications
Testing is a commonly used approach to ensure the quality of software, of
which model-based testing is a hot topic to test GUI programs such as Android
applications (apps). Existing approaches mainly either dynamically construct a
model that only contains the GUI information, or build a model in the view of
code that may fail to describe the changes of GUI widgets during runtime.
Besides, most of these models do not support back stack that is a particular
mechanism of Android. Therefore, this paper proposes a model LATTE that is
constructed dynamically with consideration of the view information in the
widgets as well as the back stack, to describe the transition between GUI
widgets. We also propose a label set to link the elements of the LATTE model to
program snippets. The user can define a subset of the label set as a target for
the testing requirements that need to cover some specific parts of the code. To
avoid the state explosion problem during model construction, we introduce a
definition "state similarity" to balance the model accuracy and analysis cost.
Based on this model, a target directed test generation method is presented to
generate event sequences to effectively cover the target. The experiments on
several real-world apps indicate that the generated test cases based on LATTE
can reach a high coverage, and with the model we can generate the event
sequences to cover a given target with short event sequences
Scripted GUI Testing of Android Apps: A Study on Diffusion, Evolution and Fragility
Background. Evidence suggests that mobile applications are not thoroughly
tested as their desktop counterparts. In particular GUI testing is generally
limited. Like web-based applications, mobile apps suffer from GUI test
fragility, i.e. GUI test classes failing due to minor modifications in the GUI,
without the application functionalities being altered.
Aims. The objective of our study is to examine the diffusion of GUI testing
on Android, and the amount of changes required to keep test classes up to date,
and in particular the changes due to GUI test fragility. We define metrics to
characterize the modifications and evolution of test classes and test methods,
and proxies to estimate fragility-induced changes.
Method. To perform our experiments, we selected six widely used open-source
tools for scripted GUI testing of mobile applications previously described in
the literature. We have mined the repositories on GitHub that used those tools,
and computed our set of metrics.
Results. We found that none of the considered GUI testing frameworks achieved
a major diffusion among the open-source Android projects available on GitHub.
For projects with GUI tests, we found that test suites have to be modified
often, specifically 5\%-10\% of developers' modified LOCs belong to tests, and
that a relevant portion (60\% on average) of such modifications are induced by
fragility.
Conclusions. Fragility of GUI test classes constitute a relevant concern,
possibly being an obstacle for developers to adopt automated scripted GUI
tests. This first evaluation and measure of fragility of Android scripted GUI
testing can constitute a benchmark for developers, and the basis for the
definition of a taxonomy of fragility causes, and actionable guidelines to
mitigate the issue.Comment: PROMISE'17 Conference, Best Paper Awar
Event trace reduction for effective bug replay of Android apps via differential GUI state analysis
© 2019 ACM. Existing Android testing tools, such as Monkey, generate a large quantity and a wide variety of user events to expose latent GUI bugs in Android apps. However, even if a bug is found, a majority of the events thus generated are often redundant and bug-irrelevant. In addition, it is also time-consuming for developers to localize and replay the bug given a long and tedious event sequence (trace). This paper presents ECHO, an event trace reduction tool for effective bug replay by using a new differential GUI state analysis. Given a sequence of events (trace), ECHO aims at removing bug-irrelevant events by exploiting the differential behavior between the GUI states collected when their corresponding events are triggered. During dynamic testing, ECHO injects at most one lightweight inspection event after every event to collect its corresponding GUI state. A new adaptive model is proposed to selectively inject inspection events based on sliding windows to differentiate the GUI states on-the-fly in a single testing process. The experimental results show that ECHO improves the effectiveness of bug replay by removing 85.11% redundant events on average while also revealing the same bugs as those detected when full event sequences are used
実行時ログから対応するソースコード箇所を推測するモバイルアプリケーションの開発支援手法
13301甲第4405号博士(工学)金沢大学博士論文本文Full 以下に掲載予定:IPSJ Transactions on Programming 9(2) pp.1-11 2016. オンライン出版. 共著者:Yuki Ono, Kouhei Sakurai, Satoshi Yaman
Pidgin Crasher: Searching for Minimised Crashing GUI Event Sequences
We present a search based testing system that automatically explores the space of all possible GUI event interleavings. Search guides our system to novel crashing sequences using Levenshtein distance and minimises the resulting fault-revealing UI sequences in a post-processing hill climb. We report on the application of our system to the SSBSE 2014 challenge program, Pidgin. Overall, our Pidgin Crasher found 20 different events that caused 2 distinct kinds of bugs, while the event sequences that caused them were reduced by 84% on average using our minimisation post processor
How do Developers Test Android Applications?
Enabling fully automated testing of mobile applications has recently become
an important topic of study for both researchers and practitioners. A plethora
of tools and approaches have been proposed to aid mobile developers both by
augmenting manual testing practices and by automating various parts of the
testing process. However, current approaches for automated testing fall short
in convincing developers about their benefits, leading to a majority of mobile
testing being performed manually. With the goal of helping researchers and
practitioners - who design approaches supporting mobile testing - to understand
developer's needs, we analyzed survey responses from 102 open source
contributors to Android projects about their practices when performing testing.
The survey focused on questions regarding practices and preferences of
developers/testers in-the-wild for (i) designing and generating test cases,
(ii) automated testing practices, and (iii) perceptions of quality metrics such
as code coverage for determining test quality. Analyzing the information
gleaned from this survey, we compile a body of knowledge to help guide
researchers and professionals toward tailoring new automated testing approaches
to the need of a diverse set of open source developers.Comment: 11 pages, accepted to the Proceedings of the 33rd IEEE International
Conference on Software Maintenance and Evolution (ICSME'17