3 research outputs found
Unblind Your Apps: Predicting Natural-Language Labels for Mobile GUI Components by Deep Learning
According to the World Health Organization(WHO), it is estimated that
approximately 1.3 billion people live with some forms of vision impairment
globally, of whom 36 million are blind. Due to their disability, engaging these
minority into the society is a challenging problem. The recent rise of smart
mobile phones provides a new solution by enabling blind users' convenient
access to the information and service for understanding the world. Users with
vision impairment can adopt the screen reader embedded in the mobile operating
systems to read the content of each screen within the app, and use gestures to
interact with the phone. However, the prerequisite of using screen readers is
that developers have to add natural-language labels to the image-based
components when they are developing the app. Unfortunately, more than 77% apps
have issues of missing labels, according to our analysis of 10,408 Android
apps. Most of these issues are caused by developers' lack of awareness and
knowledge in considering the minority. And even if developers want to add the
labels to UI components, they may not come up with concise and clear
description as most of them are of no visual issues. To overcome these
challenges, we develop a deep-learning based model, called LabelDroid, to
automatically predict the labels of image-based buttons by learning from
large-scale commercial apps in Google Play. The experimental results show that
our model can make accurate predictions and the generated labels are of higher
quality than that from real Android developers.Comment: Accepted to 42nd International Conference on Software Engineerin
Large-Scale Analysis of Framework-Specific Exceptions in Android Apps
Mobile apps have become ubiquitous. For app developers, it is a key priority
to ensure their apps' correctness and reliability. However, many apps still
suffer from occasional to frequent crashes, weakening their competitive edge.
Large-scale, deep analyses of the characteristics of real-world app crashes can
provide useful insights to guide developers, or help improve testing and
analysis tools. However, such studies do not exist -- this paper fills this
gap. Over a four-month long effort, we have collected 16,245 unique exception
traces from 2,486 open-source Android apps, and observed that
framework-specific exceptions account for the majority of these crashes. We
then extensively investigated the 8,243 framework-specific exceptions (which
took six person-months): (1) identifying their characteristics (e.g.,
manifestation locations, common fault categories), (2) evaluating their
manifestation via state-of-the-art bug detection techniques, and (3) reviewing
their fixes. Besides the insights they provide, these findings motivate and
enable follow-up research on mobile apps, such as bug detection, fault
localization and patch generation. In addition, to demonstrate the utility of
our findings, we have optimized Stoat, a dynamic testing tool, and implemented
ExLocator, an exception localization tool, for Android apps. Stoat is able to
quickly uncover three previously-unknown, confirmed/fixed crashes in Gmail and
Google+; ExLocator is capable of precisely locating the root causes of
identified exceptions in real-world apps. Our substantial dataset is made
publicly available to share with and benefit the community.Comment: ICSE'18: the 40th International Conference on Software Engineerin