Translating Video Recordings of Mobile App Usages into Replayable Scenarios

Screen recordings of mobile applications are easy to obtain and capture a wealth of information pertinent to software developers (e.g., bugs or feature requests), making them a popular mechanism for crowdsourced app feedback. Thus, these videos are becoming a common artifact that developers must manage. In light of unique mobile development constraints, including swift release cycles and rapidly evolving platforms, automated techniques for analyzing all types of rich software artifacts provide benefit to mobile developers. Unfortunately, automatically analyzing screen recordings presents serious challenges, due to their graphical nature, compared to other types of (textual) artifacts. To address these challenges, this paper introduces V2S, a lightweight, automated approach for translating video recordings of Android app usages into replayable scenarios. V2S is based primarily on computer vision techniques and adapts recent solutions for object detection and image classification to detect and classify user actions captured in a video, and convert these into a replayable test scenario. We performed an extensive evaluation of V2S involving 175 videos depicting 3,534 GUI-based actions collected from users exercising features and reproducing bugs from over 80 popular Android apps. Our results illustrate that V2S can accurately replay scenarios from screen recordings, and is capable of reproducing $\approx$ 89% of our collected videos with minimal overhead. A case study with three industrial partners illustrates the potential usefulness of V2S from the viewpoint of developers.Comment: In proceedings of the 42nd International Conference on Software Engineering (ICSE'20), 13 page

Towards Efficient Record and Replay: A Case Study in WeChat

WeChat, a widely-used messenger app boasting over 1 billion monthly active users, requires effective app quality assurance for its complex features. Record-and-replay tools are crucial in achieving this goal. Despite the extensive development of these tools, the impact of waiting time between replay events has been largely overlooked. On one hand, a long waiting time for executing replay events on fully-rendered GUIs slows down the process. On the other hand, a short waiting time can lead to events executing on partially-rendered GUIs, negatively affecting replay effectiveness. An optimal waiting time should strike a balance between effectiveness and efficiency. We introduce WeReplay, a lightweight image-based approach that dynamically adjusts inter-event time based on the GUI rendering state. Given the real-time streaming on the GUI, WeReplay employs a deep learning model to infer the rendering state and synchronize with the replaying tool, scheduling the next event when the GUI is fully rendered. Our evaluation shows that our model achieves 92.1% precision and 93.3% recall in discerning GUI rendering states in the WeChat app. Through assessing the performance in replaying 23 common WeChat usage scenarios, WeReplay successfully replays all scenarios on the same and different devices more efficiently than the state-of-the-practice baselines

Overcoming Language Dichotomies: Toward Effective Program Comprehension for Mobile App Development

Mobile devices and platforms have become an established target for modern software developers due to performant hardware and a large and growing user base numbering in the billions. Despite their popularity, the software development process for mobile apps comes with a set of unique, domain-specific challenges rooted in program comprehension. Many of these challenges stem from developer difficulties in reasoning about different representations of a program, a phenomenon we define as a "language dichotomy". In this paper, we reflect upon the various language dichotomies that contribute to open problems in program comprehension and development for mobile apps. Furthermore, to help guide the research community towards effective solutions for these problems, we provide a roadmap of directions for future work.Comment: Invited Keynote Paper for the 26th IEEE/ACM International Conference on Program Comprehension (ICPC'18

Continuous, Evolutionary and Large-Scale: A New Perspective for Automated Mobile App Testing

Mobile app development involves a unique set of challenges including device fragmentation and rapidly evolving platforms, making testing a difficult task. The design space for a comprehensive mobile testing strategy includes features, inputs, potential contextual app states, and large combinations of devices and underlying platforms. Therefore, automated testing is an essential activity of the development process. However, current state of the art of automated testing tools for mobile apps poses limitations that has driven a preference for manual testing in practice. As of today, there is no comprehensive automated solution for mobile testing that overcomes fundamental issues such as automated oracles, history awareness in test cases, or automated evolution of test cases. In this perspective paper we survey the current state of the art in terms of the frameworks, tools, and services available to developers to aid in mobile testing, highlighting present shortcomings. Next, we provide commentary on current key challenges that restrict the possibility of a comprehensive, effective, and practical automated testing solution. Finally, we offer our vision of a comprehensive mobile app testing framework, complete with research agenda, that is succinctly summarized along three principles: Continuous, Evolutionary and Large-scale (CEL).Comment: 12 pages, accepted to the Proceedings of 33rd IEEE International Conference on Software Maintenance and Evolution (ICSME'17

How do Developers Test Android Applications?

Enabling fully automated testing of mobile applications has recently become an important topic of study for both researchers and practitioners. A plethora of tools and approaches have been proposed to aid mobile developers both by augmenting manual testing practices and by automating various parts of the testing process. However, current approaches for automated testing fall short in convincing developers about their benefits, leading to a majority of mobile testing being performed manually. With the goal of helping researchers and practitioners - who design approaches supporting mobile testing - to understand developer's needs, we analyzed survey responses from 102 open source contributors to Android projects about their practices when performing testing. The survey focused on questions regarding practices and preferences of developers/testers in-the-wild for (i) designing and generating test cases, (ii) automated testing practices, and (iii) perceptions of quality metrics such as code coverage for determining test quality. Analyzing the information gleaned from this survey, we compile a body of knowledge to help guide researchers and professionals toward tailoring new automated testing approaches to the need of a diverse set of open source developers.Comment: 11 pages, accepted to the Proceedings of the 33rd IEEE International Conference on Software Maintenance and Evolution (ICSME'17

A Secure Protocol for M-commerce Secure SMS Mobile Payment

The worldwide utilization of the Internet has profoundly supplied the development of e-commerce. Technological advancement in mobile phones (e.g. Mobile phones) has likewise added to doing e-commerce by means of mobile phones (m-business). Mcommerce includes the utilization of mobile phones, for example, mobile phones in doing electronic exchanges. Applications in this area range from ordinary data utilization to high security money related electronic exchanges. Much the same as e-commerce, the security of m-business applications is basic, particularly when it includes applications that arrangement with client touchy information, In the usage, the purchaser utilizes the program to shop online of course, at the checkout he is sent to the versatile application gave and secured by the bank. The gatherings imparts through three distinctive channels (VPN burrowing, SMS informing and HTTPS) to improve the security. This convention guarantees that the purchaser basic data is never ignored the web to have the capacity to buy and in the meantime the scanning knowledge has little interference. Additionally it presents minimal overhead on the vendor and the banks to have the capacity to give such administration. To ensure the convention acts as expected, an android application was made that speaks with web servers speaking to the purchaser bank, the shipper site and vendor bank. All interchanges between the application and the purchaser bank were through VPN and SMS

Techniques for Efficient and Effective Mobile Testing

The booming mobile app market attracts a large number of developers. As a result, the competition is extremely tough. This fierce competition leads to high standards required for mobile apps, which mandates efficient and effective testing. Efficient testing requires little effort to use, while effective testing checks that the app under test behaves as expected. Manual testing is highly effective, but it is costly. Automatic testing should come to the rescue, but current automatic methods are either ineffective or inefficient. Methods using implicit specifications – for instance, “an app should not crash” for catching fail-stop errors – are ineffective because they cannot find semantic problems. Methods using explicit specifications such as test scripts are inefficient because they require huge developer effort to create and maintain specifications. In this thesis, we present our two approaches for solving these challenges. We first built the AppDoctor system which efficiently tests mobile apps. It quickly explores an app then slowly but accurately verifies the potential problems to identify bugs without introducing false positives. It uses dependencies discovered between actions to simplify its reports. Our second approach, implemented in the AppFlow system, leverages the ample opportunity of reusing test cases between apps to gain efficiency without losing effectiveness. It allows common UI elements to be used in test scripts then recognizes these UI elements in real apps using a machine learning approach. The system also allows tests to be specified in reusable pieces, and provides a system to synthesize complete test cases from these reusable pieces. It enables robust tests to be created and reused across apps in the same category. The combination of these two approaches enables a developer to quickly test an app on a great number of combinations of actions for fail-stop problems, and effortlessly and efficiently test the app on most common scenarios for semantic problems. This combination covers most of her test requirements and greatly reduces her burden in testing the app