Gamified Exploratory GUI Testing of Web Applications: a Preliminary Evaluation

In the context of Software Engineering, testing is a well-known phase that plays a critical role, as is needed to ensure that the designed and produced code provides the expected results, avoiding faults and crashes. Exploratory GUI testing allows the tester to manually define test cases by directly interacting with the user interface of the finite system. However, testers often loosely perform exploratory GUI testing, as they perceive it as a time-consuming, repetitive and unappealing activity. We defined a gamified framework for GUI testing to address this issue, which we developed and integrated into the Augmented testing tool, Scout. Gamification is perceived as a means to enhance the performance of human testers by stimulating competition and encouraging them to achieve better results in terms of both efficiency and effectiveness. We performed a preliminary evaluation of the gamification layer with a small sample of testers to assess the benefits of the technique compared with the standard version of the same tool. Test sequences defined with the gamified tool achieved higher coverage (i.e., higher efficiency) and a slightly higher percentage of bugs found. The user's opinion was almost unanimously in favor of the gamified version of the tool

Poster: Kotlin Assimilating the Android Ecosystem - An Appraisal of Diffusion and Impact on Maintainability

Kotlin is a language alternative to Java, introduced in 2011. It promises to address many of Java's limitations and lead to better application maintainability. In 2017, it became a first-class language for Android development with full tool support. We mined a dataset of 2708 Android applications on which we based our study. Our empirical assessment of the diffusion of Kotlin in Android app development shows that it is now used in around 40% of projects. Kotlin adoption has a significant positive effect on code maintainability metrics and in popularity among end-users and developers. Overall, Kotlin appears to be successfully fulfilling its promise of being a better Java for Android development

A Review on Tools, Mechanics, Benefits, and Challenges of Gamified Software Testing

Gamification is an established practice in Software Engineering to increase effectiveness and engagement in many practices. This manuscript provides a characterisation of the application of gamification to the Software Testing area. Such practice in fact reportedly suffers from low engagement by both personnel in industrial contexts and learners in educational contexts. Our goal is to identify the application areas and utilised gamified techniques and mechanics, the provided benefits and drawbacks, as well as the open challenges in the field. To this purpose, we conducted a Multivocal Literature Review to identify white and grey literature sources addressing gamified software testing. We analysed 73 contributions and summarised the most common gamified mechanics, concepts, tools and domains where they are mostly applied. We conclude that gamification in software testing is mostly applied to the test creation phase with simple white-box unit or mutation testing tools, and is mostly used to foster good behaviours by promoting the testers’ accomplishment. Key research areas and main challenges in the field are: careful design of tailored gamified mechanics for specific testing techniques; the need for technological improvements to enable crowdsourcing, cooperation, and concurrency; the necessity for empirical and large-scale evaluation of the benefits delivered by gamification mechanics

A Metric Framework for the Gamification of Web and Mobile GUI Testing

System testing through the Graphical User Interface (GUI) is a valuable form of Verification & Validation for modern applications, especially in graphically-intensive domains like web and mobile applications. However, the practice is often overlooked by developers mostly because of its costly nature and the absence of immediate feedback about the quality of test sequence. This paper describes a proposal for the Gamification of exploratory GUI testing. We define - in a tool and domain- agnostic way - the basic concepts, a set of metrics, a scoring scheme and visual feedbacks to enable a gamified approach to the practice; we finally discuss the potential implications and envision a roadmap for the evaluation of the approach

Is ChatGPT Capable of Crafting Gamification Strategies for Software Engineering Tasks?

Gamification has gained significant attention in the last decade for its potential to enhance engagement and motivation in various domains. During the last year ChatGPT, a state-of-the-art large language model has received even more attention both in the field of scientific research and in common use by individuals or companies. In this study, we investigate the possibility of adopting ChatGPT as a tool for designing gamification platforms in the Software Engineering domain. Leveraging the capabilities of ChatGPT, we assess how good is it at generating effective suggestions and ideas for designers or developers. To evaluate ChatGPT's potential as a gamification platform creator we narrowed the context to one particular Software Engineering activity, asking for possible aspects of the activity to be gamified. Each proposed aspect was subsequently unraveled by ChatGPT both asking in a shared and separate context, first following the conversational nature of the model, then applying a validated design framework. The study assesses ChatGPT's ability to select and integrate game elements to build a thriving gamification environment by framing the design of the platform to a state-of-the-art conceptual framework. To evaluate the goodness of the design choices made we relied both on the Octalysis framework and on personal experience. The findings of the papers show that ChatGPT can only create simple playful experiences not very effective. Although, by instructing the model with more specific desired mechanics and dynamics, it is possible to guide it toward the application of the ideas suggested. We argue that ChatGPT is not capable of building a gamified environment on its own, but it could still be used to build the foundation of a gamification platform as long as the designers refine and rough out the advice gained from a user-centered solution

GERRY: A Gamified Browser Tool for GUI Testing

Graphical User Interface (GUI) testing is a relevant step of the soft- ware development process which is not often performed thoroughly due to its unappealing nature, to the inherent fragility of test cases, and to the fact that test cases – composed of long and complicated sequences of operations – have to be manually written by testers. We propose GERRY , a Capture & Replay GUI testing tool which implements an approach based on Gamification, i.e., the application of gaming elements to non-ludic activities. The purpose of the tool is to increase the engagement of the testers when performing GUI test case definition tasks. The tool makes use of mechanics typical of games such as progress indicators, leaderboards, and unlockable rewards, to increase user interest and involvement. GERRY also generates reports (i.e., traces of all actions and milestones reached during a session), written logs of the performed testing sessions, and scripts compatible with existing GUI testing tools (SikuliX and Selenium) for replay purposes

IntelliGame in Action: An Experience Report on Gamifying JavaScript Unit Tests

This paper investigates the integration and assessment of IntelliGame, a gamification plugin initially designed for Java development, within the realm of JavaScript unit testing. We aim to verify the generalizability of IntelliGame to JavaScript development and to provide valuable insights into the experiment's design. For this, we first customize IntelliGame for JavaScript, and then conduct a controlled experiment involving 152 participants utilizing the Jest testing framework, and finally examine its influence on testing behavior and the overall developer experience. The findings from this study provide valuable insights for improving JavaScript testing methodologies through the incorporation of gamification

An analysis of widget layout attributes to support Android GUI-based testing

In the context of GUI testing, identifying robust locators (i.e., attributes to unambiguously identify on-screen widgets to be used in test sequences) is still considered an unsolved challenge by the researchers' community. The frequent variation of attributes between different releases of the System Under Test (SUT) leads in fact to testing fragility, i.e., test case failing because of invalidated locators. Recent studies have highlighted the benefits of adopting multi-locator approach, i.e., the combination of multiple locators to enhance the robustness of widget identification. The objective of this work is to provide insights into the composition of Android applications, assessing the characteristics of different layout-based properties and their suitability to be used as locators for widgets in the context of GUI-based testing. We investigated the state of the practice by analysing the distribution of widget values within 30 real apps selected from the Google Play Store. For those apps, we selected two different versions to examine how they evolved over time from both visual and structural perspectives. The results of our analysis showed that missing values, variability, and instability of attributes make it hardly possible to identify a single attribute or technique (either coordinate-, property-, or visual-based) capable of providing robust GUI testing over multiple releases of mobile SUTs

Survival of the Tested: Gamified Unit Testing Inspired by Battle Royale

While testing takes on a fundamental role to verify software quality and correctness, it often results to be overlooked in the educational field and students often approach it unwillingly, due to its repetitiveness. Our aim is to exploit gamification to engage students by providing them with dynamics like competition, self-expression, and personal improvement. We designed and developed Unit Brawl, a gamified application meant to manage multiple rounds, each one consisting of students developing Java programs and unit tests to be executed on each other. The players collect points by writing correct code that does not make the other players’ test cases fail, or by writing test cases capable of detecting defects in the other players’ code. The results of a preliminary evaluation to assess the functionality and performance of Unit Brawl look promising. They make us confident about its stability, so we plan an evaluation with students in order to verify the effectiveness of the applied game elements in enhancing the students’ interest towards testing topics and their learning

Guidelines for GUI Testing Maintenance: A Linter for Test Smell Detection

Context: GUI Test suites suffer from high fragility, in fact modifications or redesigns of the user interface are commonly frequent and often invalidate the tests. This leads, for both DOM- and visual-based techniques, to frequent need for careful maintenance of test suites, which can be expensive and time-consuming. Objective: The goal of this work is to present a set of guidelines to write cleaner and more robust test code, reducing the cost of maintenance and producing more understandable code. Based on the provided recommendations, a static test suite analyzer and code linter has been developed. Method: An ad-hoc grey literature research was conducted on the state of the practice, by performing a semi-systematic literature review. Authors’ experience was coded into a set of recommendations, by applying the grounded theory methodology. Based on these results, we developed a linter in the form of a plugin for Visual Studio Code, implementing 17 of the provided guidelines. The plugin highlights test smells in the Java and Javascript languages. Finally, we conducted a preliminary validation of the tool against test suites from real GitHub projects. Conclusions: The preliminary evaluation, meant to be an attempt of application of the plugin to real test suites, detected three main smells, namely the usage of global variables, the lack of adoption of the Page Object design pattern, and the usage of fragile locator such as the XPath