Six reasons for rejecting an industrial survey paper

Context: Despite their importance in any empirically based research program, industrial surveys are not very common in the software engineering literature. In our experience, a possible reason is their difficulty of publication. Goal: We would like to understand what are the issues that may prevent the publication of papers reporting industrial surveys. Method: In this preliminary work, we analyzed the surveys we conducted and extracted the main lessons learned in terms of issues and problems. Results: Most common critics posed to industrial surveys are: lack of novelty, limitation of the geographic scope and sampling issues. Conclusions: Most objections that led to reject a survey paper actually are not easy to overcome and others are not so serious. These objections could restrain researchers from conducting this type of studies that represent an important methodological asset. For these reasons, we think that reviewers should be less severe to judge survey papers provided that all the limitations of the study are well explained and highlighte

Scripted GUI Testing of Android Apps: A Study on Diffusion, Evolution and Fragility

Background. Evidence suggests that mobile applications are not thoroughly tested as their desktop counterparts. In particular GUI testing is generally limited. Like web-based applications, mobile apps suffer from GUI test fragility, i.e. GUI test classes failing due to minor modifications in the GUI, without the application functionalities being altered. Aims. The objective of our study is to examine the diffusion of GUI testing on Android, and the amount of changes required to keep test classes up to date, and in particular the changes due to GUI test fragility. We define metrics to characterize the modifications and evolution of test classes and test methods, and proxies to estimate fragility-induced changes. Method. To perform our experiments, we selected six widely used open-source tools for scripted GUI testing of mobile applications previously described in the literature. We have mined the repositories on GitHub that used those tools, and computed our set of metrics. Results. We found that none of the considered GUI testing frameworks achieved a major diffusion among the open-source Android projects available on GitHub. For projects with GUI tests, we found that test suites have to be modified often, specifically 5\%-10\% of developers' modified LOCs belong to tests, and that a relevant portion (60\% on average) of such modifications are induced by fragility. Conclusions. Fragility of GUI test classes constitute a relevant concern, possibly being an obstacle for developers to adopt automated scripted GUI tests. This first evaluation and measure of fragility of Android scripted GUI testing can constitute a benchmark for developers, and the basis for the definition of a taxonomy of fragility causes, and actionable guidelines to mitigate the issue.Comment: PROMISE'17 Conference, Best Paper Awar

Quantitative Assessment of the Impact of Automatic Static Analysis Issues on Time Efficiency

Background: Automatic Static Analysis (ASA) tools analyze source code and look for code patterns (aka smells) that might cause defective behavior or might degrade other dimensions of software quality, e.g. efficiency. There are many potentially negative code patterns, and ASA tools typically report a huge list of them even in small programs. Moreover, so far, little evidence is available about the negative impact on performance of code patterns identified by such tools. A consequence is that programmers cannot appreciate the benefits of ASA tools and tend not to include them in their workflow. Aims: Quantitatively assess the impact of issues signaled by ASA tools on time efficiency. Method: We select 20 issues and for each of them we set up two source code fragments: one containing the issue and the corresponding refactored version, functionally identical but without the issue. We set up three different platforms, isolated from network and other user programs, then we execute the code fragments, and measure the execution time of both code versions. Results: We find that eleven issues have an actual negative impact on performance. We also compute for each issue an estimation for the delay provoked by a single execution. Conclusions: We produce a set of issues with a verified negative impact on performance. They can be checked easily with an analysis tool and code can be refactored to obtain a provably more efficient code. We also provide the estimated delay cost of each issue in the environments where we conduct the tests. These results can be improved with the help of other researchers: repeating the tests in several platforms would make it possible to build up a wider benchmar

Classification of Language Interactions

Context: the presence of several languages interacting each other within the same project is an almost universal feature in software development. Earlier work shows that this interaction might be source of problems. Goal: we aim at identifying and characterizing the cross-language interactions at semantic level.% among artifacts written in different languages. Method: we took the commits of an open source project and analyzed the cross-language pairs of files occurring in the same commit to identify possible semantic interactions. We both defined a taxonomy and applied it. Result: we identify 6 categories of semantic interactions. The most common category is the one based on shared ids, the next is when an artifact provides a description of another artifact. Conclusions: the deeper knowledge of cross-language interactions represents the basis for implementing a tool supporting the management of this kind of interactions and the detection of related problems at compile time

Profiling Power Consumption on Mobile Devices

The proliferation of mobile devices, and the migration of the information access paradigm to mobile platforms, motivate studies of power consumption behaviors with the purpose of increasing the device battery life. The aim of this work is to profile the power consumption of a Samsung Galaxy I7500 and a Samsung Nexus S, in order to understand how such feature has evolved over the years. We performed two experiments: the first one measures consumption for a set of usage scenarios, which represent common daily user activities, while the second one analyzes a context-aware application with a known source code. The first experiment shows that the most recent device in terms of OS and hardware components shows significantly lower consumption than the least recent one. The second experiment shows that the impact of different configurations of the same application causes a different power consumption behavior on both smartphones. Our results show that hardware improvements and energy-aware software applications greatly impact the energy efficiency of mobile device

Language Interaction and Quality Issues: An Exploratory Study

Most software systems are complex and often composed of a large number of artifacts. To realize each different artifacts specific techniques are used resorting on different abstractions, languages and tools. Successful composition of different elements requires coherence among them. Unfortunately constraints between artifacts written in different languages are usually not formally expressed neither checked by supporting tools; as consequence they can be a source of problems. In this paper we explore the role of the relations between artifacts written in different languages by means of a case study on the Hadoop open source project. We present the problem introducing the related terminology, we quantify the phenomenon and investigate the relation with defect pronenes

Integrating Software Engineering Key Practices into an OOP Massive In-Classroom Course: an Experience Report

Programming and software engineering courses in computer science curricula typically focus on both providing theoretical knowledge of programming languages and best-practices, and developing practical development skills. In a massive course --- several hundred students --- the teachers are not able to adequately attend to the practical part, therefore process automation and incentives to students must be used to drive the students in the right direction. Our goals was to design an automated programming assignment infrastructure capable of supporting massive courses. The infrastructure should encourage students to apply the key software engineering (SE) practices --- automated testing, configuration management, and Integrated Development Environment (IDE) --- and acquire the basic skills for using the corresponding tools. We selected a few widely adopted development tools used to support the key software engineering practices and mapped them to the basic activities in our exam assignment management process. This experience report describes the results from the past academic year. The infrastructure we built has been used for a full academic year and supported four exam sessions for a total of over a thousand students. The satisfaction level reported by the students is generally high