An experimental study of the effectiveness of Panorama as a maintenance tool

It is a well known fact that the amount of time and money spent debugging and maintaining software far outstrips the amount spent developing it. Reports suggest that a majority of this effort is related to information-seeking, concept-location, and software comprehension. In addition to being inherently complex activities, these problems are exacerbated due to the fact that the original developers are often unavailable to help with the maintenance activities. To perform a maintenance task programmers first form dynamic and static mental models of the program based on their prior knowledge of programming and on information available about the program. Aside from high level design documents, Javadoc-like documentation has been the primary means of obtaining such information about programs. There are two severe shortcomings of this form of documentation: a) as recognized by proponents of AOP, the code for many of the concerns of interest to the maintainer are scattered across different classes and files and b) there are a lot of classes and files for large systems leading to information overload and it is very hard and time-consuming to identify the few segments of code and documentation that are relevant for a specific maintenance activity. To address these problems, we have developed an Eclipse plug-in, Panorama, which allow master developers to document sections of code related to a specific developer concern and provides ways to make such information readily available to maintainers. This tool helps directly with information-seeking, concept location, and software comprehension activities and focuses maintainer\u27s attention on the specific code and guidance necessary for a particular task. To validate the tool and our approach, we conducted an experimental study with 19 subjects assigning them several maintenance tasks on a fairly complex application (a multi-user Poker game). The study showed that the experimental group who had access to Panorama was able to complete more maintenance tasks in less time than the control group who did not have access to the tool. In addition, preliminary results indicate that the tool will have more impact on productivity of persons new to the application and programming environment than on experts

On the Relationship between Code Verifiability and Understandability

Proponents of software verification have argued that simpler code is easier to verify: that is, that verification tools issue fewer false positives and require less human intervention when analyzing simpler code. We empirically validate this assumption by comparing the number of warnings produced by four state-of-the-art verification tools on 211 snippets of Java code with 20 metrics of code comprehensibility from human subjects in six prior studies. Our experiments, based on a statistical (meta-)analysis, show that, in aggregate, there is a small correlation (r = 0.23) between understandability and verifiability. The results support the claim that easy-to-verify code is often easier to understand than code that requires more effort to verify. Our work has implications for the users and designers of verification tools and for future attempts to automatically measure code comprehensibility: verification tools may have ancillary benefits to understandability, and measuring understandability may require reasoning about semantic, not just syntactic, code properties.Comment: to appear at Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE'23

Towards a Systematic Approach for Driving Micro-Level Software Development Process

Software developers encounter many new and usually difficult problems in their every day work. Solving every new problem requires taking an appropriate strategy and careful planning, that needs full concentration and focus on the problem and enough consciousness about the current state of development. However, due to many factors, such as working on more than one task at the same time, the high degree of granularity of tasks, interruptions and the complexity of problems, developers usually have difficulties in obtaining the required concentration and focus on their current local problems and making progress in their work. This paper proposes a novel approach, which lets a developer follows a systematic step by step problem solving process in order to be focused on only one target in every moment while all needed details from the past and future steps are present to her. Moreover, this approach helps a developer to better understand the software development process and discover new ways for improving it

Reading the documentation of invoked API functions in program comprehension

Comprehending an unfamiliar code fragment requires an awareness of explicit usage directives that may be present in the documentation of some invoked functions. Since it is not practical for developers to thoroughly investigate every call, directives may be missed and errors may occur. We previously reported on a tool called eMoose, which high-lights calls to methods with associated directives, and on a controlled comparative lab study in which eMoose users were more successful at fixing bugs in given code fragments. In this paper we attempt to shed light on the factors behind these differences with a detailed analysis of videos from the study. We argue that information foraging theory may explain the subjects ’ reading choices and the impact of our tool. We also suggest ways to structure documentation to increase the prospects of knowledge acquisition. 1

An empirical study on code comprehension: DCI compared to OO

Comprehension of source code affects software development, especially its maintenance where reading code is the most time consuming performed activity. A programming paradigm imposes a style of arranging the source code that is aligned with a way of thinking toward a computable solution. Then, a programming paradigm with a programming language represents an important factor for source code comprehension. Object-Oriented (OO) is the dominant paradigm today. Although, it was criticized from its beginning and recently an alternative has been proposed. In an OO source code, system functions cannot escape outside the definition of classes and their descriptions live inside multiple class declarations. This results in an obfuscated code, a lost sense the run-time, and in a lack of global knowledge that weaken the understandability of the source code at system level. A new paradigm is emerging to address these and other OO issues, this is the Data Context Interaction (DCI) paradigm. We conducted the first human subject related controlled experiment to evaluate the effects of DCI on code comprehension compared to OO. We looked for correctness, time consumption, and focus of attention during comprehension tasks. We also present a novel approach using metrics from Social Network Analysis to analyze what we call the Cognitive Network of Language Elements (CNLE) that is built by programmers while comprehending a system. We consider this approach useful to understand source code properties uncovered from code reading cognitive tasks. The results obtained are preliminary in nature but indicate that DCI-trygve approach produces more comprehensible source code and promotes a stronger focus the attention in important files when programmers are reading code during program comprehension. Regarding reading time spent on files, we were not able to indicate with statistical significance which approach allows programmers to consume less time

Using contextual knowledge in interactive fault localization

Tool support for automated fault localization in program debugging is limited because state-of-the-art algorithms often fail to provide efficient help to the user. They usually offer a ranked list of suspicious code elements, but the fault is not guaranteed to be found among the highest ranks. In Spectrum-Based Fault Localization (SBFL) – which uses code coverage information of test cases and their execution outcomes to calculate the ranks –, the developer has to investigate several locations before finding the faulty code element. Yet, all the knowledge she a priori has or acquires during this process is not reused by the SBFL tool. There are existing approaches in which the developer interacts with the SBFL algorithm by giving feedback on the elements of the prioritized list. We propose a new approach called iFL which extends interactive approaches by exploiting contextual knowledge of the user about the next item in the ranked list (e. g., a statement), with which larger code entities (e. g., a whole function) can be repositioned in their suspiciousness. We implemented a closely related algorithm proposed by Gong et al. , called Talk . First, we evaluated iFL using simulated users, and compared the results to SBFL and Talk . Next, we introduced two types of imperfections in the simulation: user’s knowledge and confidence levels. On SIR and Defects4J, results showed notable improvements in fault localization efficiency, even with strong user imperfections. We then empirically evaluated the effectiveness of the approach with real users in two sets of experiments: a quantitative evaluation of the successfulness of using iFL , and a qualitative evaluation of practical uses of the approach with experienced developers in think-aloud sessions

An empirically-based characterization and quantification of information seeking through mailing lists during Open Source developers' software evolution

Context Several authors have proposed information seeking as an appropriate perspective for studying software evolution. Empirical evidence in this area suggests that substantial time delays can accrue, due to the unavailability of required information, particularly when this information must travel across geographically distributed sites. Objective As a first step in addressing the time delays that can occur in information seeking for distributed Open Source (OS) programmers during software evolution, this research characterizes the information seeking of OS developers through their mailing lists. Method A longitudinal study that analyses 17 years of developer mailing list activity in total, over 6 different OS projects is performed, identifying the prevalent information types sought by developers, from a qualitative, grounded analysis of this data. Quantitative analysis of the number-of-responses and response time-lag is also performed. Results The analysis shows that Open Source developers are particularly implementation centric and team focused in their use of mailing lists, mirroring similar findings that have been reported in the literature. However novel findings include the suggestion that OS developers often require support regarding the technology they use during development, that they refer to documentation fairly frequently and that they seek implementation-oriented specifics based on system design principles that they anticipate in advance. In addition, response analysis suggests a large variability in the response rates for different types of questions, and particularly that participants have difficulty ascertaining information on other developer's activities. Conclusion The findings provide insights for those interested in supporting the information needs of OS developer communities: They suggest that the tools and techniques developed in support of co-located developers should be largely mirrored for these communities: that they should be implementation centric, and directed at illustrating "how" the system achieves its functional goals and states. Likewise they should be directed at determining the reason for system bugs: a type of question frequently posed by OS developers but less frequently responded to