Navigating and querying code without getting lost

A development task related to a crosscutting concern is chal-lenging because a developer can easily get lost when explor-ing scattered elements of code and the complex tangle of re-lationships between them. In this paper we present a source browsing tool that improves the developer’s ability to work with crosscutting concerns by providing better support for exploring code. Our tool helps the developer to remain ori-ented while exploring and navigating across a code base. The cognitive burden placed on a developer is reduced by avoiding disorienting view switches and by providing an ex-plicit representation of the exploration process in terms of exploration paths. 1

A Comparative Study of Code Query Technologies

When analyzing software systems we are faced with the challenge of how to implement a particular analysis for different programming languages. A solution for this problem is to write a single analysis using a code query language abstracting from the specificities of languages being analyzed. Over the past ten years many code query technologies have been developed, based on different formalisms. Each technology comes with its own query language and set of features. To determine the state of the art of code querying we compare the languages and tools for seven code query technologies: Grok, Rscript, JRelCal, SemmleCode, JGraLab, CrocoPat and JTransformer. The specification of a package stability metric is used as a running example to compare the languages. The comparison involves twelve criteria, some of which are concerned with properties of the query language (paradigm, types, parametrization, polymorphism, modularity, and libraries), and some of which are concerned with the tool itself (output formats, interactive interface, API support, interchange formats, extraction support, and licensing). We contextualize the criteria in two usage scenarios: interactive and tool integration. We conclude that there is no particularly weak or dominant tool. As important improvement points, we identify the lack of library mechanisms, interchange formats, and possibilities for integration with source code extraction components

Identifying aspects using fan-in analysis

The issues of code scattering and tangling, thus of achieving a better modularity for a system's concerns, are addressed by the paradigm of aspect orientation. Aspect mining is a reverse engineering process that aims at finding crosscutting concerns in existing systems. This paper describes a technique based on determining methods that are called from many different places (and hence have a high 'fan-in') to identify candidate aspects in a number of open-source Java systems. The most interesting aspects identified are discussed in detail, which includes several concerns not previously discussed in the aspect-oriented literature. The results show that a significant number of aspects can be recognized using fan-in analysis, and that the technique is suitable for a high degree of automatio

Declarative visitors to ease fine-grained source code mining with full history on billions of AST nodes

Software repositories contain a vast wealth of information about software development. Mining these repositories has proven useful for detecting patterns in software development, testing hypotheses for new software engineering approaches, etc. Specifically, mining source code has yielded significant insights into software development artifacts and processes. Unfortunately, mining source code at a large-scale remains a difficult task. Previous approaches had to either limit the scope of the projects studied, limit the scope of the mining task to be more coarse-grained, or sacrifice studying the history of the code due to both human and computational scalability issues. In this paper we address the substantial challenges of mining source code: a) at a very large scale; b) at a fine-grained level of detail; and c) with full history information. To address these challenges, we present domain-specific language features for source code mining. Our language features are inspired by object-oriented visitors and provide a default depth-first traversal strategy along with two expressions for defining custom traversals. We provide an implementation of these features in the Boa infrastructure for software repository mining and describe a code generation strategy into Java code. To show the usability of our domain-specific language features, we reproduced over 40 source code mining tasks from two large-scale previous studies in just 2 person-weeks. The resulting code for these tasks show between 2.0x--4.8x reduction in code size. Finally we perform a small controlled experiment to gain insights into how easily mining tasks written using our language features can be understood, with no prior training. We show a substantial number of tasks (77%) were understood by study participants, in about 3 minutes per task

Source-code queries with graph databases - With application to programming language usage and evolution

Program querying and analysis tools are of growing importance, and occur in two main variants. Firstly there are source-code query languages which help software engineers to explore a system, or to find code in need of refactoring as coding standards evolve. These also enable language designers to understand the practical uses of language features and idioms over a software corpus. Secondly there are program analysis tools in the style of Coverity which perform deeper program analysis searching for bugs as well as checking adherence to coding standards such as MISRA. The former class are typically implemented on top of relational or deductive databases and make ad-hoc trade-offs between scalability and the amount of source-code detail held - with consequent limitations on the expressiveness of queries. The latter class are more commercially driven and involve more ad-hoc queries over program representations, nonetheless similar pressures encourage user-visible domain-specific languages to specify analyses. We argue that a graph data model and associated query language provides a unifying conceptual model and gives efficient scalable implementation even when storing full source-code detail. It also supports overlays allowing a query DSL to pose queries at a mixture of syntax-tree, type, control-flow-graph or data-flow levels. We describe a prototype source-code query system built on top of Neo4j using its Cypher graph query language; experiments show it scales to multi-million-line programs while also storing full source-code detail

An experimental study of the effectiveness of Panorama as a maintenance tool

It is a well known fact that the amount of time and money spent debugging and maintaining software far outstrips the amount spent developing it. Reports suggest that a majority of this effort is related to information-seeking, concept-location, and software comprehension. In addition to being inherently complex activities, these problems are exacerbated due to the fact that the original developers are often unavailable to help with the maintenance activities. To perform a maintenance task programmers first form dynamic and static mental models of the program based on their prior knowledge of programming and on information available about the program. Aside from high level design documents, Javadoc-like documentation has been the primary means of obtaining such information about programs. There are two severe shortcomings of this form of documentation: a) as recognized by proponents of AOP, the code for many of the concerns of interest to the maintainer are scattered across different classes and files and b) there are a lot of classes and files for large systems leading to information overload and it is very hard and time-consuming to identify the few segments of code and documentation that are relevant for a specific maintenance activity. To address these problems, we have developed an Eclipse plug-in, Panorama, which allow master developers to document sections of code related to a specific developer concern and provides ways to make such information readily available to maintainers. This tool helps directly with information-seeking, concept location, and software comprehension activities and focuses maintainer\u27s attention on the specific code and guidance necessary for a particular task. To validate the tool and our approach, we conducted an experimental study with 19 subjects assigning them several maintenance tasks on a fairly complex application (a multi-user Poker game). The study showed that the experimental group who had access to Panorama was able to complete more maintenance tasks in less time than the control group who did not have access to the tool. In addition, preliminary results indicate that the tool will have more impact on productivity of persons new to the application and programming environment than on experts

Matching dependence-related queries in the system dependence graph.

In software maintenance and evolution, it is common that develop-ers want to apply a change to a number of similar places. Due to the size and complexity of the code base, it is challenging for develop-ers to locate all the places that need the change. A main challenge in locating the places that need the change is that, these places share certain common dependence conditions but existing code searching techniques can hardly handle dependence relations satisfactorily. In this paper, we propose a technique that enables developers to make queries involving dependence conditions and textual condi-tions on the system dependence graph of the program. We carried out an empirical evaluation on four searching tasks taken from the development history of two real-world projects. The results of our evaluation indicate that, compared with code-clone detection, our technique is able to locate many required code elements that code-clone detection cannot locate, and compared with text search, our technique is able to effectively reduce false positives without losing any required code elements