A large-scale empirical exploration on refactoring activities in open source software projects

Refactoring is a well-established practice that aims at improving the internal structure of a software system without changing its external behavior. Existing literature provides evidence of how and why developers perform refactoring in practice. In this paper, we continue on this line of research by performing a large-scale empirical analysis of refactoring practices in 200 open source systems. Specifically, we analyze the change history of these systems at commit level to investigate: (i) whether developers perform refactoring operations and, if so, which are more diffused and (ii) when refactoring operations are applied, and (iii) which are the main developer-oriented factors leading to refactoring. Based on our results, future research can focus on enabling automatic support for less frequent refactorings and on recommending refactorings based on the developer's workload, project's maturity and developer's commitment to the project

Behind the Scenes: On the Relationship Between Developer Experience and Refactoring

Refactoring is widely recognized as one of the efficient techniques to manage technical debt and maintain a healthy software project through enforcing best design practices, or coping with design defects. Previous refactoring surveys have shown that code refactoring activities are mainly executed by developers who have sufficient knowledge of the system’s design, and disposing of leadership roles in their development teams. However, these surveys were mainly limited to specific projects and companies. In this paper, we explore the generalizability of the previous results by analyzing 800 open-source projects. We mine their refactoring activities, and we identify their corresponding contributors. Then, we associate an experience score to each contributor in order to test various hypotheses related to whether developers with higher scores tend to 1) perform a higher number of refactoring operations 2) exhibit different motivations behind their refactoring, and 3) better document their refactoring activity. We found that (1) although refactoring is not restricted to a subset of developers, those with higher contribution score tend to perform more refactorings than others; (2) while there is no correlation between experience and motivation behind refactoring, top contributed developers are found to perform a wider variety of refactoring operations, regardless of their complexity; and (3) top contributed developer tend to document less their refactoring activity. Our qualitative analysis of three randomly sampled projects show that the developers who are responsible for the majority of refactoring activities are typically in advanced positions in their development teams, demonstrating their extensive knowledge of the design of the systems they contribute to

A study of refactorings during software change tasks

Developers frequently undertake software change tasks that could be partially or fully automated by refactoring tools. As has been reported by others, all too often, these refactoring steps are instead performed manually by developers. These missed opportunities are referred to as occasions of disuse of refactoring tools. We perform an observational study in which 17 developers with professional experience attempt to solve three change tasks with steps amenable to the use of refactoring tools. We found that the strategies developers use to approach these tasks shape their workflow, which, in turn, shape the opportunities for refactoring tool use. We report on a number of findings about developer strategies, demonstrating the difficulty of aligning the kind of refactoring steps that emerge during a change task based on the strategy with the tools available. We also report on findings about refactoring tools, such as the difficulties developers face in controlling the scope of application of the tools. Our findings can help inform the designers of refactoring tools.publishedVersio

Towards the Repayment of Self-Admitted Technical Debt

Technical Debt is a metaphor used to express sub-optimal source code implementations that are introduced for short-term benefits that often must be paid back later, at an increased cost. In recent years, various empirical studies have focused on investigating source code comments that indicate Technical Debt, often referred to as Self-Admitted Technical Debt (SATD). In this thesis, we survey research work on SATD, analyzing characteristics of current approaches and techniques for SATD, dividing literature in three categories: detection, comprehension, and repayment. To set the stage for novel and improved work on SATD, we compile tools, resources, and data sets made publicly available. We also identify areas that are missing investigation, open challenges, and discuss potential future research avenues. From the literature survey, we conclude that most findings and contributions have focused on techniques to identify, classify, and comprehend SATD. Few studies focused on the repayment or management of SATD, which is an essential goal of studying technical debt for software maintenance. Therefore, we perform an empirical study towards SATD repayment. We conducted a preliminary online survey with developers to understand the elements they consider to prioritize SATD. With the acquired knowledge from the survey responses and previous literature work, we select metrics to estimate SATD repayment effort. We examine SATD instances found in software systems to see how it has been repaid and investigate the possibility of using historical data at the time of SATD introduction as indicators for SATD that should be addressed. We find two SATD repayment effort metrics that can be consistently modeled in our studied projects and surface the best early indicators for important SATD

Detection of Rename Local Variable Refactoring Instances in Commit History

Detecting refactoring instances occurred in successive revisions of software systems can provide wealthy information for several purposes, e.g., to facilitate the code review process, to devise more accurate code merging techniques, to help the developers of API clients to ease their adaptation to API changes, and to enable more accurate empirical studies on the refactoring practice. In the literature there are several techniques proposed for refactoring detection, supporting a wide variety of refactoring types. Yet, almost all of them have missed an extensively-applied refactoring type, i.e., Rename Local Variable refactoring. In addition, all these techniques rely on similarity thresholds (which are difficult to tune), or need the systems under analysis to be fully built (which is usually a daunting task), or depend on specific IDEs (which drastically limits their effectiveness and usability). In this thesis, we extend the state-of-the-art refactoring detection tool, RefactoringMiner, by defining necessary rules and extending its core algorithms to tailor it for accurately detecting Rename Local Variable refactoring instances. We have evaluated the proposed technique on two large-scale open-source systems, namely dnsjava and Tomcat. Our comparison with REPENT, the state-of-the-art tool in detecting Rename Local Variable refactoring instances, shows that our approach is superior in terms of precision and recall. Moreover, to automatically create a reference corpus of refactoring instances which is required for the evaluation, we have built a fully-automated infrastructure (called RefBenchmark) that is able to invoke several refactoring detection tools and find the agreements/disagreements between their results

Software restructuring: understanding longitudinal architectural changes and refactoring

The complexity of software systems increases as the systems evolve. As the degradation of the system's structure accumulates, maintenance effort and defect-proneness tend to increase. In addition, developers often opt to employ sub-optimal solutions in order to achieve short-time goals, in a phenomenon that has been recently called technical debt. In this context, software restructuring serves as a way to alleviate and/or prevent structural degradation. Restructuring of software is usually performed in either higher or lower levels of granularity, where the first indicates broader changes in the system's structural architecture and the latter indicates refactorings performed to fewer and localised code elements. Although tools to assist architectural changes and refactoring are available, there is still no evidence these approaches are widely adopted by practitioners. Hence, an understanding of how developers perform architectural changes and refactoring in their daily basis and in the context of the software development processes they adopt is necessary. Current software development is iterative and incremental with short cycles of development and release. Thus, tools and processes that enable this development model, such as continuous integration and code review, are widespread among software engineering practitioners. Hence, this thesis investigates how developers perform longitudinal and incremental architectural changes and refactoring during code review through a wide range of empirical studies that consider different moments of the development lifecycle, different approaches, different automated tools and different analysis mechanisms. Finally, the observations and conclusions drawn from these empirical investigations extend the existing knowledge on how developers restructure software systems, in a way that future studies can leverage this knowledge to propose new tools and approaches that better fit developers' working routines and development processes

An Exploratory Study on the Relationship between Changes and Refactoring

Refactoring aims at improving the internal structure of a software system without changing its external behavior. Previous studies empirically assessed, on the one hand, the benefits of refactoring in terms of code quality and developers' productivity, and on the other hand, the underlying reasons that push programmers to apply refactoring. Results achieved in the latter investigations indicate that besides personal motivation such as the responsibility concerned with code authorship, refactoring is mainly performed as a consequence of changes in the requirements rather than driven by software quality. However, these findings have been derived by surveying developers, and therefore no software repository study has been carried out to corroborate the achieved findings. To bridge this gap, we provide a quantitative investigation on the relationship between different types of code changes (i.e., Fault Repairing Modification, Feature Introduction Modification, and General Maintenance Modification) and 28 different refactoring types coming from 3 open source projects. Results showed that developers tend to apply a higher number of refactoring operations aimed at improving maintainability and comprehensibility of the source code when fixing bugs. Instead, when new features are implemented, more complex refactoring operations are performed to improve code cohesion. Most of the times, the underlying reasons behind the application of such refactoring operations are represented by the presence of duplicate code or previously introduced self-admitted technical debts