Towards Surgically-Precise Technical Debt Estimation: Early Results and Research Roadmap

The concept of technical debt has been explored from many perspectives but its precise estimation is still under heavy empirical and experimental inquiry. We aim to understand whether, by harnessing approximate, data-driven, machine-learning approaches it is possible to improve the current techniques for technical debt estimation, as represented by a top industry quality analysis tool such as SonarQube. For the sake of simplicity, we focus on relatively simple regression modelling techniques and apply them to modelling the additional project cost connected to the sub-optimal conditions existing in the projects under study. Our results shows that current techniques can be improved towards a more precise estimation of technical debt and the case study shows promising results towards the identification of more accurate estimation of technical debt.Comment: 6 page

Further Investigation of the Survivability of Code Technical Debt Items

Context: Technical Debt (TD) discusses the negative impact of sub-optimal decisions to cope with the need-for-speed in software development. Code Technical Debt Items (TDI) are atomic elements of TD that can be observed in code artefacts. Empirical results on open-source systems demonstrated how code-smells, which are just one type of TDIs, are introduced and "survive" during release cycles. However, little is known about whether the results on the survivability of code-smells hold for other types of code TDIs (i.e., bugs and vulnerabilities) and in industrial settings. Goal: Understanding the survivability of code TDIs by conducting an empirical study analysing two industrial cases and 31 open-source systems from Apache Foundation. Method: We analysed 133,670 code TDIs (35,703 from the industrial systems) detected by SonarQube (in 193,196 commits) to assess their survivability using survivability models. Results: In general, code TDIs tend to remain and linger for long periods in open-source systems, whereas they are removed faster in industrial systems. Code TDIs that survive over a certain threshold tend to remain much longer, which confirms previous results. Our results also suggest that bugs tend to be removed faster, while code smells and vulnerabilities tend to survive longer.Comment: Submitted to the Journal of Software: Evolution and Process (JSME

The temporality of technical debt introduction on new code and confounding factors

Code Technical Debt (TD) is intentionally or unintentionally created when developers introduce inefficiencies in the codebase. This can be attributed to various reasons such as heavy workload, tight delivery schedule, or developers’ lack of experience. Since a software system grows mostly through the addition of new code, it is interesting to study how TD fluctuates along this process. Specifically, in this paper, we investigate: (a) the temporality of code TD introduction in new code, i.e., whether the introduction of TD is stable across the lifespan of the project, or if its evolution presents spikes; and (b) the relation of TD introduction to the development team’s workload in a given period, as well as to the experience of the development team. To answer these questions, we have performed a case study on 47 open-source projects from two well-known ecosystems (Apache and Eclipse) as well as additional isolated projects from GitHub (not selected from a specific ecosystem) and inspected the number of TD issues introduced in 6-month sliding temporal windows. The results of the study suggested that: (a) overall, the number of TD issues introduced through new code is a stable measure, although it presents spikes; and (b) the number of commits performed, as well as developers’ experience are not strongly correlated to the number of introduced TD issues.</p

Evolution of technical debt remediation in Python: A case study on the Apache Software Ecosystem

In recent years, the evolution of software ecosystems and the detection of technical debt received significant attention by researchers from both industry and academia. While a few studies that analyze various aspects of technical debt evolution already exist, to the best of our knowledge, there is no large-scale study that focuses on the remediation of technical debt over time in Python projects -- i.e., one of the most popular programming languages at the moment. In this paper, we analyze the evolution of technical debt in 44 Python open-source software projects belonging to the Apache Software Foundation. We focus on the type and amount of technical debt that is paid back. The study required the mining of over 60K commits, detailed code analysis on 3.7K system versions, and the analysis of almost 43K fixed issues. The findings show that most of the repayment effort goes into testing, documentation, complexity and duplication removal. Moreover, more than half of the Python technical debt in the ecosystem is short-term being repaid in less than two months. In particular, the observations that a minority of rules account for the majority of issues fixed and spent effort, suggest that addressing those kinds of debt in the future is important for research and practice