Evolution of technical debt remediation in Python: A case study on the Apache Software Ecosystem

In recent years, the evolution of software ecosystems and the detection of technical debt received significant attention by researchers from both industry and academia. While a few studies that analyze various aspects of technical debt evolution already exist, to the best of our knowledge, there is no large-scale study that focuses on the remediation of technical debt over time in Python projects -- i.e., one of the most popular programming languages at the moment. In this paper, we analyze the evolution of technical debt in 44 Python open-source software projects belonging to the Apache Software Foundation. We focus on the type and amount of technical debt that is paid back. The study required the mining of over 60K commits, detailed code analysis on 3.7K system versions, and the analysis of almost 43K fixed issues. The findings show that most of the repayment effort goes into testing, documentation, complexity and duplication removal. Moreover, more than half of the Python technical debt in the ecosystem is short-term being repaid in less than two months. In particular, the observations that a minority of rules account for the majority of issues fixed and spent effort, suggest that addressing those kinds of debt in the future is important for research and practice