3 research outputs found
Why Research on Test-Driven Development is Inconclusive?
[Background] Recent investigations into the effects of Test-Driven
Development (TDD) have been contradictory and inconclusive. This hinders
development teams to use research results as the basis for deciding whether and
how to apply TDD. [Aim] To support researchers when designing a new study and
to increase the applicability of TDD research in the decision-making process in
the industrial context, we aim at identifying the reasons behind the
inconclusive research results in TDD. [Method] We studied the state of the art
in TDD research published in top venues in the past decade, and analyzed the
way these studies were set up. [Results] We identified five categories of
factors that directly impact the outcome of studies on TDD. [Conclusions] This
work can help researchers to conduct more reliable studies, and inform
practitioners of risks they need to consider when consulting research on TDD.Comment: ESEM '20: ACM / IEEE International Symposium on Empirical Software
Engineering and Measurement (ESEM), October 8--9, 2020, Bari, Ital
Does the performance of TDD hold across software companies and premises?:a group of industrial experiments on TDD
Abstract
Test-Driven Development (TDD) has been claimed to increase external software quality. However, the extent to which TDD increases external quality has been seldom studied in industrial experiments. We conduct four industrial experiments in two different companies to evaluate the performance of TDD on external quality. We study whether the performance of TDD holds across the premises of the same company and across companies. We identify participant-level characteristics impacting results. Iterative-Test Last (ITL), the reverse approach of TDD, outperforms TDD in three out of four premises. ITL outperforms TDD in both companies. The larger the experience with unit testing and testing tools, the larger the difference in performance between ITL and TDD (in favour of ITL). Technological environment (i.e., programming language and testing tool) seems not to impact results. Evaluating participant-level characteristics impacting results in industrial experiments may ease the understanding of TDD’s performance in realistic settings
A Family of Experiments on Test-Driven Development
Context: Test-driven development (TDD) is an agile software development
approach that has been widely claimed to improve software quality. However, the
extent to which TDD improves quality appears to be largely dependent upon the
characteristics of the study in which it is evaluated (e.g., the research
method, participant type, programming environment, etc.). The particularities
of each study make the aggregation of results untenable. Objectives: The goal
of this paper is to: increase the accuracy and generalizability of the results
achieved in isolated experiments on TDD, provide joint conclusions on the
performance of TDD across different industrial and academic settings, and
assess the extent to which the characteristics of the experiments affect the
quality-related performance of TDD. Method: We conduct a family of 12
experiments on TDD in academia and industry. We aggregate their results by
means of meta-analysis. We perform exploratory analyses to identify variables
impacting the quality-related performance of TDD. Results: TDD novices achieve
a slightly higher code quality with iterative test-last development (i.e., ITL,
the reverse approach of TDD) than with TDD. The task being developed largely
determines quality. The programming environment, the order in which TDD and ITL
are applied, or the learning effects from one development approach to another
do not appear to affect quality. The quality-related performance of
professionals using TDD drops more than for students. We hypothesize that this
may be due to their being more resistant to change and potentially less
motivated than students. Conclusion: Previous studies seem to provide
conflicting results on TDD performance (i.e., positive vs. negative,
respectively). We hypothesize that these conflicting results may be due to
different study durations, experiment participants being unfamiliar with the
TDD process..