Background: Code review is a cognitively demanding and time-consuming
process. Previous qualitative studies hinted at how decomposing change sets
into multiple yet internally coherent ones would improve the reviewing process.
So far, literature provided no quantitative analysis of this hypothesis.
Aims: (1) Quantitatively measure the effects of change decomposition on the
outcome of code review (in terms of number of found defects, wrongly reported
issues, suggested improvements, time, and understanding); (2) Qualitatively
analyze how subjects approach the review and navigate the code, building
knowledge and addressing existing issues, in large vs. decomposed changes.
Method: Controlled experiment using the pull-based development model
involving 28 software developers among professionals and graduate students.
Results: Change decomposition leads to fewer wrongly reported issues,
influences how subjects approach and conduct the review activity (by increasing
context-seeking), yet impacts neither understanding the change rationale nor
the number of found defects.
Conclusions: Change decomposition reduces the noise for subsequent data
analyses but also significantly supports the tasks of the developers in charge
of reviewing the changes. As such, commits belonging to different concepts
should be separated, adopting this as a best practice in software engineering