1 research outputs found
Practitioners' Perceptions of the Goals and Visual Explanations of Defect Prediction Models
Software defect prediction models are classifiers that are constructed from
historical software data. Such software defect prediction models have been
proposed to help developers optimize the limited Software Quality Assurance
(SQA) resources and help managers develop SQA plans. Prior studies have
different goals for their defect prediction models and use different techniques
for generating visual explanations of their models. Yet, it is unclear what are
the practitioners' perceptions of (1) these defect prediction model goals, and
(2) the model-agnostic techniques used to visualize these models. We conducted
a qualitative survey to investigate practitioners' perceptions of the goals of
defect prediction models and the model-agnostic techniques used to generate
visual explanations of defect prediction models. We found that (1) 82%-84% of
the respondents perceived that the three goals of defect prediction models are
useful; (2) LIME is the most preferred technique for understanding the most
important characteristics that contributed to a prediction of a file, while
ANOVA/VarImp is the second most preferred technique for understanding the
characteristics that are associated with software defects in the past. Our
findings highlight the significance of investigating how to improve the
understanding of defect prediction models and their predictions. Hence,
model-agnostic techniques from explainable AI domain may help practitioners to
understand defect prediction models and their predictions.Comment: Accepted for publication at the International Conference on Mining
Software Repositories (MSR'21) (10 pages + 2 references