Taking Stock of Qualitative Methods of Evaluation:A Study of Practices and Quality Criteria

Abstract

Research on evaluation has mapped the landscape of quantitative evaluation methods. There are far fewer overviews for qualitative methods of evaluation. We present a review of scholarly articles from five widely read evaluation research journals, examining the types of methods used and the transparency of their quality criteria. We briefly look at a large sample of 1070 articles and then randomly select 50 for in-depth study. We document a remarkable variety of qualitative methods, but some stand out: Case studies and stakeholder analysis, often combined with interview techniques. Articles rarely define and conceptualize their methods explicitly. This is understandable from a practical point of view, but it can make it difficult to critically interrogate findings and build systematic knowledge. Finally, we find that the transparency of qualitative criteria required in the literature is not always sufficient, which can hinder the synthesis of results