Development and testing of an analytic rubric for a master's course systematic review of the literature : A cross-sectional study

Background: Conducting grading of systematic reviews in master's level programs of health sciences education is a complex process. Students conduct systematic reviews under the supervision of course faculty in seminar groups where both draft version and definite version of the literature review are graded/assessed. The aim of this study was to develop a systematic review of the literature rubric (SRL-rubric) for the evaluation of systematic reviews in the course of SRL in a master's Program of Health Care Sciences and to investigate students and faculty experiences with and the usability of the SRL-rubric. Methods: The SRL-rubric was developed using a seven-step approach. Usability was investigated with cross-sectional survey. Results: The SRL-rubric included nine categories and five proficiency levels. Fifty-Two of 59 students and all six faculty members at Utrecht University Program of Health Care Sciences completed the survey. Students rated the ease of working with the rubric with an average 6.6 (10-point scale). Faculty ratings ranged from 7 to 9. Problems were identified with distinction among cells describing proficiency levels and final grading. Discussion: A structured process focused on the requisite actions to develop the SRL-rubric. It was useful in writing and grading systematic reviews. However, some students indicated that they missed specific feedback and suggestions describing how to improve their weaknesses. Further development and research is needed to enhance grading reliability of SLR-rubric and to establish content validity and maintain consistency with criteria for conducting and reporting reviews,