Integrated online formative assessments in the biomedical sciences for medical students: benefits for learning

Abstract Background Online formative assessments have a sound theoretical basis, and are prevalent and popular in higher education settings, but data to establish their educational benefits are lacking. This study attempts to determine whether participation and performance in integrated online formative assessments in the biomedical sciences has measurable effects on learning by junior medical students. Methods Students enrolled in Phase 1 (Years 1 and 2) of an undergraduate Medicine program were studied over two consecutive years, 2006 and 2007. In seven consecutive courses, end-of-course (EOC) summative examination marks were analysed with respect to the effect of participation and performance in voluntary online formative assessments. Online evaluation surveys were utilized to gather students' perceptions regarding online formative assessments. Results Students rated online assessments highly on all measures. Participation in formative assessments had a statistically significant positive relationship with EOC marks in all courses. The mean difference in EOC marks for those who participated in formative assessments ranged from 6.3% (95% confidence intervals 1.6 to 11.0; p = 0.009) in Course 5 to 3.2% (0.2 to 6.2; p = 0.037) in Course 2. For all courses, performance in formative assessments correlated significantly with EOC marks (p Conclusion The results support the contention that well designed formative assessments can have significant positive effects on learning. There is untapped potential for use of formative assessments to assist learning by medical students and postgraduate medical trainees.</p

Knowledge maps: a tool for online assessment with automated feedback

In higher education, most assessments or examinations comprise either multiple-choice items or open-ended questions such as modified essay questions (MEQs). Online concept and knowledge maps are potential tools for assessment, which might emphasize meaningful, integrated understanding of phenomena. We developed an online knowledge-mapping assessment tool, which provides automated feedback on student-submitted maps. We conducted a pilot study to investigate the potential utility of online knowledge mapping as a tool for automated assessment by comparing the scores generated by the software with manual grading of a MEQ on the same topic for a cohort of first-year medical students. In addition, an online questionnaire was used to gather students’ perceptions of the tool. Map items were highly discriminating between students of differing knowledge of the topic overall. Regression analysis showed a significant correlation between map scores and MEQ scores, and responses to the questionnaire regarding use of knowledge maps for assessment were overwhelmingly positive. These results suggest that knowledge maps provide a similar indication of students’ understanding of a topic as a MEQ, with the advantage of instant, consistent computer grading and time savings for educators. Online concept and knowledge maps could be a useful addition to the assessment repertoire in higher education

Additional file 1 of What works in radiology education for medical students: a systematic review and meta-analysis

Supplementary Material 1: Data Points Extracted from Shortlisted Article

Adaptive tutorials versus web-based resources in radiology: a mixed methods comparison of efficacy and student engagement

Rationale and Objectives: Diagnostic imaging is under-represented in medical curricula globally. Adaptive tutorials, online intelligent tutoring systems that provide a personalized learning experience, have the potential to bridge this gap. However, there is limited evidence of their effectiveness for learning about diagnostic imaging. Materials and Methods: We performed a randomized mixed methods crossover trial to determine the impact of adaptive tutorials on perceived engagement and understanding of the appropriate use and interpretation of common diagnostic imaging investigations. Although concurrently engaged in disparate blocks of study, 99 volunteer medical students (from years 1-4 of the 6-year program) were randomly allocated to one of two groups. In the first arm of the trial on chest X-rays, one group received access to an adaptive tutorial, whereas the other received links to an existing peer-reviewed Web resource. These two groups crossed over in the second arm of the trial, which focused on computed tomography scans of the head, chest, and abdomen. At the conclusion of each arm of the trial, both groups completed an examination-style assessment, comprising questions both related and unrelated to the topics covered by the relevant adaptive tutorial. Online questionnaires were used to evaluate student perceptions of both learning resources. Results: In both arms of the trial, the group using adaptive tutorials obtained significantly higher assessment scores than controls. This was because of higher assessment scores by senior students in the adaptive tutorial group when answering questions related to topics covered in those tutorials. Furthermore, students indicated significantly better engagement with adaptive tutorials than the Web resource and rated the tutorials as a significantly more valuable tool for learning. Conclusions: Medical students overwhelmingly accept adaptive tutorials for diagnostic imaging. The tutorials significantly improve the understanding of diagnostic imaging by senior students

Predictive validity of a new integrated selection process for medical school admission

Background: This paper is an evaluation of an integrated selection process utilising previous academic achievement [Universities Admission Index (UAI)], a skills test [Undergraduate Medicine and Health Sciences Admission Test (UMAT)], and a structured interview, introduced (in its entirety) in 2004 as part of curriculum reform of the undergraduate Medicine Program at the University of New South Wales (UNSW), Australia. Demographic measures of gender, country of birth, educational background and rurality are considered. Method: Admission scores and program outcomes of 318 students enrolled in 2004 and 2005 were studied. Regression analyses were undertaken to determine whether selection scores predicted overall, knowledge-based and clinical-based learning outcomes after controlling for demographics. Results: UAI attained the highest values in predicting overall and knowledge-based outcomes. The communication dimension of the interview achieved similar predictive values as UAI for clinical-based outcomes, although predictive values were relatively low. The UMAT did not predict any performance outcome. Female gender, European/European-derived country of birth and non-rurality were significant predictors independent of UAI scores. Conclusion: Results indicate promising validity for an integrated selection process introduced for the Medicine Program at UNSW, with UAI and interview predictive of learning outcomes. Although not predictive, UMAT may have other useful roles in an integrated selection process. Further longitudinal research is proposed to monitor and improve the validity of the integrated student selection process.10 page(s

Impact on learning of an e-learning module on leukaemia: a randomised controlled trial

<p>Abstract</p> <p>Background</p> <p>e-learning resources may be beneficial for complex or conceptually difficult topics. Leukaemia is one such topic, yet there are no reports on the efficacy of e-learning for leukaemia. This study compared the learning impact on senior medical students of a purpose-built e-learning module on leukaemia, compared with existing online resources.</p> <p>Methods</p> <p>A randomised controlled trial was performed utilising volunteer senior medical students. Participants were randomly allocated to Study and Control groups. Following a pre-test on leukaemia administered to both groups, the Study group was provided with access to the new e-learning module, while the Control group was directed to existing online resources. A post-test and an evaluation questionnaire were administered to both groups at the end of the trial period.</p> <p>Results</p> <p>Study and Control groups were equivalent in gender distribution, mean academic ability, pre-test performance and time studying leukaemia during the trial. The Study group performed significantly better than the Control group in the post-test, in which the group to which the students had been allocated was the only significant predictor of performance. The Study group’s evaluation of the module was overwhelmingly positive.</p> <p>Conclusions</p> <p>A targeted e-learning module on leukaemia had a significant effect on learning in this cohort, compared with existing online resources. We believe that the interactivity, dialogic feedback and integration with the curriculum offered by the e-learning module contributed to its impact. This has implications for e-learning design in medicine and other disciplines.</p