AM Last Page: Quality Criteria in Qualitative and Quantitative Research

Item does not contain fulltex

The AMEE Research Committee: Initiatives to stimulate research and practice

10.3109/0142159X.2012.671976Medical Teacher346458-461MEDT

The pursuit of fairness in assessment: Looking beyond the objective COMMENT

Health professions education has undergone significant changes over the last few decades, including the rise of competency-based medical education, a shift to authentic workplace-based assessments, and increased emphasis on programmes of assessment. Despite these changes, there is still a commonly held assumption that objectivity always leads to and is the only way to achieve fairness in assessment. However, there are well-documented limitations to using objectivity as the 'gold standard' to which assessments are judged. Fairness, on the other hand, is a fundamental quality of assessment and a principle that almost no one contests. Taking a step back and changing perspectives to focus on fairness in assessment may help re-set a traditional objective approach and identify an equal role for subjective human judgement in assessment alongside objective methods. This paper explores fairness as a fundamental quality of assessments. This approach legitimises human judgement and shared subjectivity in assessment decisions alongside objective methods. Widening the answer to the question: 'What is fair assessment' to include not only objectivity but also expert human judgement and shared subjectivity can add significant value in ensuring learners are better equipped to be the health professionals required of the 21st century

Semantic competence and prototypical verbalizations are associated with higher OSCE and global medical degree scores: a multi-theory pilot study on year 6 medical student verbalizations

Objectives: The organization of medical knowledge is reflected in language and can be studied from the viewpoints of semantics and prototype theory. The purpose of this study is to analyze student verbalizations during an Objective Structured Clinical Examination (OSCE) and correlate them with test scores and final medical degree (MD) scores. We hypothesize that students whose verbalizations are semantically richer and closer to the disease prototype will show better academic performance.Methods: We conducted a single-center study during a year 6 (Y6) high-stakes OSCE where one probing intervention was included at the end of the exam to capture students' reasoning about one of the clinical cases. Verbalizations were transcribed and coded. An assessment panel categorized verbalizations regarding their semantic value (Weak, Good, Strong). Semantic categories and prototypical elements were compared with OSCE, case-based exam and global MD scores.Results: Students with Semantic 'Strong' verbalizations displayed higher OSCE, case-based exam and MD scores, while the use of prototypical elements was associated with higher OSCE and MD scores.Conclusions: Semantic competence and verbalizations matching the disease prototype may identify students with better organization of medical knowledge. This work provides empirical groundwork for future research on language analysis to support assessment decisions

Impact of increased authenticity in instructional format on preclerkship students' performance: a two-year, prospective, randomized study

Item does not contain fulltextPURPOSE: To address whether increasingly authentic instructional formats are more effective in improving preclerkship medical students' performance. METHOD: From 2007 to 2009, the authors conducted a prospective, randomized, crossover study with second-year medical students in a clinical reasoning course at the Uniformed Services University of the Health Sciences. The authors randomly assigned students to one of three cohorts and used instructional formats of differing authenticity (paper, DVD, standardized patient) to teach three subject areas (abdominal pain, anemia, polyuria). Each cohort received one instructional format for each subject area. The authors collected outcome measures (objective structured clinical exam, video quiz, and essay exam scores) at the end of each academic year. They stratified the students into tertiles by first-year grade point average to investigate the impact of instructional formats on learners of different abilities. RESULTS: Outcomes for students in the top tertile improved with increased authenticity of the instructional format compared with outcomes for students in the middle and bottom tertiles (0.188 versus -0.038 and -0.201, P=.001 and .027, respectively). However, outcomes for students in the bottom tertile decreased when students were given only the paper case, compared with the middle and top tertiles (-0.374 versus 0.043 and 0.023, respectively, P=.001), but subsequently improved with more authentic instructional formats. CONCLUSIONS: The authors could not demonstrate that increased authenticity of the instructional format resulted in improved learner performance. However, they believe that there may be some benefit to tailoring preclerkship clinical education based on students' ability