Location of Repository

The one minute mentor : a pilot study assessing medical students’ and residents’ professional behaviours through recordings of clinical preceptors’ immediate feedback

By D. Topps, Rebecca Evans, Jill Thistlethwaite, Rod Nan Tie and Rachel Ellaway

Abstract

Introduction: The assessment of professional development and behaviour is an important issue in the training of medical students and physicians. Several methods have been developed for doing so. What is still needed is a method that combines assessment of actual behaviour in the workplace with timely feedback to learners.\ud Goal: We describe the development, piloting and evaluation of a method for assessing professional behaviour using digital audio recordings of clinical supervisors’ brief feedback. We evaluate the inter-rater reliability, acceptability and feasibility of this approach.\ud Methods: Six medical students in Year 5 and three GP registrars (residents) took part in this pilot project. Each had a personal digital assistant (PDA) and approached their clinical supervisors to give approximately one minute of verbal feedback on professionalism-related behaviours they had observed in the registrar’s clinical encounters. The comments, both in transcribed text format and audio, were scored by five evaluators for competence (the learner’s performance) and confidence (how confident the evaluator was that the comment clearly described an observed behaviour or attribute that was relevant). Students and evaluators were surveyed for feedback on the process.\ud Results: Study evaluators rated 29 comments from supervisors in text and audio format. There was good inter-rater reliability (Cronbach α around 0.8) on competence scores. There was good agreement (paired t-test) between scores across supervisors for assessments of comments in both written and audio formats. Students found the method helpful in providing feedback on professionalism. Evaluators liked having a relatively objective approach for judging behaviours and attributes but found scoring audio comments to be time-consuming.\ud Discussion: This method of assessing learners’ professional behaviour shows potential for providing both formative and summative assessment in a way that is feasible and acceptable to students and evaluators. Initial data shows good reliability but to be valid, training of clinical supervisors is necessary to help them provide useful comments based on defined behaviours and attributes of students. In addition, the validity of the scoring method remains to be confirmed

Topics: LB, RA
Publisher: Maastricht University
Year: 2009
OAI identifier: oai:wrap.warwick.ac.uk:4418

Suggested articles

Preview

Citations

  1. (2005). 360 degree feedback. Best practice guidelines., from www.dti.gov.uk/bestpractice.
  2. (2009). A licence to publish this material has been given to Education for Health:
  3. (2000). Challenges in using rater judgements in medical education. doi
  4. (2002). Evaluating physicians' professionalism and humanism: the case for humanism "connoisseurs". doi
  5. (2004). How to conceptualize professionalism: a qualitative study. doi
  6. (2005). Measuring professionalism: a review of studies with instruments reported in the literature between 1982 and doi
  7. (1997). Professionalism must be taught. doi
  8. (2003). Teaching professionalism to residents. doi
  9. (2005). The changing face of assessment: swings and roundabouts.
  10. (2003). To be and not to be: the paradox of the emerging professional stance. doi
  11. (2004). Unprofessional behavior in medical school is associated with subsequent disciplinary action by a state medical board. doi

To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.