Competency-based simulation assessment of resuscitation skills in emergency medicine postgraduate trainees – a Canadian multi-centred study

Background: The use of high-fidelity simulation is emerging as a desirable method for competency-based assessment in postgraduate medical education. We aimed to demonstrate the feasibility and validity of a multi-centre simulation-based Objective Structured Clinical Examination (OSCE) of resuscitation competence with Canadian Emergency Medicine (EM) trainees.Method: EM postgraduate trainees (n=98) from five Canadian academic centres participated in a high fidelity, 3-station simulation-based OSCE.  Expert panels of three emergency physicians evaluated trainee performances at each centre using the Queen’s Simulation Assessment Tool (QSAT).  Intraclass correlation coefficients were used to measure the inter-rater reliability, and analysis of variance was used to measure the discriminatory validity of each scenario.  A fully crossed generalizability study was also conducted for each examination centre.   Results: Inter-rater reliability in four of the five centres was strong with a median absolute intraclass correlation coefficient (ICC) across centres and scenarios of 0.89 [0.65-0.97]. Discriminatory validity was also strong (p < 0.001 for scenarios 1 and 3; p < 0.05 for scenario 2). Generalizability studies found significant variations at two of the study centres.Conclusions: This study demonstrates the successful pilot administration of a multi-centre, 3-station simulation-based OSCE for the assessment of resuscitation competence in post-graduate Emergency Medicine trainees

Cela va dans les deux sens : le point de vue des résidents sur l’efficacité des relations de coaching dans l’environnement d’apprentissage clinique

Background: Coaching has gained traction in postgraduate medical education to enhance interactions between residents and clinical teachers, but these relationships present unique challenges and tensions. In order to realize the promises of coaching in medical education, we must understand how coaching relationships can be enacted to optimize resident development. The purpose of this study was to explore residents’ perceptions of key characteristics for effective clinical teacher-resident (CT-R) coaching relationships.  Methods: We conducted four focus groups and eight interviews with residents at a Canadian academic center. Using a social constructionist approach, focus groups and interviews were audio recorded, transcribed verbatim, and thematically analyzed.  Results: Residents described three main characteristics that contributed to effective CT-R coaching relationships: safe, meaningful, and collaborative. Residents emphasized that these characteristics needed to be bidirectional in nature to be most effective, in that both the resident and clinical teacher embodied these characteristics.  Conclusions: Residents identified that effective coaching relationships were shaped not only by clinical teacher behaviours, but importantly, the quality of the interpersonal relationship that was fostered. Thus, it is imperative to consider the bidirectional nature of the CT-R coaching relationship when striving to enhance resident development.Contexte : Bien que le coaching ait gagné du terrain en éducation médicale postgraduée, notamment comme moyen de renforcer les interactions entre les résidents et les cliniciens enseignants, les rapports entre eux présentent néanmoins des défis et des tensions uniques. Pour atteindre les résultats souhaités en lien avec le coaching en éducation médicale, il faut comprendre comment ces interactions peuvent être mises à profit de façon à optimiser le perfectionnement des résidents. L’objectif de cette étude était d’explorer les perceptions des résidents quant aux caractéristiques clés d’une relation de coaching efficace entre le clinicien enseignant et le résident.  Méthodes : Nous avons organisé quatre groupes de discussion et huit entretiens avec des résidents d’un centre universitaire canadien. Suivant une approche socioconstructiviste, les groupes de discussion et les entrevues ont été enregistrés, les verbatims ont été transcrits et ensuite analysés de façon thématique.  Résultats : Les résidents ont décrit trois caractéristiques principales de la relation de coaching pouvant contribuer à son efficacité : elle doit être sécuritaire, significative et collaborative. Ils ont insisté sur le fait que pour atteindre une efficacité optimale des interactions de coaching, ces caractéristiques devaient être bidirectionnelles, à savoir présentes tant chez le résident que le clinicien enseignant.  Conclusions : D’après les résidents, si l’efficacité de la relation de coaching est déterminée par les comportements des cliniciens enseignants, elle dépend surtout de la qualité des rapports interpersonnels cultivés. Il est donc impératif de tenir compte de la nature bidirectionnelle de la relation de coaching entre cliniciens enseignants et résidents lorsqu’on vise à améliorer le développement de ces derniers

It’s a ‘two-way street’: resident perspectives of effective coaching relationships in the clinical learning environment

Background: Coaching has gained traction in postgraduate medical education to enhance interactions between residents and clinical teachers, but these relationships present unique challenges and tensions. In order to realize the promises of coaching in medical education, we must understand how coaching relationships can be enacted to optimize resident development. The purpose of this study was to explore residents’ perceptions of key characteristics for effective clinical teacher-resident (CT-R) coaching relationships. Methods: We conducted four focus groups and eight interviews with residents at a Canadian academic center. Using a social constructionist approach, focus groups and interviews were audio recorded, transcribed verbatim, and thematically analyzed. Results: Residents described three main characteristics that contributed to effective CT-R coaching relationships: safe, meaningful, and collaborative. Residents emphasized that these characteristics needed to be bidirectional in nature to be most effective, in that both the resident and clinical teacher embodied these characteristics. Conclusions: Residents identified that effective coaching relationships were shaped not only by clinical teacher behaviours, but importantly, the quality of the interpersonal relationship that was fostered. Thus, it is imperative to consider the bidirectional nature of the CT-R coaching relationship when striving to enhance resident development.Contexte : Bien que le coaching ait gagné du terrain en éducation médicale postgraduée, notamment comme moyen de renforcer les interactions entre les résidents et les cliniciens enseignants, les rapports entre eux présentent néanmoins des défis et des tensions uniques. Pour atteindre les résultats souhaités en lien avec le coaching en éducation médicale, il faut comprendre comment ces interactions peuvent être mises à profit de façon à optimiser le perfectionnement des résidents. L’objectif de cette étude était d’explorer les perceptions des résidents quant aux caractéristiques clés d’une relation de coaching efficace entre le clinicien enseignant et le résident. Méthodes : Nous avons organisé quatre groupes de discussion et huit entretiens avec des résidents d’un centre universitaire canadien. Suivant une approche socioconstructiviste, les groupes de discussion et les entrevues ont été enregistrés, les verbatims ont été transcrits et ensuite analysés de façon thématique. Résultats : Les résidents ont décrit trois caractéristiques principales de la relation de coaching pouvant contribuer à son efficacité : elle doit être sécuritaire, significative et collaborative. Ils ont insisté sur le fait que pour atteindre une efficacité optimale des interactions de coaching, ces caractéristiques devaient être bidirectionnelles, à savoir présentes tant chez le résident que le clinicien enseignant. Conclusions : D’après les résidents, si l’efficacité de la relation de coaching est déterminée par les comportements des cliniciens enseignants, elle dépend surtout de la qualité des rapports interpersonnels cultivés. Il est donc impératif de tenir compte de la nature bidirectionnelle de la relation de coaching entre cliniciens enseignants et résidents lorsqu’on vise à améliorer le développement de ces derniers

Starting to Think Like an Expert:An Analysis of Resident Cognitive Processes During Simulation-Based Resuscitation Examinations

Study objective: Simulation is commonly used to teach crisis resource management skills and assess them in emergency medicine residents. However, our understanding of the cognitive processes underlying crisis resource management skills is limited because these processes are difficult to assess and describe. The objective of this study is to uncover and characterize the cognitive processes underlying crisis resource management skills and to describe how these processes vary between residents according to performance in a simulation-based examination. Methods: Twenty-two of 24 eligible emergency medicine trainees from 1 tertiary academic center completed 1 or 2 resuscitation-based examinations in the simulation laboratory. Resident performance was assessed by a blinded expert using an entrustment-based scoring tool. Participants wore eye-tracking glasses that generated first-person video that was used to augment subsequent interviews led by an emergency medicine faculty member. Interviews were audio recorded and then transcribed. An emergent thematic analysis was completed with a codebook that was developed by 4 research assistants, with subsequent analyses conducted by the lead research assistant with input from emergency medicine faculty. Themes from high- and low-performing residents were subsequently qualitatively compared. Results: Higher-performing residents were better able to anticipate, selectively attend to relevant information, and manage cognitive demands, and took a concurrent (as opposed to linear) approach to managing the simulated patient. Conclusion: The results provide new insights into residents' cognitive processes while managing simulated patients in an examination environment and how these processes vary with performance. More work is needed to determine how best to apply these findings to improve crisis resource management education

A new way to look at simulation-based assessment: the relationship between gaze-tracking and exam performance

Objective A key task of the team leader in a medical emergency is effective information gathering. Studying information gathering patterns is readily accomplished with the use of gaze-tracking glasses. This technology was used to generate hypotheses about the relationship between performance scores and expert-hypothesized visual areas of interest in residents across scenarios in simulated medical resuscitation examinations. Methods Emergency medicine residents wore gaze-tracking glasses during two simulation-based examinations (n=29 and 13 respectively). Blinded experts assessed video-recorded performances using a simulation performance assessment tool that has validity evidence in this context. The relationships between gaze patterns and performance scores were analyzed and potential hypotheses generated. Four scenarios were assessed in this study: diabetic ketoacidosis, bradycardia secondary to beta-blocker overdose, ruptured abdominal aortic aneurysm and metabolic acidosis caused by antifreeze ingestion. Results Specific gaze patterns were correlated with objective performance. High performers were more likely to fixate on task-relevant stimuli and appropriately ignore task-irrelevant stimuli compared with lower performers. For example, shorter latency to fixation on the vital signs in a case of diabetic ketoacidosis was positively correlated with performance (r=0.70, p</p