Competency-based simulation assessment of resuscitation skills in emergency medicine postgraduate trainees – a Canadian multi-centred study

Background: The use of high-fidelity simulation is emerging as a desirable method for competency-based assessment in postgraduate medical education. We aimed to demonstrate the feasibility and validity of a multi-centre simulation-based Objective Structured Clinical Examination (OSCE) of resuscitation competence with Canadian Emergency Medicine (EM) trainees.Method: EM postgraduate trainees (n=98) from five Canadian academic centres participated in a high fidelity, 3-station simulation-based OSCE.  Expert panels of three emergency physicians evaluated trainee performances at each centre using the Queen’s Simulation Assessment Tool (QSAT).  Intraclass correlation coefficients were used to measure the inter-rater reliability, and analysis of variance was used to measure the discriminatory validity of each scenario.  A fully crossed generalizability study was also conducted for each examination centre.   Results: Inter-rater reliability in four of the five centres was strong with a median absolute intraclass correlation coefficient (ICC) across centres and scenarios of 0.89 [0.65-0.97]. Discriminatory validity was also strong (p < 0.001 for scenarios 1 and 3; p < 0.05 for scenario 2). Generalizability studies found significant variations at two of the study centres.Conclusions: This study demonstrates the successful pilot administration of a multi-centre, 3-station simulation-based OSCE for the assessment of resuscitation competence in post-graduate Emergency Medicine trainees

The Impact of Electronic Data to Capture Qualitative Comments in a Competency-Based Assessment System

Introduction Digitalizing workplace-based assessments (WBA) holds the potential for facilitating feedback and performance review, wherein we can easily record, store, and analyze data in real time. When digitizing assessment systems, however, it is unclear what is gained and lost in the message as a result of the change in medium. This study evaluates the quality of comments generated in paper vs. electronic media and the influence of an assessor's seniority. Methods Using a realist evaluation framework, a retrospective database review was conducted with paper-based and electronic medium comments. A sample of assessments was examined to determine any influence of the medium on the word count and the Quality of Assessment for Learning (QuAL) score. A correlation analysis evaluated the relationship between word count and QuAL score. Separate univariate analyses of variance (ANOVAs) were used to examine the influence of the assessor's seniority and medium on word count, QuAL score, and WBA scores. Results The analysis included a total of 1,825 records. The average word count for the electronic comments (M=16) was significantly higher than the paper version (M=12; p=0.01). Longer comments positively correlated with QuAL score (r=0.2). Paper-based comments received lower QuAL scores (0.41) compared to electronic (0.51; p0.01). Years in practice was negatively correlated with QuAL score (r=-0.08; p0.001) as was word count (r=-0.2; p0.001). Conclusion Digitization of WBAs increased the length of comments and did not appear to jeopardize the quality of WBAs; these results indicate higher-quality assessment data. True digital transformation may be possible by harnessing trainee data repositories and repurposing them to analyze for faculty-relevant metrics

Faces on a screen: A qualitative study of the virtual and in-person conference experience.

INTRODUCTION: The strengths and weaknesses of virtual and in-person formats within continuing professional development (CPD) are incompletely understood. This study sought to explore attendees perspectives across multiple specialties regarding benefits and limitations of conference formats and strategies for successful virtual and hybrid (i.e., in-person conferences with a virtual option) conferences. METHODS: From December 2020 to January 2021, semistructured interviews were conducted with participants who attended both virtual and in-person CPD conferences. Purposive sampling was utilized to ensure diverse representation of gender, years in practice, location, academic rank, specialty, and practice type. Multiple specialties were intentionally sought to better understand the broader experience among physicians in general, rather than among a specific specialty. Using modified grounded theory approach with a constructivist-interpretivist paradigm, two investigators independently analyzed all interview transcripts. Discrepancies were resolved by in-depth discussion and negotiated consensus. RESULTS: Twenty-six individuals across 16 different specialties were interviewed. We identified three overarching concepts: motivations to attend conferences, benefits and limitations of different conference formats, and strategies to optimize virtual and hybrid conferences. Specific motivators included both professional and personal factors. Benefits of in person included networking/community, immersion, and wellness, while the major limitation was integration with personal life. Benefits of virtual were flexibility, accessibility, and incorporation of technology, while limitations included technical challenges, distractions, limitations for tactile learning, and communication/connection. Benefits of hybrid included more options for access, while limitations included challenges with synchrony of formats and dilution of experiences. Strategies to improve virtual/hybrid conferences included optimizing technology/production, facilitating networking and engagement, and deliberate selection of content. CONCLUSIONS: This study identified several benefits and limitations of each medium as well as strategies to optimize virtual and hybrid CPD conferences. This may help inform future CPD conference planning for both attendees and conference planners alike

Inter-physician conflict in the workplace: an under-explored contributor and manifestation of burnout

Learning Objective: The goal of this study was to gain a deeper understanding of the personal and professional impact inter-physician conflict has on physicians’ well-being.Background: Despite the recognized importance of collaboration and communication, interpersonal conflict amongst healthcare providers in different specialties remains a pervasive issue. Recent work elucidated some of the social conditions and processes that contribute to conflict between EM and IM physicians at the time of admission. However, little is known about the consequences that inter-physician conflict has on providers’ well-being.Objective: The goal of this study was to gain a deeper understanding of the personal and professional impact inter-physician conflict has on physicians’ well-being.Methods: In this study, the authors used constructivist grounded theory to explore themes related to the impact of conflict on individual providers. A purposive sampling approach was used to recruit participants (n=18), which included EM residents and attending physicians as well as IM attending physicians. Two authors conducted hour-long, semi-structured interviews and then coded the transcripts following Charmaz’s three stages of coding. The authors used constant comparative analysis until thematic sufficiency was reached.Results: Participants described personal impacts of inter-physician conflict including emotional and moral distress which occurs in the context of feeling demoralized by colleagues. Conflict further impacted participants professionally by promoting cynicism, job dissatisfaction, and a negative view of their professional identity and professional accomplishments. Finally, participants described the ‘emotional residue’ that remained after conflict, which tended to perpetuate future conflict.Conclusions: Physicians attributed emotional exhaustion, cynicism, and reduced sense of personal accomplishment to conflicts arising during conversations around admission. In this way, inter-physician conflict is an under-recognized contributor to physician burnout that should be explored to support the well-being of trainees and attending physicians

Team and Leadership Performance: An Exploratory Mixed-Methods Analysis Using Interprofessional In Situ Simulation

Learning Objective: To assess leader and team performance during interprofessional in situ simulation and identify characteristics of effective teams.Background: Patient care in the emergency department (ED) is dependent on highly effective interprofessional teams.Objectives: To assess leader and team performance during interprofessional in situ simulation and identify characteristics of effective teams.Methods: This mixed-methods study employed case study methodology. Eligible nurses, technicians, pharmacists, and PGY 2-4 emergency medicine residents at a tertiary academic ED participated in a 10-minute in situ simulation of a critically ill patient. Participants self-rated team performance using the Team Performance Observation Tool (TPOT) 2.0. Two raters independently reviewed simulation videos and rated performance using the TPOT 2.0, Team Emergency Assessment Measure (TEAM), and Ottawa Crisis Resource Management Global Rating Scale (Ottawa GRS). Following the simulations we conducted semi-structured interviews. Transcripts were coded by two coders and underwent thematic analysis.Results: 23 simulations took place between January-April 2021. Two raters’ scores were on the high end of the scales for the TPOT 2.0 (R1 4.90,SD=0.17;R2 4.53,SD=0.27), TEAM (R1 3.89,SD=0.19;R2 3.58,SD=0.39), and Ottawa GRS (R1 6.6,SD=.56;R2 6.2,SD=.54). Team leader attributes including year in residency, age, and gender did not correlate with performance scores. We identified 6 themes: leadership tone, interdependent energy, strategic staffing, optimal communication, simulation empowering team performance, and team entrustment. Participants acknowledged the effectiveness of in situ simulation in promoting entrustment in the clinical setting.Conclusions: The TPOT and TEAM were not discriminatory for high-performing ED teams. Our qualitative analysis revealed features such as entrustability, energy, and team size are important for effective team dynamics but are not completely captured on current tools. ED-specific assessments of interprofessional entrustment may be useful in optimizing readiness for the clinical setting

Belongingness among first-generation students at Stanford School of Medicine [version 1; peer review: 1 approved, 3 approved with reservations]

Background Nationally, underrepresented minorities represent a significant proportion of the first-generation student population. These students also tend to report lower levels of belongingness compared to their peers, which may impact their wellness and be an important factor in their academic success. This study aimed to explore whether status as a first-generation student was associated with belongingness amongst medical students. Methods In 2019, a previously validated 16-item survey was used to examine potential disparities in belongingness amongst groups of first-generation medical students. Differences between groups were assessed using a Mann-Whitney U-test for each individual item and three composite groupings of items regarding social belonging, academic belonging, and institutional support. Results First-generation to college and first-generation to graduate school students reported lower belongingness across most individual items as well as in all three composite groups. Conclusions Given that peer relationships and institutional support play an important role in medical student belonging, these findings represent an opportunity to address the specific needs of individuals from underrepresented groups in medicine. Doing so can support the academic and professional success of first-generation students and help close the diversity gap in medicine