Rétroaction sur la rétroaction : une voie à double sens entre les résidents et les superviseurs

Background: Workplace-based assessment (WBA), foundational to competency-based medical education, relies on preceptors providing feedback to residents. Preceptors however get little timely, formative, specific, actionable feedback on the effectiveness of that feedback. Our study aimed to identify useful qualities of feedback for family medicine residents and to inform improving feedback-giving skills for preceptors in PGME training program. Methods: This study employed a two-phase exploratory design. Phase 1 collected qualitative data from preceptor feedback given to residents through Field Notes (FNs) and quantitative data from residents who provided feedback to preceptor about the quality of the feedback given. Phase 2 employed focus groups to explore ways in which residents are willing to provide preceptors with constructive feedback about the quality of the feedback they receive. Descriptive statistics and a thematic approach were used for data analysis. Findings: We collected 22 FNs identified by residents as being impactful to their learning; analysis of these FNs resulted in five themes. Functionality was then added to the electronic FNs allowing residents to indicate impactful feedback with a “Thumbs Up” icon. Over one year, 895 out of 8,496 FNs (11%) had a “Thumbs up” added, divided into reasons of: confirmation of learning (28.6%), practice improvement (21.2%), new learning (18.8%), motivation (17.7%), and evoking reflection (13.7%). Two focus groups (12 residents, convenience sampling) explored residents’ perception of constructive feedback and willingness to also provide constructive feedback to preceptors. Conclusion: Adding constructive feedback to existing positive feedback choices will provide preceptors with holistic information about the impact of their feedback on learners, which, in turn, should allow them to provide more effective feedback to learners. However, power differential, relationship impact, and institutional support were concerns for residents that would need to be addressed for this to be optimally operationalized.Contexte : L’évaluation en milieu de travail (EMT), qui est à la base de la formation médicale fondée sur les compétences, repose sur la rétroaction donnée par les superviseurs aux résidents. En revanche, il est rare que les superviseurs reçoivent à leur tour, et en temps utile, une rétroaction formative, précise et pratique sur l’efficacité des commentaires qu’ils ont offerts. L’objectif de notre étude était de déceler les caractéristiques qu’un tel retour doit avoir pour être utile aux résidents en médecine familiale afin de guider l’amélioration des compétences en rétroaction des superviseurs de programmes d’éducation médicale postdoctorale. Méthodes : Nous avons utilisé un devis exploratoire en deux phases. Lors de la première phase, nous avons recueilli les données qualitatives à partir des commentaires fournis par les précepteurs aux résidents par le biais de feuilles de route (FR) et les données quantitatives de rétroactiondes résidents sur la qualité de la rétroaction qui leur a été offerte par les superviseurs. Dans la phase 2, des groupes de discussion ont été constitués pour explorer les moyens par lesquels les résidents sont prêts à fournir aux superviseurs une rétroaction constructive sur la qualité de la rétroaction qu’ils reçoivent. L’analyse des données a été faite à l’aide de statistiques descriptives et d’une approche thématique. Résultats : Les auteurs ont recueilli 22 feuilles de route (FR) qui, d’après les résidents, ont eu un effet sur leur apprentissage. L’analyse des FR a permis de relever cinq thèmes. Une fonctionnalité a par la suite été ajoutée aux FR électroniques, permettant aux résidents d’indiquer qu’une rétroaction a été efficace à l’aide d’un pictogramme « Pouce levé ». En un an, 895 des 8496 FR (11 %) ont reçu un « Pouce levé » et les raisons qui l’expliquent sont réparties de la manière suivante : la rétroaction confirmait au résident ses acquis (28,6 %), elle l’aidait à améliorer sa pratique (21,2 %), elle lui apprenait quelque chose de nouveau (18,8 %), elle stimulait sa motivation (17,7 %) ou encore sa réflexion (13,7 %). Deux groupes de discussion (12 résidents, échantillon de convenance) ont exploré ce que les résidents perçoivent comme étant une rétroaction constructive, et s’ils sont prêts à fournir à leur tour un commentaire constructif aux superviseurs. Conclusion : Le fait d’ajouter des commentaires constructifs aux commentaires positifs renseignerait les superviseurs de manière plus complète sur l’effet qu’a eu leur rétroaction pour les apprenants, et les aiderait ainsi à améliorer l’efficacité des rétroactions qu’ils donnent. Cependant, pour mettre en place ce processus de manière optimale, il faut tenir compte des préoccupations qu’ont les résidents quant au déséquilibre de pouvoir, aux répercussions sur leur relation avec le superviseur et à l’appui de l’établissement

Making assessment a team sport: a qualitative study of facilitated group feedback in internal medicine residency

Purpose: Competency-based medical education relies on feedback from workplace-based assessment (WBA) to direct learning. Unfortunately, WBAs often lack rich narrative feedback and show bias towards Medical Expert aspects of care. Building on research examining interactive assessment approaches, the Queen’s University Internal Medicine residency program introduced a facilitated, team-based assessment initiative (“Feedback Fridays”) in July 2017, aimed at improving holistic assessment of resident performance on the inpatient medicine teaching units. In this study, we aim to explore how Feedback Fridays contributed to formative assessment of Internal Medicine residents within our current model of competency-based training. Method: A total of 53 residents participated in facilitated, biweekly group assessment sessions during the 2017 and 2018 academic year. Each session was a 30-minute facilitated assessment discussion done with one inpatient team, which included medical students, residents, and their supervising attending. Feedback from the discussion was collected, summarized, and documented in narrative form in electronic WBA forms by the program’s assessment officer for the residents. For research purposes, verbatim transcripts of feedback sessions were analyzed thematically. Results: The researchers identified four major themes for feedback: communication, intra- and inter-personal awareness, leadership and teamwork, and learning opportunities. Although feedback related to a broad range of activities, it showed strong emphasis on competencies within the intrinsic CanMEDS roles. Additionally, a clear formative focus in the feedback was another important finding. Conclusions: The introduction of facilitated team-based assessment in the Queen’s Internal Medicine program filled an important gap in WBA by providing learners with detailed feedback across all CanMEDS roles and by providing constructive recommendations for identified areas for improvement

Analysis of factors affecting Canadian medical students’ success in the residency match

Background: In North America, there is limited data to support deliberate application strategies for post-graduate residency training. There is significant interest in determining what factors play a role in Canadian medical graduate (CMG) matching to their first choice discipline and heightened concern about the number of students going unmatched altogether. Methods: We analyzed matching outcomes of CMGs based on seven years (2013-2019) of residency application data (n= 13,499) from the Canadian Residency Matching Service (CaRMS) database using descriptive and binary logistic regression modeling techniques. Results: The sample was 54% female, with 60% between the ages of 26 and 29, and 60% attended medical schools in Ontario. Applicants who received more rankings from residency programs were more likely (OR = 1.185, p < 0.001) to match. Higher research activities (OR = 0.985, p < 0.001) and number of applications submitted (OR = 0.920, p < 0.001) were associated with a reduced likelihood of matching. Number of volunteer activities and self-report publications did not significantly affect matching. Being male (OR = 0.799, p < 0.05) aged <25 (OR = 0.756, p < 0.05), and from Eastern (OR = 0.497, p < 0.01), or Western (OR = 0.450, p < 0.001) Canadian medical schools were predictors of remaining unmatched. Conclusions: This study identified several significant associations of demographic and application factors that affected matching outcomes. The results will help to better inform medical student application strategies and highlight possible biases in the selection process

The Timely Open Communication for Patient Safety Project

Background: Concern is growing over increased numbers of adverse events experienced by patients when admitted to acute care hospitals in Canada due to breakdowns in communication. The purpose of the Timely Open Communication for Patient Safety (TOC) project was to create a culture of patient safety through enhanced interprofessional communication by developing resources for caregivers and patients. Methods and Findings: The research was framed by a mixed methods design that included pre- and post-surveys and focus groups, online educational modules, face-to-face activities, and the development of patient orientation materials. Three clinical sites participated in the study. The findings indicate that supporting healthcare teams to identify strengths, challenges, and future directions of communicating, clarifying roles, functioning, and collaborating, coupled with educational interventions that raise awareness of patient safety,may enhance patient safety. The study was limited by the absence of data regarding the incidence of adverse events during the research period. Conclusion: The data showed improvement in team members' perceptions of interprofessional collaborative practice within the participating Collaborative Learning Units (CLUs). If the CLU model of care is adopted within the healthcare system, the safety of patients/clients may improve

Rétroaction multisources à la suite d’une simulation en réanimation : une étude qualitative

Background: The direct observation and assessment of learners’ resuscitation skills by an attending physician is challenging due to the unpredictable and time-sensitive nature of these events. Multisource feedback (MSF) may address this challenge and improve the quality of assessments provided to learners. We aimed to describe the similarities and differences in the assessment rationale of attending physicians, registered nurses, and resident peers in the context of a simulation-based resuscitation curriculum. Methods: We conducted a qualitative content analysis of narrative MSF of medical residents in their first postgraduate year of training who were participating in a simulation-based resuscitation course at two Canadian institutions. Assessments included an entrustment score and narrative comments from attending physicians, registered nurses, and resident peers in addition to self-assessment. Narrative comments were transcribed and analyzed thematically using a constant comparative method. Results: All 87 residents (100%) participating in the 2017-2018 course provided consent. A total of 223 assessments were included in our analysis. Four themes emerged from the narrative data: 1) Communication, 2) Leadership, 3) Demeanor, and 4) Medical Expert. Relative to other assessor groups, feedback from nurses focused on patient-centred care and communication while attending physicians focused on the medical expert theme. Peer feedback was the most positive. Self-assessments included comments within each of the four themes. Conclusions: In the context of a simulation-based resuscitation curriculum, MSF provided learners with different perspectives in their narrative assessment rationale and may offer a more holistic assessment of resuscitation skills within a competency-based medical education (CBME) program of assessment.Contexte : Le contexte imprévisible et contraignant au niveau du temps lors de l’observation directe et de la rétroaction associée sur les compétences en réanimation des apprenants constituent un défi pour un médecin superviseur. La rétroaction multisources (RMS) peut être un moyen de relever ce défi et d'améliorer la qualité des rétroactions fournies aux apprenants. Nous visons à décrire les similitudes et les différences quant à la démarche évaluative auprès de médecins traitants, d’infirmières cliniciennes et de pairs résidents dans le cadre d'un cours de réanimation offert par simulation. Méthodes : Nous avons réalisé une analyse de contenu à partir des rétroactions narratives offertes aux résidents en première année de formation postdoctorale dans deux universités canadiennes dans le cadre d’un cours de réanimation offert par simulation. En plus de l’auto-évaluation, la rétroaction comportait un score de confiance et des commentaires narratifs de la part de médecins superviseurs, d’infirmières cliniciennes et des pairs. Les commentaires ont été transcrits et analysés par thèmes en appliquant la méthode générale de comparaison constante. Résultats : Un consentement pour participer à l’étude a été obtenu auprès des 87 résidents (100 %) qui ont suivi le cours en 2017-2018. Nous avons analysé un total de 223 rétroactions. Quatre thèmes ont émergé à partir des données narratives soit : 1) la communication, 2) le leadership, 3) le comportement, et 4) l’expertise médicale. Alors que les infirmières ont ciblé leurs commentaires sur les soins centrés sur le patient et la communication, les médecins superviseurs ont les ont ciblés sur l’expertise médicale. Les commentaires des pairs étaient les plus positifs. Les auto-évaluations comportaient des commentaires sur chacun des quatre thèmes. Conclusions : Dans le contexte d’un cours de réanimation offert par simulation, la RMS a permis aux apprenants d’obtenir des évaluations narratives selon différentes perspectives. Permettant ainsi une approche plus holistique de rétroaction sur les habiletés en réanimation dans le cadre d’un programme d’évaluation axé sur les compétences

Participant perceptions of the faculty development Educational Research Series

Interest in the Scholarship of Teaching and Learning (SoTL) is driven in part by the need to provide systematic academic development for faculty anchored in evidence-based practice such as the introduction of quality assurance frameworks. This article reports on a mixed-method evaluation of one institution’s grassroots multidisciplinary faculty development program, called the Educational Research Series, to determine if it met the needs of its faculty, graduate student, and staff participants. Conducted at one mid-sized university in southern Ontario and framed, as was the program design and implementation, by both adult learning theory and constructivism, the evaluation collected data from session exit surveys, attendee interviews, and facilitator focus groups. The data analysis revealed that reasons for participating included increasing levels of understanding, receiving individual support, and learning about colleagues’ research interests. The major strengths of the program included individual learning, resources, facilitator expertise, interactive sessions, and the multidisciplinary focus. The main challenges centered on depth versus breadth of the sessions, time, and educational language and theory. Participants recommended additional resources, communication among facilitators, institutional recognition, and increased depth of content. As a result of this evaluation, an Advanced Educational Research Series is being offered at the institution. This article will inform other institutions wishing to build SoTL as a field within their institutions

Making assessment a team sport: a qualitative study of facilitated group feedback in internal medicine residency

Purpose: Competency-based medical education relies on feedback from workplace-based assessment (WBA) to direct learning. Unfortunately, WBAs often lack rich narrative feedback and show bias towards Medical Expert aspects of care. Building on research examining interactive assessment approaches, the Queen’s University Internal Medicine residency program introduced a facilitated, team-based assessment initiative (“Feedback Fridays”) in July 2017, aimed at improving holistic assessment of resident performance on the inpatient medicine teaching units. In this study, we aim to explore how Feedback Fridays contributed to formative assessment of Internal Medicine residents within our current model of competency-based training. Method: A total of 53 residents participated in facilitated, biweekly group assessment sessions during the 2017 and 2018 academic year. Each session was a 30-minute facilitated assessment discussion done with one inpatient team, which included medical students, residents, and their supervising attending. Feedback from the discussion was collected, summarized, and documented in narrative form in electronic WBA forms by the program’s assessment officer for the residents. For research purposes, verbatim transcripts of feedback sessions were analyzed thematically. Results: The researchers identified four major themes for feedback: communication, intra- and inter-personal awareness, leadership and teamwork, and learning opportunities. Although feedback related to a broad range of activities, it showed strong emphasis on competencies within the intrinsic CanMEDS roles. Additionally, a clear formative focus in the feedback was another important finding. Conclusions: The introduction of facilitated team-based assessment in the Queen’s Internal Medicine program filled an important gap in WBA by providing learners with detailed feedback across all CanMEDS roles and by providing constructive recommendations for identified areas for improvement

Le curriculum caché dans les disciplines médicales : examen de sa portée, de ses incidences et de son contexte

Background: While research suggests that manifestations of the hidden curriculum (HC) phenomenon have the potential to reinforce or undermine the values of an institution, very few studies have comprehensively measured its scope, impact, and the varied clinical teaching and learning contexts within which they occur. We explored the HC and examined the validity of newly developed constructs and determined the influence of context on the HC. Methods: We surveyed medical students (n =182), residents (n =148), and faculty (n = 140) from all disciplines at our institution between 2019 and 2020. Based on prior research and expertise, we measured participants’ experience with the HC including perceptions of respect and disrespect for different medical disciplines, settings in which the HC is experienced, impact of the HC, personal actions, efficacy, and their institutional perceptions. We examined the factor structure, reliability, and validity of the HC constructs using exploratory factor analysis Cronbach’s alpha, regression analysis and Pearson’s correlations. Results: Expert judges (physician faculty and medical learners) confirmed the content validity of the items used and the analysis revealed new HC constructs reflecting negative expressions, positive impacts and expressions, negative impacts, personal actions, and positive institutional perceptions of the HC. Evidence for criterion validity was found for the negative impacts and the personal actions constructs and were significantly associated with the stage of respondents’ career and gender. Support for convergent validity was obtained for HC constructs that were significantly correlated with certain contexts within which the HC occurs. Conclusion: More unique dimensions and contexts of the HC exist than have been previously documented. The findings demonstrate that specific clinical contexts can be targeted to improve negative expressions and impacts of the HC.Contexte : Bien que la recherche suggère que les expressions du curriculum caché (CC) ont le potentiel de renforcer ou de miner les valeurs d’un établissement, très peu d’études ont mesuré de manière exhaustive sa portée, ses effets et les divers contextes d’enseignement et d’apprentissage cliniques dans lesquels elles se produisent. Nous avons exploré le CC, examiné la validité de nouvelles notions et déterminé l’influence du contexte sur le CC. Méthodes : Entre 2019 et 2020, nous avons interrogé des étudiants (n =182), des résidents (n =148) et des membres du corps professoral (n = 140) de notre établissement, toutes disciplines médicales confondues. Sur la base de recherches et d’expertises antérieures, nous avons mesuré l’expérience des participants par rapport au CC, y compris leurs perceptions du respect ou du non-respect des diverses disciplines médicales, les contextes dans lesquels ils ont été confrontés au CC, les effets et l’efficacité du CC, les perceptions de l’établissement et les actions personnelles des participants. Nous avons examiné la structure factorielle, la fiabilité et la validité des notions du CC à l’aide d’une analyse factorielle exploratoire, du coefficient alpha de Cronbach, d’une analyse de régression et des corrélations de Pearson. Résultats : Des juges experts (médecins enseignants et apprenants) ont confirmé la validité du contenu des éléments utilisés et l’analyse a révélé de nouvelles notions du CC reflétant des expressions et des effets négatifs, des expressions et des effets positifs, des actions personnelles et des perceptions positives du CC au sein des établissements. La validité de critère a été démontrée pour les notions d’impacts négatifs et d’actions personnelles et a été associée de manière significative à l’étape de la carrière des répondants et à leur sexe. La validité convergente a été confirmée pour les notions de CC qui étaient significativement corrélées à certains contextes dans lesquels le CC se manifeste. Conclusion : Il existe plus de dimensions et de contextes uniques du CC que ceux qui avaient été documentés par le passé. Nos résultats montrent que des contextes cliniques spécifiques peuvent être ciblés pour améliorer les expressions et les effets négatifs du CC

'Science Rendezvous' - Promoting STEM Education

This session describes the implementation of Science Rendezvous (SR) Kingston, a community outreach event targeted to people of all ages. The Queen’s University Community Outreach Centre (QCOC) situated in the Faculty of Education, provides Science, Technology, Engineering and Mathematics (STEM) education through public outreach. This session details how the QCOC partners with community organizations, museums, artists, the City of Kingston and other university and college faculties to collaboratively provide an annual large-scale, one-day event that showcases hands-on activities, demonstrations, experiments, and scientific research to promote public interest, engagement, and understanding of STEM education and careers. The presenters in this session will share their findings from a two-year study aimed at (a) determining the strengths and challenges of community-university partnerships in the context of SR, (b) providing a set of actionable recommendations for future iterations of SR, and (c) refining outreach models that attempt to clarify the phenomena which occur in the process of adopting and implementing a STEM-based community initiative through collaborative partnerships. The research involved 42 interviews with community partners and an online survey completed by 43% of SR participants over 18 years of age. The findings revealed that first-order barriers (e.g., cost, access, advertisement, and schedules) were easier to address than the second-order barriers that emerged from the analysis. The study found that intrinsic beliefs towards public education events such as public perceptions about STEM careers, faculty beliefs about the role of universities in public outreach, and authentic collaborative community-university partnerships were barriers associated with engagement, organizational structures, and sustainability. The data analysis assisted in understanding the importance of community-university partnerships when engaging in sustainable public outreach events that aim to showcase STEM education through teaching, research and knowledge mobilization. During the session, participants will be provided with resources designed in collaboration with all 25 participating community partners