Targeting Simulation-Based Assessment for the Pediatric Milestones: A Survey of Simulation Experts and Program Directors.

OBJECTIVE: To determine which of the 21 general pediatrics milestone subcompetencies are most difficult to assess using traditional methodologies and which are best suited to simulation-based assessment. METHODS: We surveyed 2 samples: pediatric simulation experts and pediatric program directors. Respondents were asked about current use of simulation for assessment and to select 5 of the 21 pediatric subcompetencies most difficult to assess using traditional methods and the 5 best suited to simulation-based assessment. Spearman rank correlation was used to determine a correlation between how the 2 samples ranked the subcompetencies. RESULTS: Forty-eight percent (29 of 60) simulation experts and 20% (115 of 571) program directors completed the survey. Few respondents reported using simulation for summative assessment. There are clear differences across the pediatric subcompetencies in perceived difficulty of assessment and suitability to simulation-based assessment. The 3 most difficult to assess subcompetencies were “recognize ambiguity,” “demonstrate emotional insight,” and “identify one’s own strengths and deficiencies.” The subcompetencies most suitable to assessment using simulation were “interprofessional teamwork,” “clinical decision making,” and “effective communication.” Program directors and simulation experts had high agreement for both questions: difficult to assess (rho = 0.76, P \u3c .001) and suitable to simulation-based assessment (rho = 0.94, P \u3c .001). CONCLUSIONS: Several general pediatrics milestone subcompetencies were identified by pediatric simulation experts and pediatric program directors as difficult to assess using current methodologies and as amenable to simulation-based assessment. The pediatric simulation community should target development of simulation-based assessment tools to these areas

Key Factors in Clinical Competency Committee Members' Decisions Regarding Residents' Readiness to Serve as Supervisors: A National Study

Entrustment has become a popular assessment framework in recent years. Most research in this area has focused on how frontline assessors determine when a learner can be entrusted. However, less work has focused on how these entrustment decisions are made. The authors sought to understand the key factors that pediatric residency program clinical competency committee (CCC) members consider when recommending residents to a supervisory role. CCC members at 14 pediatric residency programs recommended residents to one of five progressive supervisory roles (from not serving as a supervisory resident to serving as a supervisory resident in all settings). They then responded to a free-text prompt, describing the key factors that led them to that decision. The authors analyzed these responses, by role recommendation, using a thematic analysis. Of the 155 CCC members at the participating programs, 84 completed 769 supervisory role recommendations during the 2015-2016 academic year. Four themes emerged from the thematic analysis: (1) Determining supervisory ability follows from demonstrated trustworthiness; (2) demonstrated performance matters, but so does experience; (3) ability to lead a team is considered; and (4) contextual considerations external to the resident are at play. CCC members considered resident and environmental factors in their summative entrustment decision making. The interplay between these factors should be considered as CCC processes are optimized and studied further.12 month embargo; published online: 25 September 2018This item from the UA Faculty Publications collection is made available by the University of Arizona with support from the University of Arizona Libraries. If you have questions, please contact us at [email protected]

I-PASS Mentored Implementation Handoff Curriculum: Champion Training Materials

IntroductionThe I-PASS Handoff Program is a comprehensive handoff curriculum that has been shown to decrease rates of medical errors and adverse events during patient handoffs. I-PASS champions are a critical part of the implementation and sustainment of this curriculum, and therefore, a rigorous program to support their training is necessary.MethodsThe I-PASS Handoff champion training materials were created for the original I-PASS Study and adapted for the Society of Hospital Medicine (SHM) I-PASS Mentored Implementation Program. The adapted materials embrace a flipped classroom approach and adult learning theory. The training includes an overview of I-PASS handoff techniques, an opportunity to practice evaluating handoffs with the I-PASS observation tools using a handoff video vignette, and other key implementation principles.ResultsAs part of the SHM I-PASS Mentored Implementation Program, 366 champions were trained at 32 sites across North America and participated in a total of 3,491 handoff observations. A total of 346 champions completed the I-PASS Champion Workshop evaluation form at the end of their training (response rate: 94.5%). After receiving the training, over 90% agreed/strongly agreed that it provided them with knowledge or skills critical to their patient care activities and that they were able to distinguish the difference between high- and poor-quality handoffs, competently use the I-PASS handoff assessment tools, and articulate the importance of handoff observations.ConclusionThe I-PASS champion training materials were rated highly by those trained and are an integral part of a successful I-PASS Handoff Program implementation

I-PASS Mentored Implementation Handoff Curriculum: Frontline Provider Training Materials

IntroductionThe I-PASS Handoff Program is a comprehensive handoff curriculum that has been shown to decrease rates of medical errors and adverse events during patient handoffs. Frontline providers are the key individuals participating in handoffs of patient care. It is important they receive robust handoff training.MethodsThe I-PASS Mentored Implementation Handoff Curriculum frontline provider training materials were created as part of the original I-PASS Study and adapted for the Society of Hospital Medicine (SHM) I-PASS Mentored Implementation Program. The adapted materials embrace a flipped classroom approach with an emphasis on adult learning theory principles. The training includes an overview of I-PASS handoff techniques, TeamSTEPPS team communication strategies, verbal handoff simulation scenarios, and a printed handoff document exercise.ResultsAs part of the SHM I-PASS Mentored Implementation Program, 2,735 frontline providers were trained at 32 study sites (16 adult and 16 pediatric) across North America. At the end of their training, 1,762 frontline providers completed the workshop evaluation form (64% response rate). After receiving the training, over 90% agreed/strongly agreed that they were able to distinguish a good- from a poor-quality handoff, articulate the elements of the I-PASS mnemonic, construct a high-quality patient summary, advocate for an appropriate environment for handoffs, and participate in handoff simulations. Universally, the training provided them with knowledge and skills relevant to their patient care activities.DiscussionThe I-PASS frontline training materials were rated highly by those trained and are an integral part of a successful I-PASS Handoff Program implementation

I-PASS Mentored Implementation Handoff Curriculum: Implementation Guide and Resources

IntroductionCommunication failures during shift-to-shift handoffs of patient care have been identified as a leading cause of adverse events in health care institutions. The I-PASS Handoff Program is a comprehensive handoff program that has been shown to decrease rates of medical errors and adverse events. As part of the spread and adaptation of this program, a comprehensive implementation guide was created to assist individuals in the implementation process.MethodsThe I-PASS Mentored Implementation Guide grew out of materials created for the original I-PASS Study, Society of Hospital Medicine (SHM) mentored implementation programs, and the experience of members of the I-PASS Study Group. The guide provides a comprehensive framework of all elements required to implement the large-scale I-PASS Handoff Program and contains detailed information on generating institutional support, training activities, a campaign, measuring impact, and sustaining the program.ResultsThirty-two sites across North America utilized the guide as part of the SHM program. The guide served as a main reference for 477 hours of mentoring phone calls between site leads and their mentors. Postprogram surveys from wave 2 sites revealed that 85% (N = 34) of respondents felt the quality of the guide was very good/excellent. Site leads noted that they referenced the guide most often during the early part of the program and that they referenced the sections on the curriculum and handoff observations most often.DiscussionThe I-PASS Mentored Implementation Guide is an essential resource for those looking to implement the large-scale I-PASS Handoff Program at their institution

Families as Partners in Hospital Error and Adverse Event Surveillance

ImportanceMedical errors and adverse events (AEs) are common among hospitalized children. While clinician reports are the foundation of operational hospital safety surveillance and a key component of multifaceted research surveillance, patient and family reports are not routinely gathered. We hypothesized that a novel family-reporting mechanism would improve incident detection.ObjectiveTo compare error and AE rates (1) gathered systematically with vs without family reporting, (2) reported by families vs clinicians, and (3) reported by families vs hospital incident reports.Design, setting, and participantsWe conducted a prospective cohort study including the parents/caregivers of 989 hospitalized patients 17 years and younger (total 3902 patient-days) and their clinicians from December 2014 to July 2015 in 4 US pediatric centers. Clinician abstractors identified potential errors and AEs by reviewing medical records, hospital incident reports, and clinician reports as well as weekly and discharge Family Safety Interviews (FSIs). Two physicians reviewed and independently categorized all incidents, rating severity and preventability (agreement, 68%-90%; κ, 0.50-0.68). Discordant categorizations were reconciled. Rates were generated using Poisson regression estimated via generalized estimating equations to account for repeated measures on the same patient.Main outcomes and measuresError and AE rates.ResultsOverall, 746 parents/caregivers consented for the study. Of these, 717 completed FSIs. Their median (interquartile range) age was 32.5 (26-40) years; 380 (53.0%) were nonwhite, 566 (78.9%) were female, 603 (84.1%) were English speaking, and 380 (53.0%) had attended college. Of 717 parents/caregivers completing FSIs, 185 (25.8%) reported a total of 255 incidents, which were classified as 132 safety concerns (51.8%), 102 nonsafety-related quality concerns (40.0%), and 21 other concerns (8.2%). These included 22 preventable AEs (8.6%), 17 nonharmful medical errors (6.7%), and 11 nonpreventable AEs (4.3%) on the study unit. In total, 179 errors and 113 AEs were identified from all sources. Family reports included 8 otherwise unidentified AEs, including 7 preventable AEs. Error rates with family reporting (45.9 per 1000 patient-days) were 1.2-fold (95% CI, 1.1-1.2) higher than rates without family reporting (39.7 per 1000 patient-days). Adverse event rates with family reporting (28.7 per 1000 patient-days) were 1.1-fold (95% CI, 1.0-1.2; P = .006) higher than rates without (26.1 per 1000 patient-days). Families and clinicians reported similar rates of errors (10.0 vs 12.8 per 1000 patient-days; relative rate, 0.8; 95% CI, .5-1.2) and AEs (8.5 vs 6.2 per 1000 patient-days; relative rate, 1.4; 95% CI, 0.8-2.2). Family-reported error rates were 5.0-fold (95% CI, 1.9-13.0) higher and AE rates 2.9-fold (95% CI, 1.2-6.7) higher than hospital incident report rates.Conclusions and relevanceFamilies provide unique information about hospital safety and should be included in hospital safety surveillance in order to facilitate better design and assessment of interventions to improve safety

Families as Partners in Hospital Error and Adverse Event Surveillance

ImportanceMedical errors and adverse events (AEs) are common among hospitalized children. While clinician reports are the foundation of operational hospital safety surveillance and a key component of multifaceted research surveillance, patient and family reports are not routinely gathered. We hypothesized that a novel family-reporting mechanism would improve incident detection.ObjectiveTo compare error and AE rates (1) gathered systematically with vs without family reporting, (2) reported by families vs clinicians, and (3) reported by families vs hospital incident reports.Design, setting, and participantsWe conducted a prospective cohort study including the parents/caregivers of 989 hospitalized patients 17 years and younger (total 3902 patient-days) and their clinicians from December 2014 to July 2015 in 4 US pediatric centers. Clinician abstractors identified potential errors and AEs by reviewing medical records, hospital incident reports, and clinician reports as well as weekly and discharge Family Safety Interviews (FSIs). Two physicians reviewed and independently categorized all incidents, rating severity and preventability (agreement, 68%-90%; κ, 0.50-0.68). Discordant categorizations were reconciled. Rates were generated using Poisson regression estimated via generalized estimating equations to account for repeated measures on the same patient.Main outcomes and measuresError and AE rates.ResultsOverall, 746 parents/caregivers consented for the study. Of these, 717 completed FSIs. Their median (interquartile range) age was 32.5 (26-40) years; 380 (53.0%) were nonwhite, 566 (78.9%) were female, 603 (84.1%) were English speaking, and 380 (53.0%) had attended college. Of 717 parents/caregivers completing FSIs, 185 (25.8%) reported a total of 255 incidents, which were classified as 132 safety concerns (51.8%), 102 nonsafety-related quality concerns (40.0%), and 21 other concerns (8.2%). These included 22 preventable AEs (8.6%), 17 nonharmful medical errors (6.7%), and 11 nonpreventable AEs (4.3%) on the study unit. In total, 179 errors and 113 AEs were identified from all sources. Family reports included 8 otherwise unidentified AEs, including 7 preventable AEs. Error rates with family reporting (45.9 per 1000 patient-days) were 1.2-fold (95% CI, 1.1-1.2) higher than rates without family reporting (39.7 per 1000 patient-days). Adverse event rates with family reporting (28.7 per 1000 patient-days) were 1.1-fold (95% CI, 1.0-1.2; P = .006) higher than rates without (26.1 per 1000 patient-days). Families and clinicians reported similar rates of errors (10.0 vs 12.8 per 1000 patient-days; relative rate, 0.8; 95% CI, .5-1.2) and AEs (8.5 vs 6.2 per 1000 patient-days; relative rate, 1.4; 95% CI, 0.8-2.2). Family-reported error rates were 5.0-fold (95% CI, 1.9-13.0) higher and AE rates 2.9-fold (95% CI, 1.2-6.7) higher than hospital incident report rates.Conclusions and relevanceFamilies provide unique information about hospital safety and should be included in hospital safety surveillance in order to facilitate better design and assessment of interventions to improve safety