Harajda, Kelli N, MD

Rooney, Kris, MD

English

Lehigh Valley Health Network: LVHN Scholarly Works

Lehigh Valley Health NetworkLVHN Scholarly WorksDepartment of PediatricsUsing Entrustable Professional Activities toImprove Pediatric Residency EvaluationsKelli N. Harajda MDLehigh Valley Health Network, Kelli_N.Harajda@lvhn.orgKris Rooney MDLehigh Valley Health Network, kris.rooney@lvhn.orgFollow this and additional works at: https://scholarlyworks.lvhn.org/pediatricsPart of the Pediatrics CommonsThis Poster is brought to you for free and open access by LVHN Scholarly Works. It has been accepted for inclusion in LVHN Scholarly Works by anauthorized administrator. For more information, please contact LibraryServices@lvhn.org.Published In/Presented AtHarajda, K. N., Rooney, K. (2019, June). Using Entrustable Professional Activities to Improve Pediatric Residency Evaluations. PosterPresented at: The LVHN Pediatric Residency Scholarly Activity Presentations. Allentown. PA.LVHN.org© 2019 Lehigh Valley Health NetworkUsing Entrustable Professional Activities to Improve Pediatric Residency Evaluations Kelli Harajda, MD and Kris Rooney, MDLehigh Valley Health Network, Allentown, Pa.BACKGROUND AND OBJECTIVESThe ACGME has recommended the use of Entrustable Professional Activities (EPAs) as a means of setting tangible goals and objectives as well as meaningful, objective assessments of resident competency. EPAs are well defined tasks that are entrusted to learners once demonstrating that they possess the competencies to perform these tasks unsupervised. We hypothesized that the use of EPA-based goals and objectives would allow for a more holistic and objective evaluation of resident performance than the current methods, using ACGME competencies and milestones. We expect that utilization of an EPA-based format will allow for  a more timely completion of evaluations by faculty and improve the clarity, feasibility, efficiency, and meaningfulness  of resident evaluations.METHODSA survey was created via Survey Monkey for both faculty and residents regarding the current state of evaluations. Faculty were asked about feasibility, efficiency, and clarity of the current evaluations as well as opinions on barriers to completion and return. Residents were asked about meaningfulness, perceived utility of feedback, and timeliness of return. Both parties were asked about their opinions on moving toward the use of EPA-based goals, objectives, and assessments.Goals and objectives as well as faculty evaluations of residents were re-written in an EPA-based format to be piloted in three core pediatric rotations: Inpatient Pediatrics, Newborn Nursery, and NICU. These pilot evaluations were distributed to faculty through a web-based application, New Innovations, just as the prior evaluations had been. After a period of 3 rotations, post-implementation data was collected through similar survey questions via Survey Monkey, and results were compared.DISCUSSIONAs demonstrated by the survey results, the new EPA-based evaluation format was viewed more favorably than the old milestone-based format by a large majority of respondents.  Additionally, as hypothesized, the new format improved the clarity, efficiency, meaningfulness, and feasibility of the evaluations. The majority of faculty respondents felt that the new evaluations were  of appropriate length which was the most commonly cited barrier  to completion. We conclude that transforming resident evaluations through use of EPAs has been a successful quality improvement process for both faculty and residents, while simultaneously meeting the ACGME’s recommendations for EPA use. Though our work on this project has been of a small scale, it is likely generalizable to other pediatric residency programs or programs within other specialties. We plan  to expand upon this project by re-writing the goals, objectives,  and evaluations for all of the rotations in our residency program.LIMITATIONSOur sample of survey respondents is of a relatively small size thus limiting the power of this study. However, our residency program size is small. 94% of residents completed the post-survey. We did  not preclude new respondents from completing the survey post-format change, thus it is possible that true comparison between surveys may be limited. Lastly, prior to changing the evaluation format, faculty members received new reminders through our  web-based application, New Innovations, also with the effort of improving completion and return times. This change may have confounded our results to some degree. SAMPLE EPA-BASED EVALUATION ITEMRESULTSFACULTY RESPONDENTS RESIDENT RESPONDENTS*from Newborn Nursery EvaluationAmong faculty respondents, 54% felt that the milestone-based evaluations had too many questions, while 10% felt the EPA-based evaluations had too many questions. 90% of post-survey respondents felt that the length of the EPA-based evaluations was appropriate. Approximately 15% of faculty felt that they never completed evaluations within the 2-week goal period and 0% felt that they always do. Conversely, after evaluations were changed to the EPA-based format, 0% of faculty stated they never return evaluations within 2 weeks, while 20% stated that they usually do, and 10% always do. 12 out of 13 faculty agreed that applying milestone based items to their experience with residents was very difficult when thinking about barriers to completing the milestone-based evaluations. Other common barriers included question length and the time it took to read items. Prior to any intervention, 75% of faculty members who completed the question and 63% of residents were in favor of moving toward the EPA-based format, while the remainder of respondents were neutral (none opposed). After implementation, 90% of faculty members and 100% of residents preferred the new format. With regard to resident respondents, a large majority (72%) felt that they rarely or sometimes received evaluations within 2 weeks of completing a rotation, prior to making changes to the format. After changes were made, 40% felt as though they frequently received evaluations within 2 weeks, and 13% always do.The majority of faculty respondents indicated that they evaluate approximately 6-10 residents per academic year. All of the post-implementation faculty respondents had used the new EPA-based evaluation at least 1 time. Among resident respondents, 8 completed inpatient pediatrics, 3 completed NICU, and 4 completed newborn nursery rotations post-implementation of the new evaluations. 0246810121416PEDIATRIC HOSPITALISTSNEONATOLOGISTS NURSERY PROVIDERSTOTALDemographics■ Pre■ Post75131042320246810121416PGY1 PGY2 PGY3 TOTALDemographics4612155 543■ Pre■ Post012345Clarity of LanguageEfficiency Meaningfulness Feasibility■ Pre■ PostAll categories with p<0.053.314.63.544.42.694.442.69Survey RespondentsWeighted averages of 5 point likert scale items012345Survey RespondentsWeighted averages of 5 point likert scale items■ Pre■ Post3.183.793.644.27Likelihood of behavior change p<0.05Meaningfulness Likelihood of impacting clinical  behavior change

Using Entrustable Professional Activities to Improve Pediatric Residency Evaluations

https://scholarlyworks.lvhn.org/cgi/viewcontent.cgi?article=1438&context=pediatrics

Using Entrustable Professional Activities to Improve Pediatric Residency Evaluations

Abstract

Similar works

Full text

Available Versions

Lehigh Valley Health Network: LVHN Scholarly Works