Association Between Workplace-Based Assessments and Self-Assessment of Entrustment

Purpose: Optimal implementation of Entrustable Professional Activities (EPA)-based curriculum requires direct observation using workplace-based assessments (WBA) coupled with a longitudinal coaching relationship between learners and faculty. Students should be engaged and considered an active participant in the entrustment decision. As part of our engagement with the AAMC Core EPA pilot, we implemented both a WBA using the Otawa Clinic Assessment Tool (OCAT) and a coaching program for our medical students in 2018-2019. We measured the association between WBAs collected during the clinical clerkships and student self-assessment. Coaches were selected from currently designated small group advisors and trained on the EPAs and the coaching conversation. Coaches and students were not assigned based upon specialty interest. Each coach worked with 8-10 students

Echocardiography curriculum development for physician assistants using entrustable professional activities

BACKGROUND: With the projected increase of cardiovascular disease in the aging population, a higher demand for echocardiography use is predicted. However, there is a shortage in the supply of cardiologists, to the point that a 2009 American College of Cardiology survey report called it a "cardiology workforce crisis". The report also recommends a more aggressive use of PAs and NPs as one of the solutions to fill the shortage. Currently, echocardiography is not routinely included in the scope of practice for PAs in cardiology. While PAs attain strong basic science knowledge and clinical training experience in PA school, they typically do not receive additional formal postgraduate training. PAs have limited training opportunities to train in echocardiography and receive certification of recognition, but a formally standardized training program and certifying examination geared specifically for PAs are yet to be developed. This study seeks to develop a pilot curriculum in training echocardiography which can be standardized for utilization across various regions and medical subspecialties. The curriculum draws on the concept of Entrustable Professional Activities (EPA), which is being actively used in graduate medical education. HYPOTHESIS: After participating in the proposed pilot curriculum which involves online didactic learning and supervised hands-on clinical training, trained PAs will be able to reach proficiency in echocardiography operation and interpretation at level 4 supervision according to the EPA guidelines. METHODS: This study proposes a pilot curriculum with framework based on the EPA titled “performing and interpreting echocardiography” by PAs. The curriculum involves didactic and clinical training in echocardiography, with the goal to achieve mastery of level 4 supervision (minimal supervision). 2 subjects will be recruited from a teaching medical institution in the Greater Boston area with an IAC accredited echocardiography laboratory. After the 12-month training, participants will take ASCeXAM/ReASCE Online Practice Exam Simulation offered by the ASE. Upon 1) achievement of individualized EPAs as assessed by supervisor, and 2) simulation exam score of >80%, participants will earn a STAR in echocardiography. CONCLUSION: The study is the first step to establishing an effective training curriculum that will eventually be a basis for creating a certifying exam in echocardiography, designed specifically for PAs. As this study merely suggests a new curriculum, future studies should focus on identifying strengths and weaknesses of the curriculum after implementation and expansion to multiple sites, and gather data to use for continual improvement of the training curriculum

Exploring the introduction of entrustment rating scales in an existing objective structured clinical examination

Background: The concept of EPAs is increasingly applied to assess trainees’ workplace performance by means of entrustment ratings. OSCEs assess performance in a simulated setting, and it is unclear whether entrustment ratings can be integrated into these exams. This study explores the introduction of an entrustment rating scale into an existing OSCE. Methods: A 6-point entrustment scale was added to the standard ratings in an OSCE administered prior to students’ final clerkship year in an undergraduate medical programme. Standard OSCE ratings assess clinical and communication skills. Assessors (n = 54) rated students’ performance (n = 227) on a diverse set of clinical tasks and evaluated the addition of entrustment scales to OSCEs. Descriptive and inferential statistics were calculated for analyses. Results: Student performance varied across the stations, as reflected in both the standard OSCE ratings and the added entrustment ratings. Students received generally high standard OSCE ratings, whereas entrustment ratings were more widely distributed. All students passed the OSCE, and only a small proportion of students did not reach the expected pass threshold of 60% on the standard ratings in the single stations. The proportion of students who did not reach the expected entrustment level in the respective stations was noticeably higher. Both the clinical and communication skill ratings were related to the entrustment rating in most OSCE stations. A majority of the assessors positively evaluated the addition of entrustment ratings into the OSCE. Discussion: The findings provide an empirical basis to broaden our understanding of the potential use of entrustment ratings in existing OSCEs. They provide directions for future, more specific studies. The ratings might be used for formative feedback on students’ readiness for workplace practice

Implementation of a Workplace-Based Assessment to Measure Performance of the Core Entrustable Professional Activities in the Pediatric Clerkship

Background: In 2013, the AAMC convened a panel of medical education experts to delineate the 13 Core Entrustable Professional Activities (Core EPAs) medical school graduates should be able to perform without direct supervision by day one of their residency. 1 Assessment of these EPAs requires direct observation to render entrustment decisions. 2 As part of our engagement in the AAMC Core EPA pilot,3 we created a workplace-based assessment (WBA) system to assist in measurement of student performance for the Core EPAs at the Virginia Commonwealth University School of Medicine. For the Pediatrics clerkship, we identified 6 Core EPAs to assess for all students during the 2018-2019 academic year: -Core EPA 1 (history and physical exam) -Core EPA 2 (prioritize a differential diagnosis) -Core EPA 3 (recommend and interpret common diagnostic and screening tests) -Core EPA 5 (document clinical encounter in patient record) -Core EPA 6 (provide an oral presentation of a clinical encounter) -Core EPA 9 (collaborate as a team member of an interprofessional team)

Knowing Their Limits: Assessing the Discernment of Pre-Clinical Medical Students

Introduction: Discernment, or awareness of one’s limitations, is one of the qualities fundamental to entrustment (ten Cate et al., 2015). The AAMC Entrustable Professional Activities (EPA) pilot project lists discernment as an expectation of medical students entering residency (AAMC, 2014). Our institution, Virginia Commonwealth University School of Medicine (VCU SOM), was selected to participate in the EPA pilot. Prior to implementing a pilot curriculum on discernment, our institution gathered data on the current practice of discernment among our pre-clinical medical students. The Practice of Clinical Medicine (PCM) at VCU SOM is a pre-clinical longitudinal course which teaches the core skills of doctoring. Students are assessed on their ability to integrate these skills at the end of each semester with an Objective Structure Clinical Exam (OSCE). Starting with the Class of 2020, a question assessing discernment was incorporated into each OSCE

The amount of supervision trainees receive during neonatal resuscitation is variable and often dependent on subjective criteria

Measure variation in delivery room supervision provided by neonatologists using hypothetical scenarios and determine the factors used to guide entrustment decisions. Study Design A survey was distributed to members of the American Academy of Pediatrics Section on Perinatal Pediatrics. Neonatologists were presented with various newborn resuscitation scenarios and asked to choose the level of supervision they thought appropriate and grade factors on their importance in making entrustment decisions. Results There was significant variation in supervision neonatologists deemed necessary for most scenarios (deviation from the mode 0.36–0.69). Post-graduate year of training and environmental circumstances influence the amount of autonomy neonatologists grant trainees. Few neonatologists have objective assessment of a trainees’ competence in neonatal resuscitation available to them and most never document how the trainee performed. Conclusion Delivery room supervision is often determined by subjective evaluation of trainees’ competence and may not provide a level of supervision congruent with their capability

Pharmacy Leadership and Management: A new high fidelity simulation to prepare students for their future practise

Objective: To design and deliver a synoptic experiential learning experience drawing on leadership and management skills coupled with clinical problem solving in a high fidelity simulation. Our vision is to produce world leading pharmacists by providing them with diverse opportunities to demonstrate and expand their knowledge, skills and behaviours and prepare them for the changing healthcare landscape of the future. Design: Pharmacy Leadership and Management (PLM) is based on the successful GIMMICS consortia blueprint. Development was informed by stakeholder events with local and national health commissioners, professionals, experts, patients and academic staff. Teams of six final-year students run their own primary care-based pharmacy business competing against four others over four blocks of three days through the year. Simulated patients and academics deliver approximately 180 acute scenarios over the module to each team, comprising face-to-face, telephone and email queries. Teams also have longer term challenges including development new services and tendering for contracts. The curriculum covers all areas of practice including communication with patients and professionals, business and people development, medicines supply, clinical practice, governance and patient safety and health promotion and public health. Assessment: Assessment uses a multi-format approach of OSCE for skills, reflective portfolio for longer term and attitudinal attributes and an online assessment for knowledge based elements. Conclusion: In this paper, we will present the development of this educational experience and describe our implementation and operationalisation of this highly complex module

The Stakeholder Experience of a large scale final year undergraduate social community research project

Objective: In 2014 The School of Pharmacy at the University of Nottingham needed to deliver individual research methods supervision by a small number of academic staff to a large number of final year students. There are limited opportunities for students to gain patient facing experience on this course. The learning initiative was designed to meet these needs. Design: Dissertation students were offered a unique opportunity to participate in a large scale community pharmacy research project. Eighty-two students collected standardised data from patients across 36 pharmacies in the Greater Nottingham area. Local data collection supervision was provided by the local community pharmacists at the data collection sites. Academic supervision was provided to students using a hub and spoke model with ‘hub’ supervision provided by two members of staff offering broad methodological support to the cohort. This was further supported by local supervisors providing individualised ‘spoke’ support to students. Students were able to examine and report on their local results. The data generated overall provides a mass dataset for further examination by academics. An independent evaluation of stakeholder experience was undertaken. Assessment: Students were assessed by a poster presentation and written report summarising one segment of local analysis. Conclusion: Academics saw the hub and spoke model of supervision as innovative and a positive and efficient use of their time. All participating stakeholders feel that students benefit from the timely development of their transferable skills for their professional career; skills cited as requirements for pharmacy education by the GPhC (2011)