Validity evidence for summative performance evaluations in postgraduate community pharmacy education

Introduction: Workplace-based assessment of competencies is complex. In this study, the validity of summative performance evaluations (SPEs) made by supervisors in a two-year longitudinal supervisor-trainee relationship was investigated in a postgraduate community pharmacy specialization program in the Netherlands. The construct of competence was based on an adapted version of the 2005 Canadian Medical Education Directive for Specialists (CanMEDS) framework. Methods: The study had a case study design. Both quantitative and qualitative data were collected. The year 1 and year 2 SPE scores of 342 trainees were analyzed using confirmatory factor analysis and generalizability theory. Semi-structured interviews were held with 15 supervisors and the program director to analyze the inferences they made and the impact of SPE scores on the decision-making process. Results: A good model fit was found for the adapted CanMEDS based seven-factor construct. The reliability/precision of the SPE measurements could not be completely isolated, as every trainee was trained in one pharmacy and evaluated by one supervisor. Qualitative analysis revealed that supervisors varied in their standards for scoring competencies. Some supervisors were reluctant to fail trainees. The competency scores had little impact on the high-stakes decision made by the program director. Conclusions: The adapted CanMEDS competency framework provided a valid structure to measure competence. The reliability/precision of SPE measurements could not be established and the SPE measurements provided limited input for the decision-making process. Indications of a shadow assessment system in the pharmacies need further investigation

Learning in and across communities of practice: health professions education students’ learning from boundary crossing

Learning to adapt to new contexts is crucial in health professions education (HPE). Boundaries between and within contexts challenge continuity in students’ learning processes. Little is known about how HPE students can make these “boundary experiences” productive for learning. We investigated how and what nursing students learn from boundary experiences while they are simultaneously growing into a community of practice (CoP). Using a boundary-crossing lens, experiences of discontinuity were identified in pre-placement and post-placement interviews and diary fragments with 14 nursing students during their placement in an academic hospital. We found that students experience discontinuity as a result of different approaches to nursing care and to learning, both between (academic and clinical) settings and within a setting. When students feel safe enough, they can convert boundary experiences into meaningful learning situations, such as critical discussions with staff. Successfully overcoming boundary experiences improves students’ understanding of healthcare and professional development and helps them to develop a personal approach to learning. Students critically address boundary experiences when they are motivated to learn and when they perceive a violation of ethical standards but not when they are concerned that it will affect their assessment. Objects designed to bridge theory and practice can generate additional barriers. This study adds to the HPE literature by demonstrating the learning potential of boundaries and to the broader literature by showing how responses to boundary experiences are intertwined with the process of growing into a CoP. The findings can be used to design future boundary objects

How do undergraduate nursing students learn in the hospital setting?: A scoping review of conceptualisations, operationalisations and learning activities

Objectives Although clinical learning is pivotal for nursing education, the learning process itself and the terminology to address this topic remain underexposed in the literature. This study aimed to examine how concepts equivalent to 'learning in practice' are used and operationalised and which learning activities are reported in the nursing education literature. The final aim was to propose terminology for future studies. Design The scoping framework proposed by Arksey and O'Malley was used to answer the research questions and address gaps in the literature. Two systematic searches were conducted in PubMed, EBSCO/ERIC and EBSCO/CINAHL between May and September 2018: first, to identify concepts equivalent to 'learning in practice' and, second, to find studies operationalising these concepts. Eligible articles were studies that examined the regular learning of undergraduate nursing students in the hospital setting. Conceptualisations, theoretical frameworks and operationalisations were mapped descriptively. Results relating to how students learn were synthesised using thematic analysis. Quality assessment was performed using the Critical Appraisal Skills Programme checklist. Results From 9360 abstracts, 17 articles were included. Five studies adopted a general, yet not explained, synonym for learning in practice, and the other approaches focused on the social, unplanned or active nature of learning. All studies used a qualitative approach. The small number of studies and medium study quality hampered a thorough comparison of concepts. The synthesis of results revealed five types of learning activities, acknowledged by an expert panel, in which autonomy, interactions and cognitive processing were central themes. Conclusions Both theoretical approaches and learning activities of the current body of research fit into experiential learning theories, which can be used to guide and improve future studies. Gaps in the literature include formal and informal components of learning, the relation between learning and learning outcomes and the interplay between behaviour and cognitive processing

Reliability of clinical oral examinations re-examined

Many medical schools still use oral examinations for the evaluation of clinical competence of students in their clerkship, although it has been proven that orals have poor reliability. This study investigates the feasibility and reliability of multiple oral examinations. Students in the last week of their Internal Medicine clerkship in an outpatient clinic were given several patient-based oral examinations. The student's performance was rated on a list of items reflecting clinical competence. A global judgement of the student's performance was also given. The results indicate that it is possible to increase the number of orals and the number of examiners in the day-to-day practice of an outpatient clinic moderately. The reliability when using a number of orals is better than the reliability of the common single oral examination. The reliability using global judgements appeared to be better than the reliability of averaged item scores

Restricted origin and distribution of projections from the lateral to the medial septal complex in rat and guinea pig

The origin and distribution of projections from the lateral to the medial septal complex were studied at the light- and electron-microscopical level in the rat and the guinea pig, with the use of sensitive anterograde tracing techniques. Injections in the lateral septal complex resulted in only weak to moderate labeling in the medial septal nucleus. In contrast, injections in all but the ventral subdivision of the lateral septal complex labeled restricted terminal arborizations in the angular portion of the vertical diagonal band. Analyses at the electron-microscopical level indicated that these fibers form both symmetrical and asymmetrical synapses with dendrites and somata, but not with axons. These findings are discussed in the light of a presumed functional circuit from the hippocampal formation, via the lateral septal complex, to cells in the medial septal complex that originate projections to the hippocampal formation

The clinical and educational outcomes of an inter-professional student-led medication review team, a pilot study

Aims: The involvement of an inter-professional healthcare student team in the review of medications used by geriatric patients could not only provide patients with optimized therapy but also provide students with a valuable inter-professional learning experience. We describe and evaluate the clinical and learning outcomes of an inter-professional student-run mediation review program (ISP). Subject and method: A variable team consisting of students in medicine, pharmacy, master advanced nursing practice, and master physician assistant reviewed the medication lists of patients attending a specialized geriatric outpatient clinic. Results: During 32 outpatient visits, 188 medications were reviewed. The students identified 14 medication-related problems, of which 4 were not recognized by healthcare professionals. The ISP team advised 95 medication changes, of which 68 (71.6%) were directly implemented. Students evaluated this pilot program positively and considered it educational (median score 4 out of 5) and thought it would contribute to their future inter-professional relationships. Conclusion: An inter-professional team of healthcare students is an innovative healthcare improvement for (academic) hospitals to increase medication safety. Most formulated advices were directly incorporated in daily practice and could prevent future medication-related harm. The ISP also offers students a first opportunity to work in an inter-professional manner and get insight into the perspectives and qualities of their future colleagues

Learning in and across communities of practice: health professions education students’ learning from boundary crossing

Learning to adapt to new contexts is crucial in health professions education (HPE). Boundaries between and within contexts challenge continuity in students’ learning processes. Little is known about how HPE students can make these “boundary experiences” productive for learning. We investigated how and what nursing students learn from boundary experiences while they are simultaneously growing into a community of practice (CoP). Using a boundary-crossing lens, experiences of discontinuity were identified in pre-placement and post-placement interviews and diary fragments with 14 nursing students during their placement in an academic hospital. We found that students experience discontinuity as a result of different approaches to nursing care and to learning, both between (academic and clinical) settings and within a setting. When students feel safe enough, they can convert boundary experiences into meaningful learning situations, such as critical discussions with staff. Successfully overcoming boundary experiences improves students’ understanding of healthcare and professional development and helps them to develop a personal approach to learning. Students critically address boundary experiences when they are motivated to learn and when they perceive a violation of ethical standards but not when they are concerned that it will affect their assessment. Objects designed to bridge theory and practice can generate additional barriers. This study adds to the HPE literature by demonstrating the learning potential of boundaries and to the broader literature by showing how responses to boundary experiences are intertwined with the process of growing into a CoP. The findings can be used to design future boundary objects

Validity evidence for summative performance evaluations in postgraduate community pharmacy education

Introduction: Workplace-based assessment of competencies is complex. In this study, the validity of summative performance evaluations (SPEs) made by supervisors in a two-year longitudinal supervisor-trainee relationship was investigated in a postgraduate community pharmacy specialization program in the Netherlands. The construct of competence was based on an adapted version of the 2005 Canadian Medical Education Directive for Specialists (CanMEDS) framework. Methods: The study had a case study design. Both quantitative and qualitative data were collected. The year 1 and year 2 SPE scores of 342 trainees were analyzed using confirmatory factor analysis and generalizability theory. Semi-structured interviews were held with 15 supervisors and the program director to analyze the inferences they made and the impact of SPE scores on the decision-making process. Results: A good model fit was found for the adapted CanMEDS based seven-factor construct. The reliability/precision of the SPE measurements could not be completely isolated, as every trainee was trained in one pharmacy and evaluated by one supervisor. Qualitative analysis revealed that supervisors varied in their standards for scoring competencies. Some supervisors were reluctant to fail trainees. The competency scores had little impact on the high-stakes decision made by the program director. Conclusions: The adapted CanMEDS competency framework provided a valid structure to measure competence. The reliability/precision of SPE measurements could not be established and the SPE measurements provided limited input for the decision-making process. Indications of a shadow assessment system in the pharmacies need further investigation