Validity evidence and reliability of a simulated patient feedback instrument

Contains fulltext : 110154.pdf (publisher's version ) (Open Access)BACKGROUND: In the training of healthcare professionals, one of the advantages of communication training with simulated patients (SPs) is the SP's ability to provide direct feedback to students after a simulated clinical encounter. The quality of SP feedback must be monitored, especially because it is well known that feedback can have a profound effect on student performance. Due to the current lack of valid and reliable instruments to assess the quality of SP feedback, our study examined the validity and reliability of one potential instrument, the 'modified Quality of Simulated Patient Feedback Form' (mQSF). METHODS: Content validity of the mQSF was assessed by inviting experts in the area of simulated clinical encounters to rate the importance of the mQSF items. Moreover, generalizability theory was used to examine the reliability of the mQSF. Our data came from videotapes of clinical encounters between six simulated patients and six students and the ensuing feedback from the SPs to the students. Ten faculty members judged the SP feedback according to the items on the mQSF. Three weeks later, this procedure was repeated with the same faculty members and recordings. RESULTS: All but two items of the mQSF received importance ratings of > 2.5 on a four-point rating scale. A generalizability coefficient of 0.77 was established with two judges observing one encounter. CONCLUSIONS: The findings for content validity and reliability with two judges suggest that the mQSF is a valid and reliable instrument to assess the quality of feedback provided by simulated patients

Teaching Feedback to First-year Medical Students: Long-term Skill Retention and Accuracy of Student Self-assessment

Giving and receiving feedback are critical skills and should be taught early in the process of medical education, yet few studies discuss the effect of feedback curricula for first-year medical students. To study short-term and long-term skills and attitudes of first-year medical students after a multidisciplinary feedback curriculum. Prospective pre- vs. post-course evaluation using mixed-methods data analysis. First-year students at a public university medical school. We collected anonymous student feedback to faculty before, immediately after, and 8 months after the curriculum and classified comments by recommendation (reinforcing/corrective) and specificity (global/specific). Students also self-rated their comfort with and quality of feedback. We assessed changes in comments (skills) and self-rated abilities (attitudes) across the three time points. Across the three time points, students’ evaluation contained more corrective specific comments per evaluation [pre-curriculum mean (SD) 0.48 (0.99); post-curriculum 1.20 (1.7); year-end 0.95 (1.5); p = 0.006]. Students reported increased skill and comfort in giving and receiving feedback and at providing constructive feedback (p < 0.001). However, the number of specific comments on year-end evaluations declined [pre 3.35 (2.0); post 3.49 (2.3); year-end 2.8 (2.1)]; p = 0.008], as did students’ self-rated ability to give specific comments. Teaching feedback to early medical students resulted in improved skills of delivering corrective specific feedback and enhanced comfort with feedback. However, students’ overall ability to deliver specific feedback decreased over time

Identifying educator behaviours for high quality verbal feedback in health professions education: literature review and expert refinement

Background Health professions education is characterised by work-based learning and relies on effective verbal feedback. However the literature reports problems in feedback practice, including lack of both learner engagement and explicit strategies for improving performance. It is not clear what constitutes high quality, learner-centred feedback or how educators can promote it. We hoped to enhance feedback in clinical practice by distinguishing the elements of an educator’s role in feedback considered to influence learner outcomes, then develop descriptions of observable educator behaviours that exemplify them. Methods An extensive literature review was conducted to identify i) information substantiating specific components of an educator’s role in feedback asserted to have an important influence on learner outcomes and ii) verbal feedback instruments in health professions education, that may describe important educator activities in effective feedback. This information was used to construct a list of elements thought to be important in effective feedback. Based on these elements, descriptions of observable educator behaviours that represent effective feedback were developed and refined during three rounds of a Delphi process and a face-to-face meeting with experts across the health professions and education. Results The review identified more than 170 relevant articles (involving health professions, education, psychology and business literature) and ten verbal feedback instruments in health professions education (plus modified versions). Eighteen distinct elements of an educator’s role in effective feedback were delineated. Twenty five descriptions of educator behaviours that align with the elements were ratified by the expert panel. Conclusions This research clarifies the distinct elements of an educator’s role in feedback considered to enhance learner outcomes. The corresponding set of observable educator behaviours aim to describe how an educator could engage, motivate and enable a learner to improve. This creates the foundation for developing a method to systematically evaluate the impact of verbal feedback on learner performance

Group: Coping with toxic working environments

Background: Each working environment has its own culture. Sometimes the culture is very difficult to deal with - it is toxic. Signs of toxic environments are for example: high turnover in people, lack of communication or negative communication, high rate of employee sickness, or little or no enthusiasm among employees. Causes of a toxic environment are for example narcissistic leadership, lack of integrity among colleagues, or extreme pressure to perform. Working in a toxic environment can affect your performance: it can lead to less productivity, stress, loose of trust, low self-esteem, and negatively affect well-being. Although surviving in a toxic work environment can be a good learning experience, in general, as a Young Medical Educator it is not the place where you would like to be. In this workshop we would like to discuss strategies to cope with a toxic environment based on evidence from the literature and inputs from the facilitators and participants experience on what works. Who Should Attend: Those who are relatively new to medical education. Structure of Workshop: After the introduction (10’) we will: • Analyze what experiences people have with toxic working or learning environments in their own context including cultural challenges. (20’) • We will present a very short summary of the literature on the causes of a toxic environment and how it affects your performance. (5’) • Group work with examples of strategies dealing with of toxic environments (30’), including large group discussion (20’). Sharing take home message will close the session. (10) Intended Outcomes: • Recognize signs of and your reactions to a toxic environment. • Be aware of several strategies to cope with toxic environments, • Share what you learn with others inside and outside of your organization. Level: Novice/Intermediat