Are We Making Progress in Medical Education?

Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/75582/1/j.1525-1497.2006.00446.x.pd

The implementation of a mobile problem-specific electronic CEX for assessing directly observed student—patient encounters

Background: Facilitating direct observation of medical students' clinical competencies is a pressing need. Methods: We developed an electronic problem-specific Clinical Evaluation Exercise (eCEX) based on a national curriculum. We assessed its feasibility in monitoring and recording students' competencies and the impact of a grading incentive on the frequency of direct observations in an internal medicine clerkship. Students (n=56) at three clinical sites used the eCEX and comparison students (n=56) at three other clinical sites did not. Students in the eCEX group were required to arrange 10 evaluations with faculty preceptors. Students in the second group were required to document a single, faculty observed ‘Full History and Physical’ encounter with a patient. Students and preceptors were surveyed at the end of each rotation. Results: eCEX increased students' and evaluators' understanding of direct-observation objectives and had a positive impact on the evaluators' ability to provide feedback and assessments. The grading incentive increased the number of times a student reported direct observation by a resident preceptor. Conclusions: eCEX appears to be an effective means of enhancing student evaluation

'Correction:'Peer chart audits: A tool to meet Accreditation Council on Graduate Medical Education (ACGME) competency in practice-based learning and improvement

<p>Abstract</p> <p>Background</p> <p>The Accreditation Council on Graduate Medical Education (ACGME) supports chart audit as a method to track competency in Practice-Based Learning and Improvement. We examined whether peer chart audits performed by internal medicine residents were associated with improved documentation of foot care in patients with diabetes mellitus.</p> <p>Methods</p> <p>A retrospective electronic chart review was performed on 347 patients with diabetes mellitus cared for by internal medicine residents in a university-based continuity clinic from May 2003 to September 2004. Residents abstracted information pertaining to documentation of foot examinations (neurological, vascular, and skin) from the charts of patients followed by their physician peers. No formal feedback or education was provided.</p> <p>Results</p> <p>Significant improvement in the documentation of foot exams was observed over the course of the study. The percentage of patients receiving neurological, vascular, and skin exams increased by 20% (from 13% to 33%) (p = 0.001), 26% (from 45% to 71%) (p < 0.001), and 18% (51%–72%) (p = 0.005), respectively. Similarly, the proportion of patients receiving a well-documented exam which includes all three components – neurological, vascular and skin foot exam – increased over time (6% to 24%, p < 0.001).</p> <p>Conclusion</p> <p>Peer chart audits performed by residents in the absence of formal feedback were associated with improved documentation of the foot exam in patients with diabetes mellitus. Although this study suggests that peer chart audits may be an effective tool to improve practice-based learning and documentation of foot care in diabetic patients, evaluating the actual performance of clinical care was beyond the scope of this study and would be better addressed by a randomized controlled trial.</p

Faculty verbal evaluations reveal strategies used to promote medical student performance

Background: Preceptors rarely follow medical students&#x2019; developing clinical performance over time and across disciplines. This study analyzes preceptors&#x2019; descriptions of longitudinal integrated clerkship (LIC) students&#x2019; clinical development and their identification of strategies to guide students&#x2019; progress. Methods: We used a common evaluation framework, reporter-interpreter-manager-educator, to guide multidisciplinary LIC preceptors&#x2019; discussions of students&#x2019; progress. We conducted thematic analysis of transcripts from preceptors&#x2019; (seven longitudinal ambulatory preceptors per student) quarterly group discussions of 15 students&#x2019; performance over one year. Results: All students&#x2019; clinical development progressed, although most experienced obstacles. Lack of structure in the history and physical exam commonly obstructed progression. Preceptors used templates for data gathering, and modeling or experiences in the inpatient setting to provide time and solidify structure. To advance students&#x2019; knowledge acquisition, many preceptors identified focused learning topics with their students; to promote application of knowledge, preceptors used reasoning strategies to teach the steps involved in synthesizing clinical data. Preceptors shared accountability for helping students advance as the LIC allowed them to follow students&#x2019; response to teaching strategies. Discussion: These results depict preceptors&#x2019; perceptions of LIC students&#x2019; developmental continuum and illustrate how multidisciplinary preceptors can use a common evaluation framework to identify strategies to improve performance and follow students&#x2019; performance longitudinally

Associations between subspecialty fellowship interest and knowledge of internal medicine: A hypothesis-generating study of internal medicine residents

<p>Abstract</p> <p>Background</p> <p>Little is known about whether and how medical knowledge relates to interest in subspecialty fellowship training. The purpose of this study was to examine the relationships between residents' interest in subspecialty fellowship training and their knowledge of internal medicine (IM).</p> <p>Methods</p> <p>A questionnaire was emailed to 48 categorical postgraduate-year (PGY) two and three residents at a New York university-affiliated IM residency program in 2007 using the Survey Monkey online survey instrument. Overall and content area-specific percentile scores from the IM in-training examination (IM-ITE) for the same year was used to determine objective knowledge.</p> <p>Results</p> <p>Forty-five of 48 residents (response rate was 93.8%) completed the survey. Twenty-two (49%) were PG2 residents and 23(51%) were PGY3 residents. Sixty percent of respondents were male. Six (13%) residents were graduates of U.S. medical schools. Eight (18%) reported formal clinical training prior to starting internal medicine residency in the U.S. Of this latter group, 6 (75%) had training in IM and 6 (75) % reported a training length of 3 years or less. Thirty-seven of 45 (82%) residents had a subspecialty fellowship interest. Residents with a fellowship interest had a greater mean overall objective knowledge percentile score (56.44 vs. 31.67; p = 0.04) as well as greater mean percentile scores in all content areas of IM. The adjusted mean difference was statistically significant (p < 0.02) across three content areas.</p> <p>Conclusions</p> <p>More than half of surveyed residents indicated interest in pursuing a subspecialty fellowship. Fellowship interest appears positively associated with general medical knowledge in this study population. Further work is needed to explore motivation and study patterns among internal medicine residents.</p

Assessing clinical communication skills in physicians: are the skills context specific or generalizable

<p>Abstract</p> <p>Background</p> <p>Communication skills are essential for physicians to practice Medicine. Evidence for the validity and domain specificity of communication skills in physicians is equivocal and requires further research. This research was conducted to adduce evidence for content and context specificity of communication skills and to assess the usefulness of a generic instrument for assessing communication skills in International Medical Graduates (IMGs).</p> <p>Methods</p> <p>A psychometric design was used for identifying the reliability and validity of the communication skills instruments used for high-stakes exams for IMG's. Data were collected from 39 IMGs (19 men – 48.7%; 20 women – 51.3%; Mean age = 41 years) assessed at 14 station OSCE and subsequently in supervised clinical practice with several instruments (patient surveys; ITERs; Mini-CEX).</p> <p>Results</p> <p>All the instruments had adequate reliability (Cronbach's alpha: .54 – .96). There were significant correlations (r range: 0.37 – 0.70, <it>p </it>< .05) of communication skills assessed by examiner with standardized patients, and of mini-CEX with patient surveys, and ITERs. The intra-item reliability across all cases for the 13 items was low (Cronbach's alpha: .20 – .56). The correlations of communication skills within method (e.g., OSCE or clinical practice) were significant but were non-significant between methods (e.g., OSCE and clinical practice).</p> <p>Conclusion</p> <p>The results provide evidence of context specificity of communication skills, as well as convergent and criterion-related validity of communication skills. Both in OSCEs and clinical practice, communication checklists need to be case specific, designed for content validity.</p

The professionalism disconnect: Do entering residents identify yet participate in unprofessional behaviors?

Background Professionalism has been an important tenet of medical education, yet defining it is a challenge. Perceptions of professional behavior may vary by individual, medical specialty, demographic group and institution. Understanding these differences should help institutions better clarify professionalism expectations and provide standards with which to evaluate resident behavior. Methods Duke University Hospital and Vidant Medical Center/East Carolina University surveyed entering PGY1 residents. Residents were queried on two issues: their perception of the professionalism of 46 specific behaviors related to training and patient care; and their own participation in those specified behaviors. The study reports data analyses for gender and institution based upon survey results in 2009 and 2010. The study received approval by the Institutional Review Boards of both institutions. Results 76% (375) of 495 PGY1 residents surveyed in 2009 and 2010 responded. A majority of responders rated all 46 specified behaviors as unprofessional, and a majority had either observed or participated in each behavior. For all 46 behaviors, a greater percentage of women rated the behaviors as unprofessional. Men were more likely than women to have participated in behaviors. There were several significant differences in both the perceptions of specified behaviors and in self-reported observation of and/or involvement in those behaviors between institutions. Respondents indicated the most important professionalism issues relevant to medical practice include: respect for colleagues/patients, relationships with pharmaceutical companies, balancing home/work life, and admitting mistakes. They reported that professionalism can best be assessed by peers, patients, observation of non-medical work and timeliness/detail of paperwork. Conclusion Defining professionalism in measurable terms is a challenge yet critical in order for it to be taught and assessed. Recognition of the differences by gender and institution should allow for tailored teaching and assessment of professionalism so that it is most meaningful. A shared understanding of what constitutes professional behavior is an important first step