Assessing clinical communication skills in physicians: are the skills context specific or generalizable

<p>Abstract</p> <p>Background</p> <p>Communication skills are essential for physicians to practice Medicine. Evidence for the validity and domain specificity of communication skills in physicians is equivocal and requires further research. This research was conducted to adduce evidence for content and context specificity of communication skills and to assess the usefulness of a generic instrument for assessing communication skills in International Medical Graduates (IMGs).</p> <p>Methods</p> <p>A psychometric design was used for identifying the reliability and validity of the communication skills instruments used for high-stakes exams for IMG's. Data were collected from 39 IMGs (19 men – 48.7%; 20 women – 51.3%; Mean age = 41 years) assessed at 14 station OSCE and subsequently in supervised clinical practice with several instruments (patient surveys; ITERs; Mini-CEX).</p> <p>Results</p> <p>All the instruments had adequate reliability (Cronbach's alpha: .54 – .96). There were significant correlations (r range: 0.37 – 0.70, <it>p </it>< .05) of communication skills assessed by examiner with standardized patients, and of mini-CEX with patient surveys, and ITERs. The intra-item reliability across all cases for the 13 items was low (Cronbach's alpha: .20 – .56). The correlations of communication skills within method (e.g., OSCE or clinical practice) were significant but were non-significant between methods (e.g., OSCE and clinical practice).</p> <p>Conclusion</p> <p>The results provide evidence of context specificity of communication skills, as well as convergent and criterion-related validity of communication skills. Both in OSCEs and clinical practice, communication checklists need to be case specific, designed for content validity.</p

Integrated problem-based learning in the neuroscience curriculum – the SUNY Downstate experience

BACKGROUND: This paper reports the author's initial experience as Block Director in converting a Conventional Curriculum into a problem-based learning model (PBL) for teaching Psychopathology. As part of a wide initiative in curriculum reform, Psychopathology, which was a six-week course in the second-year medical school curriculum, became integrated into a combined Neuroscience block. The study compares curriculum conversion at State University of New York (SUNY), Downstate, with the experiences at other medical centres that have instituted similar curricula reform. METHODS: Student satisfaction with the Conventional and PBL components of the Neuroscience curriculum was compared using questionnaires and formal discussions between faculty and a body of elected students. The PBL experience in Psychopathology was also compared with that of the rest of the Neuroscience Block, which used large student groups and expert facilitators, while the Psychopathology track was limited to small groups using mentors differing widely in levels of expertise. RESULTS: Students appeared to indicate a preference toward conventional lectures and large PBL groups using expert facilitators in contrast to small group mentors who were not experts. Small PBL groups with expert mentors in the Psychopathology track were also rated favorably. CONCLUSION: The study reviews the advantages and pitfalls of the PBL system when applied to a Neuroscience curriculum on early career development. At SUNY, conversion from a Conventional model to a PBL model diverged from that proposed by Howard S. Barrows where student groups define the learning objectives and problem-solving strategies. In our model, the learning objectives were faculty-driven. The critical issue for the students appeared to be the level of faculty expertise rather than group size. Expert mentors were rated more favorably by students in fulfilling the philosophical objectives of PBL. The author, by citing the experience at other major Medical Faculties, makes a cautious attempt to address the challenges involved in the conversion of a Psychopathology curriculum into a PBL dominated format

A comparison of course-related stressors in undergraduate problem-based learning (PBL) versus non-PBL medical programmes

Background: Medical students report high levels of stress related to their medical training as well as to other personal and financial factors. The aim of this study is to investigate whether there are differences in course-related stressors reported by medical students on undergraduate problem-based learning (PBL) and non-PBL programmes in the UK. Method: A cross-sectional study of second-year medical students in two UK medical schools (one PBL and one non-PBL programme) was conducted. A 16-question self-report questionnaire, derived from the Perceived Medical Student Stress Scale and the Higher Education Stress Inventory, was used to measure course-related stressors. Following univariate analysis of each stressor between groups, multivariate logistic regression was used to determine which stressors were the best predictors of each course type, while controlling for socio-demographic differences between the groups. Results: A total of 280 students responded. Compared to the non-PBL students (N = 197), the PBL students (N = 83) were significantly more likely to agree that: they did not know what the faculty expected of them (Odds Ratio (OR) = 0.38, p = 0.03); there were too many small group sessions facilitated only by students resulting in an unclear curriculum (OR = 0.04, p < 0.0001); and that there was a lack of opportunity to explore academic subjects of interest (OR = 0.40, p = 0.02). They were significantly more likely to disagree that: there was a lack of encouragement from teachers (OR = 3.11, p = 0.02); and that the medical course fostered a sense of anonymity and feelings of isolation amongst students (OR = 3.42, p = 0.008). Conclusion: There are significant differences in the perceived course-related stressors affecting medical students on PBL and non-PBL programmes. Course designers and student support services should therefore tailor their work to minimise, or help students cope with, the specific stressors on each course type to ensure optimum learning and wellbeing among our future doctors

Long-term follow up of factual knowledge after a single, randomised problem-based learning course

BACKGROUND: The long-term effect of problem-based learning (PBL) on factual knowledge is poorly investigated. We took advantage of a previous randomised comparison between PBL and traditional teaching in a 3(rd )year course to follow up factual knowledge of the students during their 4(th )and 5(th )year of medical school training. METHODS: 3(rd )year medical students were initially randomized to participate in a problem-based (PBL, n = 55), or a lecture-based (LBL, n = 57) course in basic pharmacology. Summative exam results were monitored 18 months later (after finishing a lecture-based course in clinical pharmacology). Additional results of an unscheduled, formative exam were obtained 27 months after completion of the first course. RESULTS: Of the initial sample of 112 students, 90 participated in the second course and exam (n = 45, 45). 32 (n = 17 PBL, n = 15 LBL) could be exposed to the third, formative exam. Mean scores (± SD) were 22.4 ± 6.0, 27.4 ± 4.9 and 20.1 ± 5.0 (PBL), or 22.2 ± 6.0, 28.4 ± 5.1 and 19.0 ± 4.7 (LBL) in the first, second and third test, respectively (maximum score: 40). No significant differences were found between the two groups. CONCLUSION: A small-scale exposure to PBL, applied under randomized conditions but in the context of a traditional curriculum, does not sizeably change long-term presence of factual knowledge within the same discipline

Exploring the divergence between self-assessment and self-monitoring

Many models of professional self-regulation call upon individual practitioners to take responsibility both for identifying the limits of their own skills and for redressing their identified limits through continuing professional development activities. Despite these expectations, a considerable literature in the domain of self-assessment has questioned the ability of the self-regulating professional to enact this process effectively. In response, authors have recently suggested that the construction of self-assessment as represented in the self-regulation literature is, itself, problematic. In this paper we report a pair of studies that examine the relationship between self-assessment (a global judgment of one’s ability in a particular domain) and self-monitoring (a moment-by-moment awareness of the likelihood that one maintains the skill/knowledge to act in a particular situation). These studies reveal that, despite poor correlations between performance and self-assessments (consistent with what is typically seen in the self-assessment literature), participant performance was strongly related to several measures of self-monitoring including: the decision to answer or defer responding to a question, the amount of time required to make that decision to answer or defer, and the confidence expressed in an answer when provided. This apparent divergence between poor overall self-assessment and effective self-monitoring is considered in terms of how the findings might inform our understanding of the cognitive mechanisms yielding both self-monitoring judgments and self-assessments and how that understanding might be used to better direct education and learning efforts

Combined student ratings and self-assessment provide useful feedback for clinical teachers

Many evaluation instruments have been developed to provide feedback to physicians on their clinical teaching but written feedback alone is not always effective. We explored whether feedback effectiveness improved when teachers’ self-assessment was added to written feedback based on student ratings. 37 physicians (10 residents, 27 attending physicians) from different specialties (Internal Medicine, Surgery, Obstetrics/Gynecology, Pediatrics, Neurology, Dermatology, Ophthalmology, ENT, and Psychiatry) were invited to fill out a self-assessment questionnaire on their teaching skills. Students completed an almost identical questionnaire to evaluate the same teachers based on their experiences during clerkships. After receiving written feedback incorporating their self-assessment and the student ratings, the teachers indicated their perceptions of the self-assessment exercise and the written feedback in a questionnaire (five-point Likert scale items) and next, in more detail, in semi-structured interviews with a purposive sample of 12 of the participating teachers. 25 physicians participated (67%). The results showed that self-assessment and student feedback were both perceived as useful (3.7, SD 1.0) but the latter was considered more effective. The physicians we interviewed considered the combination of self-assessment with student ratings more effective than either self-assessment or written feedback alone. Notably, discrepancies between student ratings and self-assessment were deemed a strong incentive for change. We conclude that self-assessment can be a useful tool to stimulate improvement of clinical teaching when it is combined with written feedback based on student ratings. Future research among larger groups is needed to confirm our findings and examine whether these combined tools actually lead to improved teaching

The relationships between problem characteristics, achievement-related behaviors, and academic achievement in problem-based learning

This study investigated the influence of five problem characteristics on students' achievement-related classroom behaviors and academic achievement. Data from 5,949 polytechnic students in PBL curricula across 170 courses were analyzed by means of path analysis. The five problem characteristics were: (1) problem clarity, (2) problem familiarity, (3) the extent to which the problem stimulated group discussion, (4) self-study, and (5) identification of learning goals. The results showed that problem clarity led to more group discussion, identification of learning goals, and self-study than problem familiarity. On the other hand, problem familiarity had a stronger and direct impact on academic achievement

Can Score Databanks Help Teaching?

Basic courses in most medical schools assess students' performance by conferring scores. The objective of this work is to use a large score databank for the early identification of students with low performance and to identify course trends based on the mean of students' grades. METHODOLOGY/PRINCIPAL FINDINGS: We studied scores from 2,398 medical students registered in courses over a period of 10 years. Students in the first semester were grouped into those whose ratings remained in the lower quartile in two or more courses (low-performance) and students who had up to one course in the lower quartile (high-performance). ROC curves were built, aimed at the identification of a cut-off average score in the first semesters that would be able to predict low performances in future semesters. Moreover, to follow the long-term pattern of each course, the mean of all scores conferred in a semester was compared to the overall course mean obtained by averaging 10 years of data. Individuals in the low-performance group had a higher risk of being in the lower quartile of at least one course in the second semester (relative risk 3.907; 95% CI: 3.378-4.519) and in the eighth semester (relative risk 2.873; 95% CI: 2.495-3.308). The prediction analysis revealed that an average score of 7.188 in the first semester could identify students that presented scores below the lower quartiles in both the second and eighth semesters (p<0.0001 for both AUC). When scores conferred by single courses were compared over time, three time-trend patterns emerged: low variation, upward trend and erratic pattern. CONCLUSION/SIGNIFICANCE: An early identification of students with low performance may be useful in promoting pedagogical strategies for these individuals. Evaluation of the time trend of scores conferred by courses may help departments monitoring changes in personnel and methodology that may affect a student's performance

Interactive seminars or small group tutorials in preclinical medical education: results of a randomized controlled trial

<p>Abstract</p> <p>Background</p> <p>Learning in small group tutorials is appreciated by students and effective in the acquisition of clinical problem-solving skills but poses financial and resource challenges. Interactive seminars, which accommodate large groups, might be an alternative. This study examines the educational effectiveness of small group tutorials and interactive seminars and students' preferences for and satisfaction with these formats.</p> <p>Methods</p> <p>Students in year three of the Leiden undergraduate medical curriculum, who agreed to participate in a randomized controlled trial (RCT, n = 107), were randomly allocated to small group tutorials (n = 53) or interactive seminars (n = 54). Students who did not agree were free to choose either format (n = 105). Educational effectiveness was measured by comparing the participants' results on the end-of-block test. Data on students' reasons and satisfaction were collected by means of questionnaires. Data was analyzed using student unpaired t test or chi-square test where appropriate.</p> <p>Results</p> <p>There were no significant differences between the two educational formats in students' test grades. Retention of knowledge through active participation was the most frequently cited reason for preferring small group tutorials, while a dislike of compulsory course components was mentioned more frequently by students preferring interactive seminars. Small group tutorials led to greater satisfaction.</p> <p>Conclusions</p> <p>We found that small group tutorials leads to greater satisfaction but not to better learning results. Interactive learning in large groups might be might be an effective alternative to small group tutorials in some cases and be offered as an option.</p