“It’s hard to tell”. The challenges of scoring patients on standardised outcome measures by multidisciplinary teams: a case study of Neurorehabilitation

Background Interest is increasing in the application of standardised outcome measures in clinical practice. Measures designed for use in research may not be sufficiently precise to be used in monitoring individual patients. However, little is known about how clinicians and in particular, multidisciplinary teams, score patients using these measures. This paper explores the challenges faced by multidisciplinary teams in allocating scores on standardised outcome measures in clinical practice. Methods Qualitative case study of an inpatient neurorehabilitation team who routinely collected standardised outcome measures on their patients. Data were collected using non participant observation, fieldnotes and tape recordings of 16 multidisciplinary team meetings during which the measures were recited and scored. Eleven clinicians from a range of different professions were also interviewed. Data were analysed used grounded theory techniques. Results We identified a number of instances where scoring the patient was 'problematic'. In 'problematic' scoring, the scores were uncertain and subject to revision and adjustment. They sometimes required negotiation to agree on a shared understanding of concepts to be measured and the guidelines for scoring. Several factors gave rise to this problematic scoring. Team members' knowledge about patients' problems changed over time so that initial scores had to be revised or dismissed, creating an impression of deterioration when none had occurred. Patients had complex problems which could not easily be distinguished from each other and patients themselves varied in their ability to perform tasks over time and across different settings. Team members from different professions worked with patients in different ways and had different perspectives on patients' problems. This was particularly an issue in the scoring of concepts such as anxiety, depression, orientation, social integration and cognitive problems. Conclusion From a psychometric perspective these problems would raise questions about the validity, reliability and responsiveness of the scores. However, from a clinical perspective, such characteristics are an inherent part of clinical judgement and reasoning. It is important to highlight the challenges faced by multidisciplinary teams in scoring patients on standardised outcome measures but it would be unwarranted to conclude that such challenges imply that these measures should not be used in clinical practice for decision making about individual patients. However, our findings do raise some concerns about the use of such measures for performance management

Learning Oral Presentation Skills: A Rhetorical Analysis with Pedagogical and Professional Implications

OBJECTIVE: Oral presentation skills are central to physician-physician communication; however, little is known about how these skills are learned. Rhetoric is a social science which studies communication in terms of context and explores the action of language on knowledge, attitudes, and values. It has not previously been applied to medical discourse. We used rhetorical principles to qualitatively study how students learn oral presentation skills and what professional values are communicated in this process. DESIGN: Descriptive study. SETTING: Inpatient general medicine service in a university-affiliated public hospital. PARTICIPANTS: Twelve third-year medical students during their internal medicine clerkship and 14 teachers. MEASUREMENTS: One-hundred sixty hours of ethnographic observation. including 73 oral presentations on rounds. Discoursed-based interviews of 8 students and 10 teachers. Data were qualitatively analyzed to uncover recurrent patterns of communication. MAIN RESULTS: Students and teachers had different perceptions of the purpose of oral presentation, and this was reflected in performance. Students described and conducted the presentation as a rule-based, data-storage activity governed by “order” and “structure.” Teachers approached the presentation as a flexible means of “communication” and a method for “constructing” the details of a case into a diagnostic or therapeutic plan. Although most teachers viewed oral presentations rhetorically (sensitive to context), most feedback that students received was implicit and acontextual, with little guidance provided for determining relevant content. This led to dysfunctional generalizations by students, sometimes resulting in worse communication skills (e.g., comment “be brief” resulted in reading faster rather than editing) and unintended value acquisition (e.g., request for less social history interpreted as social history never relevant). CONCLUSIONS: Students learn oral presentation by trial and error rather than through teaching of an explicit rhetorical model. This may delay development of effective communication skills and result in acquisition of unintended professional values. Teaching and learning of oral presentation skills may be improved by emphasizing that context determines content and by making explicit the tacit rules of presentation