Aim: This study reports inter-rater reliability of the Adapted Measure of Support in Conversation (MSC) and Measure of Participation in Conversation (MPC) for TBI interactions. 
Method: The MSC and MPC were adapted to reflect theoretical models of cognitive-communication support for people with TBI. 10 casual and 10 purposeful TBI interactions were independently rated.  
Results: Strong inter-rater agreement was established on the MSC (ICC = 0.85-0.97) and the MPC (ICC = 0.85-0.97). All ratings scored within 0.5 on a 9 point scale.
Conclusion: This is the first scale to measure the communication partner during TBI interactions. It shows promise in evaluating communication partner training programs

McDonald, Skye

Power, Emma

Rietdijk, Rachel

Tate, Robyn

Togher, Leanne

The Aphasiology Archive

   1 Traumatic brain injury (TBI) can result in cognitive communication impairments which may significantly affect interpersonal relationships. Examining interactions with everyday communication partners is consistent with the WHO ICF call to consider environmental and other factors during assessment. However, few such measures are currently available.  One exception, developed for use with volunteers in conversations with people with aphasia (PWA) (Kagan et al., 2004), is the Measure of skill in Supported Conversation (MSC).  The MSC rates the uninjured communication partner’s ability to (i) acknowledge and (ii) reveal communication competence of the PWA.  The Measure of Participation in Conversation (MPC) examines the PWA’s ability to participate in the interactional and transactional elements of conversation (Kagan et al., 2004). While these measures were useful in evaluating communication training for volunteers who work with PWA, they were initially designed to evaluate skills in supporting PWA. There are no similar rating scales which evaluate communication partner behaviour in TBI interactions.  The structure and main elements of the Kagan scales provide a solid basis for use in TBI, however, the nature of support required in TBI interactions is different. Skills theorised to be important for supporting people with TBI have been developed by Ylvisaker and colleagues including scaffolding, cognitive supports, collaboration and elaboration techniques (Ylvisaker, Edelman & Sellars, 1998). These techniques are currently being evaluating in a clinical trial examining communication partner training in improving communications skills for people with severe TBI. With a paucity of measures to evaluate the contributions of communication partners in addition to those of the person with TBI, we sought to adapt the MSC and MPC to capture the specific conversational supports that were relevant to TBI interactions.       2 This study aims to: 1. Describe the modification of the Measure of Support in Conversation (MSC) and Measure of Participation in Conversation (MPC) (Kagan et al., 2004) for people with TBI and their communication partners based on current theoretical perspectives (Ylvisaker et al., 1998) and 2. Report on the inter-rater reliability of these adapted measures using the same conversation text types as will be employed in the clinical trial.  METHODOLOGY The original MPC and MSC scales are 9-point Likert scales, presented as a range of 0 to 4 with 0.5 levels for ease of scoring. The scale ranges from 0 (no participation) through 2 (adequate participation) to 4 (full participation in conversation).Within the MPC, there are 2 subscales encompassing Interaction and Transaction, while the MSC has 2 subscales including Acknowledging Competence and Revealing Competence. The Revealing Competence subscale is, in turn, composed of 3 elements which are scored separately and averaged to give the score for this subscale. The elements are: (a) Ensuring the adult understands, (b) Ensuring the adult has a means of responding and (c) Verification.    Development of the Adapted MPC and MSC scales occurred over a number of stages over approximately a one year period. Firstly, we matched behavioural descriptors taken from Ylvisaker’s collaborative and elaborative approach to the themes and categories in the original MPC / MSC (version 1). All MPC and MSC original descriptors were then combined with Ylvisaker’s concepts (version 2). Following this, we undertook a process of deletion of overlapping and irrelevant information to TBI (version 3). Scale descriptors and anchors were then modified as piloting indicated that ratings of the scale (“very poor, adequate and outstanding”) produced binomial results as raters had difficulty differentiating “adequate” and “outstanding”.  We therefore changed the anchors to: MPC: “No participation / Some participation / Full participation”    3 MSC: “Not supportive / Basic skill in support / Highly skilled support”  The final version was developed after group discussion between the authors and pilot testing on 40 conversational samples of people with TBI from previous studies (Appendix 1). Two raters (EP & RR) then independently rated 10 casual conversations, and 10 purposeful conversations to match conversation text types used in the clinical trial.  Procedure for rating Ten, five minute unstructured casual conversational samples between a person with TBI and their everyday communication partner (ECP) were randomised and rated on the Adapted MSC and MPC scales independently by two trained raters. Then ten, five minute purposeful conversational samples between a person with TBI and their ECP were randomised and rated. In the purposeful sample, participants engaged in a jointly constructed narrative retelling task. Inter-rater reliability was examined with two raters (the authors) using intra-class correlation (ICC). The calculations of Walter, Eliasziw, and Donner (1998) indicated 20 samples were required to provide sufficient power to detect fair (ICC ≥ 0.4) to excellent (ICC ≥ 0.75) levels of reliability (as defined by Cicchetti, 1994). Data were entered in SPSS and analysis conducted using Intraclass correlation coefficients (ICC).   RESULTS Results of the inter-rater reliability ratings are presented in Table 1. Inter-rater reliability for both the Adapted MPC and the MSC scales was excellent ranging from ICC = 0.84 for the Adapted MPC Interaction and Transaction scales to ICC = 0.97 for the Adapted MSC Acknowledge Competence scale. The ICC ratings were comparable with those reported by Kagan in 2001 and 2004. All ratings were scored within 0.5 on a 9 point scale for the 20 samples.  .    4 DISCUSSION With recent acknowledgement of the need to assess communication performance in real-life contexts (Coelho, Ylvisaker & Turkstra, 2005) there has been renewed focus on the development of socially valid tools.  Two broad approaches have been taken including: (1) report from the person with TBI or a close-other; or (2) direct observation of the communication skills of the person with TBI in real situations. These approaches have resulted in questionnaire tools, such as the La Trobe Communication Questionnaire (Douglas, O’Flaherty and Snow, 2000) to gain information on perceptions of communicative ability from everyday communication partners, and direct observation of conversations using fine-grained analysis techniques.  Observational assessments range from frequency counts of the occurrences of inappropriate conversational behaviours (Coelho, 2007), ratings of frequencies of behaviours based on a four-point scale (Linscott, Knight & Godfrey, 1996), to an overall rating of language content and communication efficiency (Bellon and Rees, 2006).   Most global conversational proficiency ratings of people with TBI focus either on the person with TBI or on the interaction as a whole (Shelton & Shryock, 2007). They do not provide insight into the specific role of the communication partner, and may not be sensitive to the effects of communication partner training. The Adapted MPC and MSC scales provide a tool which specifically focuses on the skills of communication partners in providing conversational support to the person with TBI, and may therefore be sensitive to detecting change following communication partner training. The results of this study lend preliminary support to the psychometric robustness of this scale. While it is recognised that future work is needed to further evaluate this scale, the Adapted MPC and MSC scales offers a new way of examining communication partner contributions to TBI interactions.       5 Table 1. Inter-rater reliability results for Adapted MSC and MPC scales. Intra class correlations (ICC) for two raters   Adapted MPC Adapted MSC  Interaction Transaction Acknowledge competence Reveal competence (average of 3 subscales) Casual Conversation (n=10 samples) ICC = 0.84, p<0.01 ICC = 0.84, p<0.01 ICC = 0.97, p<0.001 ICC = 0.85, p<0.001 Purposeful Conversation  (n=10 samples) ICC = 0.91, p<0.001 ICC = 0.93, p<0.001 ICC = 0.89, p<0.001 ICC = 0.88, p<0.001 Kagan  et al. 2001 / Kagan et al., 2004 (Original scales) ICC = 0.85 / 0.93 ICC = 0.73 / 0.94 ICC = 0.83 / 0.91 ICC = 0.89 / 0.96       6   REFERENCES   Bellon, M.L., & Rees, R.J. (2006). The effect of context on communication: a study of the language and communication skills of adults with acquired brain injury. Brain Injury, 20(10), 1069-1078. Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290. Coelho, C., Ylvisaker, M. & Turkstra, L.S. (2005). Nonstandardized assessment approaches for individuals with traumatic brain injuries. Seminars in Speech & Language, (4), 223-241.  Coelho, C. A. (2007). Management of discourse deficits following traumatic brain injury: Progress, caveats, and needs. Seminars in Speech & Language 28(2), 122-135.  Douglas, J., C. O'Flaherty, C.A. (2000). Measuring perception of communicative ability: the development and evaluation of the La Trobe Communication Questionnaire. Aphasiology, 14(3), 251-268.  Kagan, A., Winckel, J., Black, S, Duchan, J.F., Simmons-Mackie, N. & Square, P.(2001). Training volunteers as conversational partners using Supported Conversation with Adults with Aphasia (SCA): A controlled trial. Journal of Speech, Language and Hearing Research, 44, 624-638. Kagan, A., Black, S., Duchan, J.F., Simmons-Mackie, N. & Square, P. (2004). A set of observational measures for rating support and participation in conversation between adults with aphasia and their conversation partners.  Topics in Stroke Rehabilitation, 11(1), 67-83. Linscott, R. J., Knight, R.G. & Godfrey, H.P.D.  (1996). The Profile of Functional Impairment of Communication (PFIC): a measure of communication impairment for clinical use. Brain Injury 10 (6): 397-412. Shelton, C., & Shryock, M. Effectiveness of communication/interaction strategies with patients who have neurological injuries in a rehabilitation setting. Brain Injury, 211259-66. Walter, S. D., Eliasziw, M., & Donner, A. (1998). Sample size and optimal designs for reliability studies. Statistics in Medicine, 17(1), 101–110. Ylvisaker, M., Sellars, C. & Edelman, L. (1998). Rehabilitation after traumatic brain injury in preschoolers. Traumatic brain injury rehabilitation. Children and adolescents. M. Ylvisaker. Newton, MA, Butterworth-Heinemann, 303-329.                       7  Appendix 1  Adapted TBI MPC Behavioural Guidelines: Summary  -   adapted from Kagan et al (2004; 2001) Think in terms of skill of TBI in participating.  Appropriateness is key (a well executed but overused technique would result in a lower score).   A. Interaction Verbal / vocal • Does TBI share responsibility for maintaining feel/flow of conversation (incl: appropriate affect)?  • Does TBI add information to maintain the topic?  • Does TBI ask questions of ECP which follow-up on the topic?  • Does TBI use appropriate turn-taking (taking their turn, passing turn to ECP appropriately)?  • Does TBI demonstrate active listening (e.g. acknowledging, backchannelling)?  • Does TBI choose appropriate topics and questions for the context?  Nonverbal • Does TBI initiate/maintain interaction with CP or make use of supports offered by CP to initiate/maintain interaction?  • Is TBI pragmatically appropriate?  • Does TBI ever acknowledge the frustration of the CP or acknowledge their competence/skill?  • Behaviours might include: o Appropriate eye contact, use of gesture, body posture and facial expression, use of writing or drawing in any form, use of resource material  Score MPC Interaction:     Full participation Some participation No participation at all   B. Transaction Verbal / vocal • Does TBI maintain exchange of information, opinions and feelings with CP, by sharing details or by inviting CP to share details?  • Does TBI present information in an organised way?   • Does TBI provide an appropriate amount of information?  • Does TBI ask clarifying questions when necessary?  Nonverbal • Does TBI ever initiate transaction? • Introducing or referring back to a previous topic • Spontaneously using a compensatory technique  • Does content of transaction appear to be accurate? (depending on context and purpose of rating, rater would have more/less access to means of verification of information)  • Does TBI use support offered by CP for purpose of transaction?  Eg., Referring to a list/diary, using    8 the organization of the conversation provided by CP (e.g. responding to closed choice questions)  Score MPC Transaction:     Some participation Full participation No participation at all  Adapted TBI MSC Behavioural Guidelines: Summary  -   adapted from Kagan et al (2004; 2001) Think in terms of skill of ECP in providing ‘support’.  Appropriateness is key (a well executed but overused technique would result in a lower score).   A. Acknowledging Competence Natural adult talk appropriate to context • Feel and flow of natural adult conversation appropriate to context,   o e.g., social chat vs. interview; respectful approach to verification (verifying that the conversation partner has understood rather than verifying that adult with brain injury knows what they want to say; not over-verifying)  • Not patronizing (loudness, tone of voice, rate, enunciation)  • Appropriate emotional tone / use of humour  • Uses collaborative talk (rather than teaching / testing)  • Establishes equal leadership roles in the conversation  • Uses true questions rather than testing questions  Sensitivity to partner • Incorrect / unclear responses handled respectfully by giving correct information in a non-punitive manner  • Sensitive to TBI’s attempts to engage in conversation, Confirms partner’s contribution.  • Encourage when appropriate, Shows enthusiasm for partner’s contribution.  • Acknowledge competence when adult with brain injury is frustrated e.g., “I know you know what you want to say.”, Acknowledges difficulties.  • “Listening attitude”, Demonstrates active listening (e.g. acknowledging, back-channelling)  • Takes on communicative burden as appropriate / making adult with brain injury feel comfortable  • Communicates respect for other person’s concerns, perspectives and abilities  • Questions in a non-demanding, supportive manner  • Takes appropriate conversational turns  Score MSC Acknow Comp:      Highly skilled support Basic skill in support Not supportive      9  B. Revealing Competence 1. Ensure adult understands (e.g. topic, questions) • Verbal (e.g. short, simple sentences; redundancy; is there some verbal adaptation?)  • Nonverbal o Gesture  Meaningful; slightly exaggerated; used to emphasize or clarify o Writing  Clear and visible; appropriate key words o Resources Used only when necessary (would something simpler  suffice?)  • Response to communicative cues (e.g., reacting to facial expressions that indicate lack of comprehension)  • Gives cues in a conversational manner  • Provides an appropriate level of cognitive support (e.g. referring to diary, making notes)  • Organises information in the conversation as clearly as possible to support comprehension (e.g., sequential order, causality, similarity and difference, association)  • Makes connections between topics  • Reviews organisation of information (e.g. summarises)  Score MSC Reveal Comp 1:      2. Ensure adult has means of responding (and elaborating) • Response to communicative cues (e.g., giving enough time to respond)  • Establishes equal leadership roles in the conversation  • Introduces and initiates topic of interest  • Allows partner to take appropriate conversational turns  • Maintains the topic by adding information  • Invites elaboration (e.g. uses open-ended questions, statements, links to experiences of TBI)  • Uses questions which are appropriate to the person’s abilities (e.g. simple questions, closed questions when necessary)  • Helps partner express thoughts when struggle occurs  Score MSC Reveal Comp 2:      3. Verification  (Accuracy of adult’s response not automatically assumed) • Response to communicative cues (e.g. partner infers the intended message of the person with brain injury, based on all available cues)  • Note: Verification often involves checking in a different way (e.g., using a yes/no question)  • Confirms understanding of what has been said (paraphrasing, checking)  • Uses clarifying questions as appropriate  Score MSC  Highly skilled support Basic skill in support Not supportive Highly skilled support Basic skill in support Not supportive Highly skilled support Basic skill in support Not supportive    10 Reveal Comp 3:      

Inter-rater reliability of the Measure of Support in Conversation and Measure of Participation in Conversation (Kagan et al., 2004) adapted for people with Traumatic Brain Injury (TBI) and their communication partners

http://aphasiology.pitt.edu/2072/1/viewpaper.pdf

Inter-rater reliability of the Measure of Support in Conversation and Measure of Participation in Conversation (Kagan et al., 2004) adapted for people with Traumatic Brain Injury (TBI) and their communication partners

Abstract

Similar works

Full text

Available Versions

The Aphasiology Archive