Assessing sensitivity to change: choosing the appropriate change coefficient

The past 20-years have seen the development and evaluation of many health status measures. Unlike the high standards demanded of those who conduct and report clinical intervention trials, the methodological rigor for studies examining the sensitivity to change of health status measures are less demanding. It is likely that the absence of a criterion standard for change in health status contributes to this shortcoming. To increase confidence in the results of these types of studies investigators have often calculated multiple change coefficients for the same patient sample. The purpose of this report is to identify the conflict that arises when multiple change coefficients are applied to the same patient sample. Three families of change coefficients based on different assumptions concerning the sample composition are identified: (1) the sample is homogeneous with respect to change; (2) subgroups of patients who truly change by different amounts exist; (3) individual patients, many of whom truly change by different amounts exist. We present several analyses which illustrate a major conceptual conflict: the signal (a measure\u27s true ability to detect change) for some of these coefficients appears in the noise term (measurement error) of the others. We speculate that this dilemma occurs as a result of insufficient preparatory work such as pilot studies to establish the likely change characteristic of the patient population of interest. Uncertainty in the choice of change coefficient could be overcome by conducting pilot studies to ascertain the likely change characteristic of the population of interest. Once the population\u27s change characteristic is identified, the choice of change coefficient should be clear

The Fit-HaNSA Demonstrates Reliability and Convergent Validity of Functional Performance in Patients with Shoulder Disorders

Study DesignPsychometric study design. Objectives To assess the test-retest reliability and convergent validity of the Functional Impairment Test-Hand and Neck/Shoulder/Arm (FIT-HaNSA) in patients with shoulder disorders. Background Performance tests that assess functional ability of patients with shoulder disorders can provide useful information for making clinical or return-to-activity decisions. No performance-based shoulder test has yet demonstrated sufficient relevance or clinical measurement properties. The FIT-HaNSA examines upper extremity performance during repetitive tasks that emphasize shoulder reaching and static postures and, therefore, has greater relevance for assessing performance. Methods Thirty-six patients with shoulder disorders and 65 healthy controls were recruited for the study. The FIT-HaNSA, the Disabilities of the Arm, Shoulder and Hand questionnaire, the Shoulder Pain and Disability Index, isometric shoulder strength, and shoulder range of motion were assessed at baseline and repeated 2 to 7 days later. Test-retest reliability was described using intraclass correlation coefficient (ICC2,1) and standard error of measurement. Pearson correlation coefficients were used to examine the level of association between the FIT-HaNSA scores and the other measures. Results The ICCs for test-retest reliability for the FIT-HaNSA ranged from 0.89 to 0.97 in the patient group and 0.79 to 0.91 in the control group. The FIT-HaNSA showed high correlation with the Disabilities of the Arm, Shoulder and Hand and the Shoulder Pain and Disability Index, and moderate correlations with shoulder range of motion and muscle strength. Conclusion The FIT-HaNSA demonstrated high test-retest reliability and convergent validity with other related outcomes in patients with shoulder disorders. Further longitudinal studies are required to evaluate the responsiveness of the FIT-HaNSA in patients with different upper extremity conditions. J Orthop Sports Phys Ther 2012;42(5):455–464, Epub 25 January 2012. doi:10.2519/jospt.2012.379

Optimizing Reliability of Digital Inclinometer and Flexicurve Ruler Measures of Spine Curvatures in Postmenopausal Women with Osteoporosis of the Spine: An Illustration of the Use of Generalizability Theory

The study illustrates the application of generalizability theory (G-theory) to identify measurement protocols that optimize reliability of two clinical methods for assessing spine curvatures in women with osteoporosis. Triplicate measures of spine curvatures were acquired for 9 postmenopausal women with spine osteoporosis by two raters during a single visit using a digital inclinometer and a flexicurve ruler. G-coefficients were estimated using a G-study, and a measurement protocol that optimized inter-rater and inter-trial reliability was identified using follow-up decision studies. The G-theory provides reliability estimates for measurement devices which can be generalized to different clinical contexts and/or measurement designs

Characterizing the recovery trajectories of knee range of motion for one year after total knee replacement

DESIGN: Retrospective analysis of routinely collected clinical data. OBJECTIVE: This study modeled the recovery in knee flexion and extension range of motion (ROM) over 1 year after total knee replacement (TKR). BACKGROUND: Recovery after TKR has been characterized for self-reported pain and functional status. Literature describing target knee ROM at different follow-up periods after TKR is scarce. METHODS: Data were extracted for patients who had undergone TKR at a tertiary care hospital at 2, 8, 12, 26, and 52 weeks after TKR. A linear mixed-effects growth model was constructed that investigated the following covariates age, sex, pre-TKR range, body mass index, duration of symptoms, and their interaction with weeks post TKR. RESULTS: Of the 559 patients included (age 64.8 ± 8.5 years), 370 were women and 189 were men. Knee ROM showed the greatest change during the first 12 weeks after TKR, plateauing by 26 weeks. For an average patient, knee flexion increased from approximately 100º 2 weeks post TKR to 117º 52 weeks post TKR. Knee extension increased from approximately 3º knee flexion 2 weeks post TKR to 1º flexion 52 weeks post TKR. CONCLUSIONS: The results showed that the maximum gains in knee ROM should be expected within the first 12 weeks with small changes occurring up to 26 weeks after TKR. In addition, age and presurgery knee ROM are associated with the gains in knee ROM and should be factored into the estimation of expected knee ROM at a given follow-up interval after TKR

Modeling early recovery of physical function following hip and knee arthroplasty

BACKGROUND: Information on early recovery after arthroplasty is needed to help benchmark progress and make appropriate decisions concerning patient rehabilitation needs. The purpose of this study was to model early recovery of physical function in patients undergoing total hip (THA) and knee (TKA) arthroplasty, using physical performance and self-report measures. METHODS: A sample of convenience of 152 subjects completed testing, of which 69 (mean age: 66.77 ± 8.23 years) underwent THA and 83 (mean age: 60.25 ± 11.19 years) TKA. Postoperatively, patients were treated using standardized care pathways and rehabilitation protocols. Using a repeated measures design, patients were assessed at multiple time points over the first four postoperative months. Outcome measures included the Lower Extremity Function Scale (LEFS), the physical function subscale of the Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC PF), the 6 minute walk test (6 MWT), timed up and go test (TUG) and a timed stair test (ST). Average recovery curves for each of the measures were characterized using hierarchical linear modeling. Predictors of recovery were sequentially modeled after validation of the basic developmental models. RESULTS: Slopes of recovery were greater in the first 6 to 9 weeks with a second-degree polynomial growth term (weeks squared) providing a reasonable fit for the data over the study interval. Different patterns of recovery were observed between the self-report measures of physical function and the performance measures. In contrast to the models for the WOMAC PF and the LEFS, site of arthroplasty was a significant predictor (p = 0.001) in all of the physical performance measure models with the patients post TKA initially demonstrating higher function. Site of arthroplasty (p = 0.025) also predicted the rate of change for patients post THA and between 9 to 11 weeks after surgery, the THA group surpassed the function of the patients post TKA. CONCLUSION: Knowledge about the predicted growth curves will assist clinicians in referencing patient progress, and determining the critical time points for measuring change. The study has contributed further evidence to highlight the benefit of using physical performance measures to learn about the patients' actual level of disability

Identification and functional analysis of SKA2 interaction with the glucocorticoid receptor

Glucocorticoid (GC) receptors (GRs) have profound anti-survival effects on human small cell lung cancer (SCLC). To explore the basis of these effects, protein partners for GRs were sought using a yeast two-hybrid screen. We discovered a novel gene, FAM33A, subsequently identified as a SKA1 partner and involved in mitosis, and so renamed Ska2. We produced an anti-peptide antibody that specifically recognized full-length human SKA2 to measure expression in human cell lines and tissues. There was a wide variation in expression across multiple cell lines, but none was detected in the liver cell line HepG2. A xenograft model of human SCLC had intense staining and archival tissue revealed SKA2 in several human lung and breast tumours. SKA2 was found in the cytoplasm, where it co-localized with GR, but nuclear expression of SKA2 was seen in breast tumours. SKA2 overexpression increased GC transactivation in HepG2 cells while SKA2 knockdown in A549 human lung epithelial cells decreased transactivation and prevented dexamethasone inhibition of proliferation. GC treatment decreased SKA2 protein levels in A549 cells, as did Staurosporine, phorbol ester and trichostatin A; all agents that inhibit cell proliferation. Overexpression of SKA2 potentiated the proliferative response to IGF-I exposure, and knockdown with shRNA caused cells to arrest in mitosis. SKA2 has recently been identified in HeLa S3 cells as part of a complex, which is critical for spindle checkpoint silencing and exit from mitosis. Our new data show involvement in cell proliferation and GC signalling, with implications for understanding how GCs impact on cell fate

Assessing stability and change of four performance measures: a longitudinal study evaluating outcome following total hip and knee arthroplasty

BACKGROUND: Physical performance measures play an important role in the measurement of outcome in patients undergoing hip and knee arthroplasty. However, many of the commonly used measures lack information on their psychometric properties in this population. The purposes of this study were to examine the reliability and sensitivity to change of the six minute walk test (6MWT), timed up and go test (TUG), stair measure (ST), and a fast self-paced walk test (SPWT) in patients with hip or knee osteoarthritis (OA) who subsequently underwent total joint arthroplasty. METHODS: A sample of convenience of 150 eligible patients, part of an ongoing, larger observational study, was selected. This included 69 subjects who had a diagnosis of hip OA and 81 diagnosed with knee OA with an overall mean age of 63.7 ± 10.7 years. Test-retest reliability, using Shrout and Fleiss Type 2,1 intraclass correlations (ICCs), was assessed preoperatively in a sub-sample of 21 patients at 3 time points during the waiting period prior to surgery. Error associated with the measures' scores and the minimal detectable change at the 90% confidence level was determined. A construct validation process was applied to evaluate the measures' abilities to detect deterioration and improvement at two different time points post-operatively. The standardized response mean (SRM) was used to quantify change for all measures for the two change intervals. Bootstrapping was used to estimate the 95% confidence intervals (CI) for the SRMs. RESULTS: The ICCs (95% CI) were as follows: 6MWT 0.94 (0.88,0.98), TUG 0.75 (0.51, 0.89), ST 0.90 (0.79, 0.96), and the SPWT 0.91 (0.81, 0.97). Standardized response means varied from .79 to 1.98, being greatest for the ST and 6MWT over the studied time intervals. CONCLUSIONS: The test-retest estimates of the 6MWT, ST, and the SPWT met the requisite standards for making decisions at the individual patient level. All measures were responsive to detecting deterioration and improvement in the early postoperative period

Inter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) Checklist

<p>Abstract</p> <p>Background</p> <p>The COSMIN checklist is a tool for evaluating the methodological quality of studies on measurement properties of health-related patient-reported outcomes. The aim of this study is to determine the inter-rater agreement and reliability of each item score of the COSMIN checklist (n = 114).</p> <p>Methods</p> <p>75 articles evaluating measurement properties were randomly selected from the bibliographic database compiled by the Patient-Reported Outcome Measurement Group, Oxford, UK. Raters were asked to assess the methodological quality of three articles, using the COSMIN checklist. In a one-way design, percentage agreement and intraclass kappa coefficients or quadratic-weighted kappa coefficients were calculated for each item.</p> <p>Results</p> <p>88 raters participated. Of the 75 selected articles, 26 articles were rated by four to six participants, and 49 by two or three participants. Overall, percentage agreement was appropriate (68% was above 80% agreement), and the kappa coefficients for the COSMIN items were low (61% was below 0.40, 6% was above 0.75). Reasons for low inter-rater agreement were need for subjective judgement, and accustom to different standards, terminology and definitions.</p> <p>Conclusions</p> <p>Results indicated that raters often choose the same response option, but that it is difficult on item level to distinguish between articles. When using the COSMIN checklist in a systematic review, we recommend getting some training and experience, completing it by two independent raters, and reaching consensus on one final rating. Instructions for using the checklist are improved.</p

The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study

BACKGROUND: Aim of the COSMIN study (COnsensus-based Standards for the selection of health status Measurement INstruments) was to develop a consensus-based checklist to evaluate the methodological quality of studies on measurement properties. We present the COSMIN checklist and the agreement of the panel on the items of the checklist. METHODS: A four-round Delphi study was performed with international experts (psychologists, epidemiologists, statisticians and clinicians). Of the 91 invited experts, 57 agreed to participate (63%). Panel members were asked to rate their (dis)agreement with each proposal on a five-point scale. Consensus was considered to be reached when at least 67% of the panel members indicated 'agree' or 'strongly agree'. RESULTS: Consensus was reached on the inclusion of the following measurement properties: internal consistency, reliability, measurement error, content validity (including face validity), construct validity (including structural validity, hypotheses testing and cross-cultural validity), criterion validity, responsiveness, and interpretability. The latter was not considered a measurement property. The panel also reached consensus on how these properties should be assessed. CONCLUSIONS: The resulting COSMIN checklist could be useful when selecting a measurement instrument, peer-reviewing a manuscript, designing or reporting a study on measurement properties, or for educational purposes.This study was financially supported by the EMGO Institute for Health and Care Research, VU University Medical Center, Amsterdam, and the Anna Foundation, Leiden, The Netherlands

Does parallel item content on WOMAC's Pain and Function Subscales limit its ability to detect change in functional status?

BACKGROUND: Although the Western Ontario and McMaster University Osteoarthritis Index (WOMAC) is considered the leading outcome measure for patients with osteoarthritis of the lower extremity, recent work has challenged its factorial validity and the physical function subscale's ability to detect valid change when pain and function display different profiles of change. This study examined the etiology of the WOMAC's physical function subscale's limited ability to detect change in the presence of discordant changes for pain and function. We hypothesized that the duplication of some items on the WOMAC's pain and function subscales contributed to this shortcoming. METHODS: Two eight-item physical function scales were abstracted from the WOMAC's 17-item physical function subscale: one contained activities and themes that were duplicated on the pain subscale (SIMILAR-8); the other version avoided overlapping activities (DISSIMILAR-8). Factorial validity of the shortened measures was assessed on 310 patients awaiting hip or knee arthroplasty. The shortened measures' abilities to detect change were examined on a sample of 104 patients following primary hip or knee arthroplasty. The WOMAC and three performance measures that included activity specific pain assessments – 40 m walk test, stair test, and timed-up-and-go test – were administered preoperatively, within 16 days of hip or knee arthroplasty, and at an interval of greater than 20 days following the first post-surgical assessment. Standardized response means were used to quantify change. RESULTS: The SIMILAR-8 did not demonstrate factorial validity; however, the factorial structure of the DISSIMILAR-8 was supported. The time to complete the performance measures more than doubled between the preoperative and first postoperative assessments supporting the theory that lower extremity functional status diminished over this interval. The DISSIMILAR-8 detected this deterioration in functional status; however, no significant change was noted for the SIMILAR-8. The WOMAC pain scale demonstrated a slight reduction in pain and the performance specific pain measures did not reflect a change in pain. All measures showed substantial improvement over the second assessment interval. CONCLUSIONS: These findings support the hypothesis that activity overlap on the pain and function subscales plays a causal role in limiting the WOMAC physical function subscale's ability to detect change