The problem with health measurement

In this review we discuss health measurement with a focus on psychometric methods and methodology. In particular, we examine some of the key issues currently facing the use of clinician and patient rating scales to measure the health outcomes of disease and treatment. We present three key facts and flag one crucial problem. First, the numbers generated by scales are increasingly used as the measurements of the central dependent variables upon which clinical decisions are frequently made. The rising profile of rating scales has significant implications for scale construction, evaluation, and selection, as well as for interpreting studies. Second, rating scale science is well established. Therefore, it is important to learn the lessons from those who have built and established the science over the last century. Finally, the goal of a rating scale is to measure. As such, over the last half century, developments in rating scale (psychometric) methods have caused a refocus in the way we should be measuring health. In particular, newer methods have significant clinical advantages over traditional approaches. These should be seriously considered for inclusion in everyday practice. This leads us to the central problem with health measurement, which is that we cannot currently be sure what most rating scales are measuring. This is because the methods we have in place to ensure the validity of rating scales fall short of what is actually required. We expand on this point, and provide some potential routes forward to help address this important problem

The Cervical Dystonia Impact Profile (CDIP-58): Can a Rasch developed patient reported outcome measure satisfy traditional psychometric criteria?

<p>Abstract</p> <p>Background</p> <p>The United States Food and Drug Administration (FDA) are currently producing guidelines for the scientific adequacy of patient reported outcome measures (PROMs) in clinical trials, which will have implications for the selection of scales used in future clinical trials. In this study, we examine how the Cervical Dystonia Impact Profile (CDIP-58), a rigorous Rasch measurement developed neurologic PROM, stands up to traditional psychometric criteria for three reasons: 1) provide traditional psychometric evidence for the CDIP-58 in line with proposed FDA guidelines; 2) enable researchers and clinicians to compare it with existing dystonia PROMs; and 3) help researchers and clinicians bridge the knowledge gap between old and new methods of reliability and validity testing.</p> <p>Methods</p> <p>We evaluated traditional psychometric properties of data quality, scaling assumptions, targeting, reliability and validity in a group of 391 people with CD. The main outcome measures used were the CDIP-58, Medical Outcome Study Short Form-36, the 28-item General Health Questionnaire, and Hospital and Anxiety and Depression Scale.</p> <p>Results</p> <p>A total of 391 people returned completed questionnaires (corrected response rate 87%). Analyses showed: 1) data quality was high (low missing data ≤ 4%, subscale scores could be computed for > 96% of the sample); 2) item groupings passed tests for scaling assumptions; 3) good targeting (except for the Sleep subscale, ceiling effect = 27%); 4) good reliability (Cronbach's alpha ≥ 0.92, test-retest intraclass correlations ≥ 0.83); and 5) validity was supported.</p> <p>Conclusion</p> <p>This study has shown that new psychometric methods can produce a PROM that stands up to traditional criteria and supports the clinical advantages of Rasch analysis.</p

Patient-orientated longitudinal study of multiple sclerosis in south west England (The South West Impact of Multiple Sclerosis Project, SWIMS) 1: protocol and baseline characteristics of cohort

<p>Abstract</p> <p>Background</p> <p>There is a need for greater understanding of the impact of multiple sclerosis (MS) from the perspective of individuals with the condition. The South West Impact of MS Project (SWIMS) has been designed to improve understanding of disease impact using a patient-centred approach. The purpose is to (1) develop improved measurement instruments for clinical trials, (2) evaluate longitudinal performance of a variety of patient-reported outcome measures, (3) develop prognostic predictors for use in individualising drug treatment for patients, particularly early on in the disease course.</p> <p>Methods</p> <p>This is a patient-centred, prospective, longitudinal study of multiple sclerosis and clinically isolated syndrome (CIS) in south west England. The study area comprises two counties with a population of approximately 1.7 million and an estimated 1,800 cases of MS. Self-completion questionnaires are administered to participants every six months (for people with MS) or 12 months (CIS). Here we present descriptive statistics of the baseline data provided by 967 participants with MS.</p> <p>Results</p> <p>Seventy-five percent of those approached consented to participate. The male:female ratio was 1.00:3.01 (n = 967). Average (standard deviation) age at time of entry to SWIMS was 51.6 (11.5) years (n = 961) and median (interquartile range) time since first symptom was 13.3 (6.8 to 24.5) years (n = 934). Fatigue was the most commonly reported symptom, with 80% of participants experiencing fatigue at baseline. Although medication use for symptom control was common, there was little evidence of effectiveness, particularly for fatigue. Nineteen percent of participants were unable to classify their subtype of MS. When patient-reported subtype was compared to neurologist assessment for a sample of participants (n = 396), agreement in disease sub-type was achieved in 63% of cases. There were 836 relapses, reported by 931 participants, in the twelve months prior to baseline. Twenty-three percent of the relapsing-remitting group and 12% of the total sample were receiving disease-modifying therapy at baseline.</p> <p>Conclusions</p> <p>Demographics of this sample were similar to published data for the UK. Overall, the results broadly reflect clinical experience in confirming high symptom prevalence, with relatively little complete symptom relief. Participants often had difficulty in defining MS relapses and their own MS type.</p

The size of the treatment effect: do patients and proxies agree?

Background: This study examined whether MS patients and proxy respondents agreed on change in disease impact, which was induced by treatment. This may be of interest in situations when patients suffer from limitations that interfere with reliable self-assessment, such as cognitive impairment.Methods: MS patients and proxies completed the Multiple Sclerosis Impact Scale (MSIS-29) before and after intravenous steroid treatment. Analyses focused on patient-proxy agreement between MSIS-29 change scores. Transition ratings were used to measure the patient's judgement of change and whether this change was reflected in the MSIS-29 change of patients and proxies. Receiver operating characteristic (ROC) analyses were also performed to examine the diagnostic properties of the MSIS-29 when completed by patients and proxies.Results: 42 patients and proxy respondents completed the MSIS-29 at baseline and follow-up. Patient-proxy differences between change scores on the physical and psychological MSIS-29 subscale were quite small, although large variability was found. The direction of mean change was in concordance with the transition ratings of the patients. Results of the ROC analyses of the MSIS-29 were similar when completed by patients (physical scale: AUC = 0.79, 95% CI: 0.65 - 0.93 and 0.66, 95% CI: 0.48 - 0.84 for the psychological scale) and proxies (physical scale: 0.80, 95% CI: 0.72 - 0.96 and 0.71, 95% CI: 0.56 - 0.87 for the psychological scale)Conclusion: Although the results need to be further explored in larger samples, these results do point towards possible use of proxy respondents to assess patient perceived treatment change at the group level

A Predictive Model for Corticosteroid Response in Individual Patients with MS Relapses

<div><p>Objectives</p><p>To derive a simple predictive model to guide the use of corticosteroids in patients with relapsing remitting MS suffering an acute relapse.</p><p>Materials and Methods</p><p>We analysed individual patient randomised controlled trial data (n=98) using a binary logistic regression model based on age, gender, baseline disability scores [physician-observed: expanded disability status scale (EDSS) and patient reported: multiple sclerosis impact scale 29 (MSIS-29)], and the time intervals between symptom onset or referral and treatment.</p><p>Results</p><p>Based on two a priori selected cut-off points (improvement in EDSS ≥ 0.5 and ≥ 1.0), we found that variables which predicted better response to corticosteroids after 6 weeks were younger age and lower MSIS-29 physical score at the time of relapse (model fit 71.2% - 73.1%).</p><p>Conclusions</p><p>This pilot study suggests two clinical variables which may predict the majority of the response to corticosteroid treatment in patients undergoing an MS relapse. The study is limited in being able to clearly distinguish factors associated with treatment response or spontaneous recovery and needs to be replicated in a larger prospective study.</p></div

Genetic risk and a primary role for cell-mediated immune mechanisms in multiple sclerosis.

Multiple sclerosis is a common disease of the central nervous system in which the interplay between inflammatory and neurodegenerative processes typically results in intermittent neurological disturbance followed by progressive accumulation of disability. Epidemiological studies have shown that genetic factors are primarily responsible for the substantially increased frequency of the disease seen in the relatives of affected individuals, and systematic attempts to identify linkage in multiplex families have confirmed that variation within the major histocompatibility complex (MHC) exerts the greatest individual effect on risk. Modestly powered genome-wide association studies (GWAS) have enabled more than 20 additional risk loci to be identified and have shown that multiple variants exerting modest individual effects have a key role in disease susceptibility. Most of the genetic architecture underlying susceptibility to the disease remains to be defined and is anticipated to require the analysis of sample sizes that are beyond the numbers currently available to individual research groups. In a collaborative GWAS involving 9,772 cases of European descent collected by 23 research groups working in 15 different countries, we have replicated almost all of the previously suggested associations and identified at least a further 29 novel susceptibility loci. Within the MHC we have refined the identity of the HLA-DRB1 risk alleles and confirmed that variation in the HLA-A gene underlies the independent protective effect attributable to the class I region. Immunologically relevant genes are significantly overrepresented among those mapping close to the identified loci and particularly implicate T-helper-cell differentiation in the pathogenesis of multiple sclerosis

Efficacy and Safety of Alemtuzumab Through 9 Years of Follow-up in Patients with Highly Active Disease: Post Hoc Analysis of CARE-MS I and II Patients in the TOPAZ Extension Study

Background: Alemtuzumab efficacy versus subcutaneous interferon-β-1a (SC IFNB-1a) was demonstrated over 2 years in patients with relapsing-remitting multiple sclerosis, with continued efficacy over 7 additional years. Alemtuzumab is included as a recommended treatment for patients with highly active disease (HAD) by the American Academy of Neurology Practice Guidelines, and the label indication in Europe was recently restricted to the treatment of HAD patients. There is currently no consensus definition for HAD, and alemtuzumab efficacy across various HAD definitions has not been explored previously. Objectives: In this post hoc analysis, we assess the efficacy and safety of alemtuzumab in Comparison of Alemtuzumab and Rebif® Efficacy in Multiple Sclerosis (CARE-MS) trial patients who met criteria for at least one of four separate definitions of HAD (one primary and three alternatives). Over 2 years, alemtuzumab-treated HAD patients were compared with SC IFNB-1a-treated HAD patients, with additional 7-year follow-up in patients from the alemtuzumab arm. Methods: Patients in the CARE-MS studies received either alemtuzumab (baseline: 5 days; 12 months later: 3 days) or SC IFNB-1a (3 times weekly). Alemtuzumab-treated patients who enrolled in the extensions could receive additional courses ≥ 12 months apart. Four definitions of HAD were applied to assess alemtuzumab efficacy: the pre-specified primary definition (two or more relapses in the year prior to baseline and at least one gadolinium [Gd]-enhancing lesion at baseline) and three alternative definitions that focused on relapse, magnetic resonance imaging (MRI), or prior treatment response criteria. Efficacy outcomes were annualized relapse rate, change in Expanded Disability Status Scale score, 6-month confirmed disability worsening, 6-month confirmed disability improvement, MRI disease activity, and brain volume change. Adverse events were summarized for HAD patients meeting the primary definition. Results: In the pooled CARE-MS population, 208 alemtuzumab-treated patients met the primary HAD definition. Annualized relapse rate was 0.27 in years 0–2 and 0.16 in years 3–9. Over 9 years, 62% of patients were free of 6-month confirmed disability worsening, 50% had 6-month confirmed disability improvement, and median cumulative change in brain volume was − 2.15%. During year 9, 62% had no evidence of disease activity, and 69% were free of MRI disease activity. Similar efficacy outcomes were observed using an alternative relapse-driven HAD definition. For patients meeting alternative HAD definitions focused on either higher MRI lesion counts or disease activity while on prior therapy, reduced efficacy for some endpoints was seen. Safety was consistent with the overall CARE-MS population through year 9. Conclusions: Over 9 years, alemtuzumab efficacy was maintained in CARE-MS HAD patients based on four HAD definitions. These results support intervention with alemtuzumab in patients with early indicators of HAD, including frequent relapse without high MRI activity. No safety signals were observed over 9 years that were unique to the HAD populations. ClinicalTrials.gov Identifiers: NCT00530348; NCT00548405; NCT00930553; NCT02255656

Assessing treatment outcomes in multiple sclerosis trials and in the clinical setting

Increasing numbers of drugs are being developed for the treatment of multiple sclerosis (MS). Measurement of relevant outcomes is key for assessing the efficacy of new drugs in clinical trials and for monitoring responses to disease-modifying drugs in individual patients. Most outcomes used in trial and clinical settings reflect either clinical or neuroimaging aspects of MS (such as relapse and accrual of disability or the presence of visible inflammation and brain tissue loss, respectively). However, most measures employed in clinical trials to assess treatment effects are not used in routine practice. In clinical trials, the appropriate choice of outcome measures is crucial because the results determine whether a drug is considered effective and therefore worthy of further development; in the clinic, outcome measures can guide treatment decisions, such as choosing a first-line disease-modifying drug or escalating to second-line treatment. This Review discusses clinical, neuroimaging and composite outcome measures for MS, including patient-reported outcome measures, used in both trials and the clinical setting. Its aim is to help clinicians and researchers navigate through the multiple options encountered when choosing an outcome measure. Barriers and limitations that need to be overcome to translate trial outcome measures into the clinical setting are also discussed