Reproducibility and responsiveness of evaluative outcome measures - Theoretical considerations illustrated by an empirical example

Objective: This article outlines basic methods for quantifying reproducibility and responsiveness of evaluative outcome measures. Methods: The background noise in stable patients provides the desired information to quantify the reproducibility. From this, the smallest real difference (SRD) for longitudinal differences can be derived. We propose to use the SRD to define responsiveness: from all patients who change according to an external criterion, we take the percentage that changes at least SRD on the outcome measure. A more complete picture of the responsiveness of the outcome measure arises when the receiver operating characteristic (ROC) is constructed, corresponding to all possible cut-off change scores. The proposed methods are illustrated with an empirical example. Results: In the illustration the methods appeared to be very useful and complemented each other. We could evaluate whether the observed change score was larger than that expected due to chance. With the methods it was possible to evaluate both the ability of an instrument to detect change if there is a real change in health status (sensitivity to change) and the ability to detect absence of change if there is no real change (specificity to change). Conclusion: We presented the use of SRDs and ROC curves for quantifying reproducibility and responsiveness. We started with the basic notions and arrived at methods that are both understandable and useful

A method for research programming in the field of evidence-based medicine

Objective: This paper presents the method we used in programming research on the efficacy of therapeutic interventions for nine chronic benign pain disorders. Methods: We started with an inventory to identify commonly applied interventions. For these interventions we searched the literature to identify gaps in evidence. First, we searched for recent reviews, of which we assessed the methodologic quality. If only reviews of poor or moderate quality were found, a new systematic review was recommended. When we found no recent reviews, we searched for randomized controlled trials (RCTs). If there was insufficient or inconsistent evidence, the performance of a new RCT was advised. This structured procedure resulted in a list with topics for which new systematic reviews were recommended and topics for which new RCTs were needed. These lists were the starting points for priority setting by four Centers for Pain Management and Research. All members of the centers were asked to state the priority of each topic. The resulting hierarchy of topics for their own center was discussed in a meeting in each center, giving participants the opportunity to elucidate their views and considerations. Results: The final result was a robust priority list for the need of research (systematic reviews and RCTs) on chronic benign pain syndromes. Discussion: The strength and weaknesses of this approach are discussed. This method of priority setting is by no means restricted to treatments for chronic pain; it is also a useful approach for programming research to enable evidence-based medicine in other fields of interest

Measuring sick leave: a comparison of self-reported data on sick leave and data from company records

The objective of this study was to compare sick leave data obtained from questionnaires with data from company records. During a period of 12 months, questionnaires were completed monthly for 6 months and then at 9 and 12 months. The sensitivity and specificity of questionnaires for detecting an episode of sick leave were determined, using the company records as a reference standard. In addition, the duration of sick leave episodes reported in the two data sets was compared. In this analysis, company records were not assumed to be superior, and agreement was assessed with intraclass correlation coefficients (ICCs). The sensitivity of questionnaires for detecting an episode of sick leave was 55% (95% CI = 0.50-0.60) and the specificity 83% (95% CI = 0.72-0.94). The ICC for all episodes was 0.58 (95% CI = 0.47-0.67). The only satisfactory ICC (0.87; 95% CI = 0.74-0.93) was found for the questionnaires at 9 and 12 months. No large systematic differences were found between the duration of episodes reported in the two data sets. In conclusion, in our study, the sensitivity of questionnaires for detecting an episode of sick leave was very low. Furthermore, when episodes were recalled, there was little agreement on the duration of the episode between questionnaire data and data in the company records. Based on these results and considering the risk of missing questionnaires, data on sick leave gathered from company records are clearly preferable as an outcome measure in research

Clinimetric properties of the motor activity log for the assessment of arm use in hemiparetic patients

Background and Purpose - The Motor Activity Log (MAL) is a semistructured interview for hemiparetic stroke patients to assess the use of their paretic arm and hand (amount of use [AOU]) and quality of movement [QOM]) during activities of daily living. Scores range from 0 to 5. The following clinimetric properties of the MAL were quantified: internal consistency (Cronbach α), test-retest agreement (Bland and Altman method), cross-sectional construct validity (correlation between AOU and QOM and with the Action Research Arm (ARA] test), longitudinal construct validity (correlation of change on the MAL during the intervention with a global change rating [GCR] and with change on the ARA), and responsiveness (effect size). Methods - Two baseline measurements 2 weeks apart and 1 follow-up measurement immediately after 2 weeks of intensive exercise therapy either with or without immobilization of the unimpaired arm (forced use) were performed in 56 chronic stroke patients. Results - Internal consistency was high (AOU: a=0.88; QOM: a=0.91). The limits of agreement were -0.70 to 0.85 and -0.61 to 0.71 for AOU and QOM, respectively. The correlation with the ARA score (Spearman p) was 0.63 (AOU and QOM). However, the improvement on the MAL during the intervention was only weakly related to the GCR and to the improvement on the ARA, Spearman p was between 0.16 and 0.22. The responsiveness ratio was 1.9 (AOU) and 2.0 (QOM). Conclusion - The MAL is internally consistent and relatively stable in chronic stroke patients not undergoing an intervention. The cross-sectional construct validity of the MAL is reasonable, but the results raise doubt about its longitudinal construct validity

Splinting vs surgery in the treatment of carpal tunnel syndrome - A randomized controlled trial

Carpal tunnel syndrome (CTS) can be treated with nonsurgical or surgical options. However, there is no consensus on the most effective method of treatment. To compare the short-term and long-term efficacy of splinting and surgery for relieving the symptoms of CTS. A randomized controlled trial conducted from October 1998 to April 2000 at 13 neurological outpatient clinics in the Netherlands. A total of 176 patients with clinically and electrophysiologically confirmed idiopathic CTS were assigned to wrist splinting during the night for at least 6 weeks (89 patients) or open carpal tunnel release (87 patients); 147 patients (84%) completed the final follow-up assessment 18 months after randomization. General improvement, number of nights waking up due to symptoms, and severity of symptoms. In the intention-to-treat analyses, surgery was more effective than splinting on all outcome measures. The success rates (based on general improvement) after 3 months were 80% for the surgery group (62/78 patients) vs 54% for the splinting group (46/86 patients), which is a difference of 26% (95% confidence interval [CI], 12%-40%; P <.001). After 18 months, the success rates increased to 90% for the surgery group (61/68 patients) vs 75% for the splinting group (59/79 patients), which is a difference of 15% (95% CI, 3%-27%; P =.02). However, by that time 41% of patients (32/79) in the splint group had also received the surgery treatment. Treatment with open carpal tunnel release surgery resulted in better outcomes than treatment with wrist splinting for patients with CT

Conservative treatment options for carpal tunnel syndrome: a systematic review of randomised controlled trials

Carpal tunnel syndrome (CTS) is a common disorder, for which various conservative treatment options are available. The objective of this study is to determine the efficacy of the various conservative treatment options for relieving the symptoms of CTS. Computer-aided searches of MEDLINE (1/1966 to 3/2000), EMBASE (1/1988 to 2/2000) and the Cochrane Controlled Trials Register (2000, issue 1) were conducted, together with reference checking. Included were randomised controlled trials evaluating the efficacy of conservative treatment options in a study population of CTS patients, with a full report published in English, German, French or Dutch. Two reviewers independently selected the studies. Fourteen randomised controlled trials were included in the review. Assessment of methodological quality and data-extraction was independently performed by two reviewers. A rating system, based on the number of studies and their methodological quality and findings, was used to determine the strength of the available evidence for the efficacy of the treatment. Diuretics, pyridoxine, non-steroidal anti-inflammatory drugs, yoga and laser-acupuncture seem to be ineffective in providing short-term symptom relief (varying levels of evidence) and steroid injections seem to be effective (limited evidence). There is conflicting evidence for the efficacy of ultrasound and oral steroids. For providing long-term relief from symptoms there is limited evidence that ultrasound is effective, and that splinting is less effective than surgery. In conclusion, there is still little known about the efficacy of most conservative treatment options for CTS. To establish stronger evidence more high quality trials are neede

The cost-effectiveness of a treatment based classification system for low back pain: design of a randomised controlled trial and economic evaluation

<p>Abstract</p> <p>Background</p> <p>Systematic reviews have shown that exercise therapy and spinal manipulation are both more effective for low back pain (LBP) than no treatment at all. However, the effects are at best modest. To enhance the clinical outcomes, recommendations are to improve the patient selection process, and to identify relevant subgroups to guide clinical decision-making. One of the systems that has potentials to improve clinical decision-making is a treatment-based classification system that is intended to identify those patients who are most likely to respond to direction-specific exercises, manipulation, or stabilisation exercises.</p> <p>Methods/Design</p> <p>The primary aim of this randomised controlled trial will be to assess the effectiveness of a classification-based system. A sample of 150 patients with subacute and chronic LBP who attend a private physical therapy clinic for treatment will be recruited. At baseline, all participants will undergo a standard evaluation by trained research physical therapists and will be classified into one of the following subgroups: direction-specific exercises, manipulation, or stabilisation. The patient will not be informed about the results of the examination. Patients will be randomly assigned to classification-based treatment or usual care according to the Dutch LBP guidelines, and will complete questionnaires at baseline, and 8, 26, and 52 weeks after the start of the treatment. The primary outcomes will be general perceived recovery, functional status, and pain intensity. Alongside this trial, an economic evaluation of cost-effectiveness and cost-utility will be conducted from a societal perspective.</p> <p>Discussion</p> <p>The present study will contribute to our knowledge about the effectiveness and cost-effectiveness of classification-based treatment in patients with LBP.</p> <p>Trial registration</p> <p>Trial registration number: NTR1176</p