The fading of reported effectiveness. A meta-analysis of randomised controlled trials

BACKGROUND: The "real" effect size of a medical therapy is constant over time. In contrast, the effect size reported in randomised controlled trials (RCTs) may change over time because the sum of all kinds of bias influencing the reported effectiveness is not necessarily constant. As this would affect the validity of meta-analyses, we tested the hypothesis that the reported effect size decreases over time. Furthermore, we tested three hypotheses that would explain a possible change. METHODS: Because of well established outcome measures, the lipid-lowering drugs Pravastatin and Atorvastatin (serum low-density lipoprotein cholesterol, LDL-C) and the anti-glaucoma drugs Timolol and Latanoprost (intraocular pressure, IOP) were chosen for this investigation. Studies were identified by a standardized MEDLINE search. RCTs investigating the above identified medications administered as monotherapy, and in defined dosages, were included. Publication year, baseline (= pre-treatment value in the treatment group of interest) and post intervention means, number of patients and the assignment to experimental or control group were extracted for each study. RESULTS: A total of 625 citations were screened; 206 met the inclusion criteria. The reported effect size of Pravastatin (change of reported effect size in five years: -3.22% LDL-C, P < .0001), Timolol (-0.56 mmHg, P < .0001) and Latanoprost (-1.78 mmHg, P = .0074) decreased over time, while there was no significant change for Atorvastatin (+0.31% LDL-C, P = .8618). Multiple regression analysis showed that baseline values were the most important influencing factor; study size or treatment group did not play a significant role. CONCLUSION: The effectiveness of medical therapies reported in RCTs decreases over time in three of the four investigated pharmaceuticals, caused mainly by baseline differences. We call this phenomenon "fading of reported effectiveness". Under this condition the validity of a meta-analysis may be impaired. Therefore we propose to observe this phenomenon in future meta-analyses in order to guarantee a maximum of transparency

Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews

BACKGROUND: Our objective was to develop an instrument to assess the methodological quality of systematic reviews, building upon previous tools, empirical evidence and expert consensus. METHODS: A 37-item assessment tool was formed by combining 1) the enhanced Overview Quality Assessment Questionnaire (OQAQ), 2) a checklist created by Sacks, and 3) three additional items recently judged to be of methodological importance. This tool was applied to 99 paper-based and 52 electronic systematic reviews. Exploratory factor analysis was used to identify underlying components. The results were considered by methodological experts using a nominal group technique aimed at item reduction and design of an assessment tool with face and content validity. RESULTS: The factor analysis identified 11 components. From each component, one item was selected by the nominal group. The resulting instrument was judged to have face and content validity. CONCLUSION: A measurement tool for the 'assessment of multiple systematic reviews' (AMSTAR) was developed. The tool consists of 11 items and has good face and content validity for measuring the methodological quality of systematic reviews. Additional studies are needed with a focus on the reproducibility and construct validity of AMSTAR, before strong recommendations can be made on its use

Quantitative Characterization of the Filiform Mechanosensory Hair Array on the Cricket Cercus

Crickets and other orthopteran insects sense air currents with a pair of abdominal appendages resembling antennae, called cerci. Each cercus in the common house cricket Acheta domesticus is approximately 1 cm long, and is covered with 500 to 750 filiform mechanosensory hairs. The distribution of the hairs on the cerci, as well as the global patterns of their movement vectors, have been characterized semi-quantitatively in studies over the last 40 years, and have been shown to be very stereotypical across different animals in this species. Although the cercal sensory system has been the focus of many studies in the areas of neuroethology, development, biomechanics, sensory function and neural coding, there has not yet been a quantitative study of the functional morphology of the receptor array of this important model system.We present a quantitative characterization of the structural characteristics and functional morphology of the cercal filiform hair array. We demonstrate that the excitatory direction along each hair's movement plane can be identified by features of its socket that are visible at the light-microscopic level, and that the length of the hair associated with each socket can also be estimated accurately from a structural parameter of the socket. We characterize the length and directionality of all hairs on the basal half of a sample of three cerci, and present statistical analyses of the distributions.The inter-animal variation of several global organizational features is low, consistent with constraints imposed by functional effectiveness and/or developmental processes. Contrary to previous reports, however, we show that the filiform hairs are not re-identifiable in the strict sense

Three-Armed Trials Including Placebo and No-Treatment Groups May Be Subject to Publication Bias: Systematic Review

Background: It has been argued that placebos may not have important clinical impacts in general. However, there is increasing evidence of a publication bias among trials published in journals. Therefore, we explored the potential for publication bias in randomized trials with active treatment, placebo, and no-treatment groups. Methods: Three-armed randomized trials of acupuncture, acupoint stimulation, and transcutaneous electrical stimulation were obtained from electronic databases. Effect sizes between treatment and placebo groups were calculated for treatment effect, and effect sizes between placebo and no-treatment groups were calculated for placebo effect. All data were then analyzed for publication bias. Results: For the treatment effect, small trials with fewer than 100 patients per arm showed more benefits than large trials with at least 100 patients per arm in acupuncture and acupoint stimulation. For the placebo effect, no differences were found between large and small trials. Further analyses showed that the treatment effect in acupuncture and acupoint stimulation may be subject to publication bias because study design and any known factors of heterogeneity were not associated with the small study effects. In the simulation, the magnitude of the placebo effect was smaller than that calculated after considering publication bias. Conclusions: Randomized three-armed trials, which are necessary for estimating the placebo effect, may be subject t

Corticosteroid therapy for critically ill patients with COVID-19: a structured summary of a study protocol for a prospective meta-analysis of randomized trials

Objectives Primary objective: To estimate the effect of corticosteroids compared with usual care or placebo on mortality up to 28 days after randomization. Secondary objectives: To examine whether the effect of corticosteroids compared with usual care or placebo on mortality up to 28 days after randomization varies between subgroups related to treatment characteristics, disease severity at the time of randomization, patient characteristics, or risk of bias. To examine the effect of corticosteroids compared with usual care or placebo on serious adverse events. Study design Prospective meta-analysis of randomized controlled trials. Both placebo-controlled and open-label trials are eligible. Participants Hospitalised, critically ill patients with suspected or confirmed COVID-19. Intervention and comparator Intervention groups will have received therapeutic doses of a steroid (dexamethasone, hydrocortisone or methylprednisolone) with IV or oral administration immediately after randomization. The comparator groups will have received standard of care or usual care or placebo. Main outcome All-cause mortality up to 28 days after randomization. Search methods Systematic searching of clinicaltrials.gov, EudraCT, the WHO ISRCTN registry, and the Chinese clinical trials registry. Additionally, research and WHO networks will be asked for relevant trials. Risk of bias assessments These will be based on the Cochrane RoB 2 tool, and will use structured information provided by the trial investigators on a form designed for this prospective meta-analysis. Summary of findings We will use GRADE to assess the certainty of the evidence. Statistical analyses Trial investigators will provide data on the numbers of participants who did and did not experience each outcome according to intervention group, overall and in specified subgroups. We will conduct fixed-effect (primary analysis) and random-effects (Paule-Mandel estimate of heterogeneity and Hartung-Knapp adjustment) meta-analyses. We will quantify inconsistency in effects between trials using I2 statistics. Evidence for subgroup effects will be quantified by ratios of odds ratios comparing effects in the subgroups, and corresponding interaction p-values. Comparisons between subgroups defined by trial characteristics will be made using random-effects meta-regression. Comparisons between subgroups defined by patient characteristics will be made by estimating trial-specific ratios of odds ratios comparing intervention effects between subgroups then combining these using random-effects meta-analysis. Steroid interventions will be classified as high or low dose according to whether the dose is greater or less than or equal to 400 mg hydrocortisone per day or equivalent. We will use network meta-analysis methods to make comparisons between the effects of high and low dose steroid interventions (because one trial randomized participants to both low and high dose steroid arms). PROSPERO registration number CRD42020197242 Full protocol The full protocol for this prospective meta-analysis is attached as an additional file, accessible from the Trials website (Additional file 1). To expedite dissemination of this material, the familiar formatting has been eliminated; this Letter serves as a summary of the key elements of the full protocol for the systematic review.</p