Overstating the evidence - double counting in meta-analysis and related problems

Background: The problem of missing studies in meta-analysis has received much attention. Less attention has been paid to the more serious problem of double counting of evidence. Methods: Various problems in overstating the precision of results from meta-analyses are described and illustrated with examples, including papers from leading medical journals. These problems include, but are not limited to, simple double-counting of the same studies, double counting of some aspects of the studies, inappropriate imputation of results, and assigning spurious precision to individual studies. Results: Some suggestions are made as to how the quality and reliability of meta-analysis can be improved. It is proposed that the key to quality in meta-analysis lies in the results being transparent and checkable. Conclusions: Existing quality check lists for meta-analysis do little to encourage an appropriate attitude to combining evidence and to statistical analysis. Journals and other relevant organisations should encourage authors to make data available and make methods explicit. They should also act promptly to withdraw meta-analyses when mistakes are found

Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): Explanation and Elaboration

In this explanatory and elaboration document Mattias Egger and colleagues provide the meaning and rationale of each checklist item on the STROBE Statement

The fading of reported effectiveness. A meta-analysis of randomised controlled trials

BACKGROUND: The "real" effect size of a medical therapy is constant over time. In contrast, the effect size reported in randomised controlled trials (RCTs) may change over time because the sum of all kinds of bias influencing the reported effectiveness is not necessarily constant. As this would affect the validity of meta-analyses, we tested the hypothesis that the reported effect size decreases over time. Furthermore, we tested three hypotheses that would explain a possible change. METHODS: Because of well established outcome measures, the lipid-lowering drugs Pravastatin and Atorvastatin (serum low-density lipoprotein cholesterol, LDL-C) and the anti-glaucoma drugs Timolol and Latanoprost (intraocular pressure, IOP) were chosen for this investigation. Studies were identified by a standardized MEDLINE search. RCTs investigating the above identified medications administered as monotherapy, and in defined dosages, were included. Publication year, baseline (= pre-treatment value in the treatment group of interest) and post intervention means, number of patients and the assignment to experimental or control group were extracted for each study. RESULTS: A total of 625 citations were screened; 206 met the inclusion criteria. The reported effect size of Pravastatin (change of reported effect size in five years: -3.22% LDL-C, P < .0001), Timolol (-0.56 mmHg, P < .0001) and Latanoprost (-1.78 mmHg, P = .0074) decreased over time, while there was no significant change for Atorvastatin (+0.31% LDL-C, P = .8618). Multiple regression analysis showed that baseline values were the most important influencing factor; study size or treatment group did not play a significant role. CONCLUSION: The effectiveness of medical therapies reported in RCTs decreases over time in three of the four investigated pharmaceuticals, caused mainly by baseline differences. We call this phenomenon "fading of reported effectiveness". Under this condition the validity of a meta-analysis may be impaired. Therefore we propose to observe this phenomenon in future meta-analyses in order to guarantee a maximum of transparency

Multiscale photosynthetic exciton transfer

Photosynthetic light harvesting provides a natural blueprint for bioengineered and biomimetic solar energy and light detection technologies. Recent evidence suggests some individual light harvesting protein complexes (LHCs) and LHC subunits efficiently transfer excitons towards chemical reaction centers (RCs) via an interplay between excitonic quantum coherence, resonant protein vibrations, and thermal decoherence. The role of coherence in vivo is unclear however, where excitons are transferred through multi-LHC/RC aggregates over distances typically large compared with intra-LHC scales. Here we assess the possibility of long-range coherent transfer in a simple chromophore network with disordered site and transfer coupling energies. Through renormalization we find that, surprisingly, decoherence is diminished at larger scales, and long-range coherence is facilitated by chromophoric clustering. Conversely, static disorder in the site energies grows with length scale, forcing localization. Our results suggest sustained coherent exciton transfer may be possible over distances large compared with nearest-neighbour (n-n) chromophore separations, at physiological temperatures, in a clustered network with small static disorder. This may support findings suggesting long-range coherence in algal chloroplasts, and provides a framework for engineering large chromophore or quantum dot high-temperature exciton transfer networks.Comment: 9 pages, 6 figures. A significantly updated version is now published online by Nature Physics (2012

Intracluster correlation coefficients in cluster randomized trials: empirical insights into how should they be reported

BACKGROUND: Increasingly, researchers are recognizing that there are many situations where the use of a cluster randomized trial may be more appropriate than an individually randomized trial. Similarly, the need for appropriate standards of reporting of cluster trials is more widely acknowledged. METHODS: In this paper, we describe the results of a survey to inform the appropriate reporting of the intracluster correlation coefficient (ICC) – the statistical measure of the clustering effect associated with a cluster randomized trial. RESULTS: We identified three dimensions that should be considered when reporting an ICC – a description of the dataset (including characteristics of the outcome and the intervention), information on how the ICC was calculated, and information on the precision of the ICC. CONCLUSIONS: This paper demonstrates the development of a framework for the reporting of ICCs. If adopted into routine practice, it has the potential to facilitate the interpretation of the cluster trial being reported and should help the development of new trials in the area

The Paired Availability Design for Historical Controls

BACKGROUND: Although a randomized trial represents the most rigorous method of evaluating a medical intervention, some interventions would be extremely difficult to evaluate using this study design. One alternative, an observational cohort study, can give biased results if it is not possible to adjust for all relevant risk factors. METHODS: A recently developed and less well-known alternative is the paired availability design for historical controls. The paired availability design requires at least 10 hospitals or medical centers in which there is a change in the availability of the medical intervention. The statistical analysis involves a weighted average of a simple "before" versus "after" comparison from each hospital or medical center that adjusts for the change in availability. RESULTS: We expanded requirements for the paired availability design to yield valid inference. (1) The hospitals or medical centers serve a stable population. (2) Other aspects of patient management remain constant over time. (3) Criteria for outcome evaluation are constant over time. (4) Patient preferences for the medical intervention are constant over time. (5) For hospitals where the intervention was available in the "before" group, a change in availability in the "after group" does not change the effect of the intervention on outcome. CONCLUSION: The paired availability design has promise for evaluating medical versus surgical interventions, in which it is difficult to recruit patients to a randomized trial

The effectiveness of ENAR® for the treatment of chronic neck pain in Australian adults: a preliminary single-blind, randomised controlled trial

<p>Abstract</p> <p>Background</p> <p>Current evidence on electrotherapies for the management of chronic neck pain is either lacking or conflicting. New therapeutic devices being introduced to the market should be investigated for their effectiveness and efficacy. The ENAR^®(Electro Neuro Adaptive Regulator) therapy device combines Western biofeedback with Eastern energy medicine.</p> <p>Methods</p> <p>A small, preliminary randomised and controlled single-blinded trial was conducted on 24 participants (ten males, 14 females) between the ages of 18 to 50 years (median age of 40.5) Consent was obtained and participants were randomly allocated to one of three groups – ENAR, Transcutaneous Electrical Nerve Stimulation (TENS), or control therapy – to test the hypothesis that ENAR therapy would result in superior pain reduction/disability and improvements in neck function compared with TENS or control intervention. The treatment regimen included twelve 15-minute treatment sessions over a six week period, followed by two assessment periods. Visual Analogue Scale (VAS) pain scores, Neck Disability Index (NDI) scores, Patient Specific Functional Scale (PSFS) scores and Short Form 36v1 (SF-36) quality of life scores reported by participants were collected at each of the assessments points throughout the trial (0, 6, 12, 18 and 24 weeks).</p> <p>Results</p> <p>Eligible participants (n = 30) were recruited and attended clinic visits for 6 months from the time of randomisation. Final trial sample (n = 24) comprised 9 within the ENAR group, 7 within the TENS group and 8 within the control group. With an overall study power of 0.92, the ENAR group showed a decrease in mean pain score from measurement at time zero (5.0 ± 0.79 95%CI) to the first follow-up measurement at six weeks (1.4 ± 0.83 95%CI). Improvement was maintained until week 24 (1.75 ± 0.9 95%CI). The TENS and control groups showed consistent pain levels throughout the trial (3.4 ± 0.96 95%CI and 4.1 ± 0.9 95%CI respectively). Wald analysis for pain intensity was significant for the ENAR group (p = 0.01). Six month NDI scores showed the disability level of the ENAR group (11.3 ± 4.5 95%CI) was approximately half that of either the TENS (22.9 ± 4.8 95%CI) or the control (29.4 ± 4.5 95%CI) groups. NDI analysis using the Wald method, indicated significant reductions in disability only for the ENAR group (p = 0.022). PSFS results also demonstrated significantly better performance of ENAR (p = 0.001) compared to both alternative interventions. Differential means analysis of the SF-36 results favoured ENAR for all of the subscales. Six of the initial 30 participants discontinued the trial protocol.</p> <p>Conclusion</p> <p>ENAR therapy participants reported a significant reduction in the intensity of neck pain (VAS) and disability (NDI), as well as a significant increased function (PSFS) and overall quality of life (SF-36) than TENS or control intervention participants. Due to the modest sample size and restricted cohort characteristics, future larger and more comprehensive trials are required to better evaluate the potential efficacy of the ENAR device in a more widely distributed sample population.</p> <p>Trial Registration</p> <p>This study has been registered with the Australian Clinical Trials Registry (ACTR): ACTRN012606000438550.</p

Tumor Necrosis Factor-Alpha G308α Gene Polymorphism and Essential Hypertension: A Meta-Analysis Involving 2244 Participants

BACKGROUND: The tumor necrosis factor-alpha (TNFα) G308A gene polymorphism has been implicated in susceptibility to essential hypertension (EH), but study results are still controversial. OBJECTIVE AND METHODS: The present meta-analysis is performed to investigate the relationship between the TNFα G308A gene polymorphism and EH. Electronic databases were searched and seven separate studies on the association of the TNF α G308A gene polymorphism with EH were analyzed. The meta-analysis involved 1092 EH patients and 1152 controls. The pooled odds ratios (ORs) and their corresponding 95% confidence interval (CI) were calculated by a fixed or random effect model. RESULTS: A significant relationship between the TNFα G308A gene polymorphism and EH was found in an allelic genetic model (OR: 1.45, 95% CI: 1.17 to 1.80, P = 0.0008), a recessive genetic model (OR: 3.181, 95% CI: 1.204 to 8.408, P = 0.02), and a homozygote model (OR: 3.454, 95% CI: 1.286 to 9.278, P = 0.014). No significant association between them was detected in both a dominant genetic model (OR: 1.55, 95% CI: 0.99 to 2.42, P = 0.06) or a heterozygote genetic model (OR: 1.45, 95% CI: 0.90 to 2.33, P = 0.13). CONCLUSION: The TNFα G308A gene polymorphism is associated with EH susceptibility