Overstating the evidence - double counting in meta-analysis and related problems

Background: The problem of missing studies in meta-analysis has received much attention. Less attention has been paid to the more serious problem of double counting of evidence. Methods: Various problems in overstating the precision of results from meta-analyses are described and illustrated with examples, including papers from leading medical journals. These problems include, but are not limited to, simple double-counting of the same studies, double counting of some aspects of the studies, inappropriate imputation of results, and assigning spurious precision to individual studies. Results: Some suggestions are made as to how the quality and reliability of meta-analysis can be improved. It is proposed that the key to quality in meta-analysis lies in the results being transparent and checkable. Conclusions: Existing quality check lists for meta-analysis do little to encourage an appropriate attitude to combining evidence and to statistical analysis. Journals and other relevant organisations should encourage authors to make data available and make methods explicit. They should also act promptly to withdraw meta-analyses when mistakes are found

Network meta-analysis-highly attractive but more methodological research is needed

Network meta-analysis, in the context of a systematic review, is a meta-analysis in which multiple treatments (that is, three or more) are being compared using both direct comparisons of interventions within randomized controlled trials and indirect comparisons across trials based on a common comparator. To ensure validity of findings from network meta-analyses, the systematic review must be designed rigorously and conducted carefully. Aspects of designing and conducting a systematic review for network meta-analysis include defining the review question, specifying eligibility criteria, searching for and selecting studies, assessing risk of bias and quality of evidence, conducting a network meta-analysis, interpreting and reporting findings. This commentary summarizes the methodologic challenges and research opportunities for network meta-analysis relevant to each aspect of the systematic review process based on discussions at a network meta-analysis methodology meeting we hosted in May 2010 at the Johns Hopkins Bloomberg School of Public Health. Since this commentary reflects the discussion at that meeting, it is not intended to provide an overview of the field

Network meta-analysis on the log-hazard scale, combining count and hazard ratio statistics accounting for multi-arm trials: a tutorial.

BACKGROUND: Data on survival endpoints are usually summarised using either hazard ratio, cumulative number of events, or median survival statistics. Network meta-analysis, an extension of traditional pairwise meta-analysis, is typically based on a single statistic. In this case, studies which do not report the chosen statistic are excluded from the analysis which may introduce bias. METHODS: In this paper we present a tutorial illustrating how network meta-analyses of survival endpoints can combine count and hazard ratio statistics in a single analysis on the hazard ratio scale. We also describe methods for accounting for the correlations in relative treatment effects (such as hazard ratios) that arise in trials with more than two arms. Combination of count and hazard ratio data in a single analysis is achieved by estimating the cumulative hazard for each trial arm reporting count data. Correlation in relative treatment effects in multi-arm trials is preserved by converting the relative treatment effect estimates (the hazard ratios) to arm-specific outcomes (hazards). RESULTS: A worked example of an analysis of mortality data in chronic obstructive pulmonary disease (COPD) is used to illustrate the methods. The data set and WinBUGS code for fixed and random effects models are provided. CONCLUSIONS: By incorporating all data presentations in a single analysis, we avoid the potential selection bias associated with conducting an analysis for a single statistic and the potential difficulties of interpretation, misleading results and loss of available treatment comparisons associated with conducting separate analyses for different summary statistics

Bivariate random-effects meta-analysis and the estimation of between-study correlation

BACKGROUND: When multiple endpoints are of interest in evidence synthesis, a multivariate meta-analysis can jointly synthesise those endpoints and utilise their correlation. A multivariate random-effects meta-analysis must incorporate and estimate the between-study correlation (ρ(B)). METHODS: In this paper we assess maximum likelihood estimation of a general normal model and a generalised model for bivariate random-effects meta-analysis (BRMA). We consider two applied examples, one involving a diagnostic marker and the other a surrogate outcome. These motivate a simulation study where estimation properties from BRMA are compared with those from two separate univariate random-effects meta-analyses (URMAs), the traditional approach. RESULTS: The normal BRMA model estimates ρ(B )as -1 in both applied examples. Analytically we show this is due to the maximum likelihood estimator sensibly truncating the between-study covariance matrix on the boundary of its parameter space. Our simulations reveal this commonly occurs when the number of studies is small or the within-study variation is relatively large; it also causes upwardly biased between-study variance estimates, which are inflated to compensate for the restriction on [Formula: see text] (B). Importantly, this does not induce any systematic bias in the pooled estimates and produces conservative standard errors and mean-square errors. Furthermore, the normal BRMA is preferable to two normal URMAs; the mean-square error and standard error of pooled estimates is generally smaller in the BRMA, especially given data missing at random. For meta-analysis of proportions we then show that a generalised BRMA model is better still. This correctly uses a binomial rather than normal distribution, and produces better estimates than the normal BRMA and also two generalised URMAs; however the model may sometimes not converge due to difficulties estimating ρ(B). CONCLUSION: A BRMA model offers numerous advantages over separate univariate synthesises; this paper highlights some of these benefits in both a normal and generalised modelling framework, and examines the estimation of between-study correlation to aid practitioners

Gradual transition from mosaic to global DNA methylation patterns during deuterostome evolution

<p>Abstract</p> <p>Background</p> <p>DNA methylation by the Dnmt family occurs in vertebrates and invertebrates, including ascidians, and is thought to play important roles in gene regulation and genome stability, especially in vertebrates. However, the global methylation patterns of vertebrates and invertebrates are distinctive. Whereas almost all CpG sites are methylated in vertebrates, with the exception of those in CpG islands, the ascidian genome contains approximately equal amounts of methylated and unmethylated regions. Curiously, methylation status can be reliably estimated from the local frequency of CpG dinucleotides in the ascidian genome. Methylated and unmethylated regions tend to have few and many CpG sites, respectively, consistent with our knowledge of the methylation status of CpG islands and other regions in mammals. However, DNA methylation patterns and levels in vertebrates and invertebrates have not been analyzed in the same way.</p> <p>Results</p> <p>Using a new computational methodology based on the decomposition of the bimodal distributions of methylated and unmethylated regions, we estimated the extent of the global methylation patterns in a wide range of animals. We then examined the epigenetic changes <it>in silico </it>along the phylogenetic tree. We observed a gradual transition from fractional to global patterns of methylation in deuterostomes, rather than a clear demarcation between vertebrates and invertebrates. When we applied this methodology to six piscine genomes, some of which showed features similar to those of invertebrates.</p> <p>Conclusions</p> <p>The mammalian global DNA methylation pattern was probably not acquired at an early stage of vertebrate evolution, but gradually expanded from that of a more ancient organism.</p

Global haplotype partitioning for maximal associated SNP pairs

<p>Abstract</p> <p>Background</p> <p>Global partitioning based on pairwise associations of SNPs has not previously been used to define haplotype blocks within genomes. Here, we define an association index based on LD between SNP pairs. We use the Fisher's exact test to assess the statistical significance of the LD estimator. By this test, each SNP pair is characterized as associated, independent, or not-statistically-significant. We set limits on the maximum acceptable proportion of independent pairs within all blocks and search for the partitioning with maximal proportion of associated SNP pairs. Essentially, this model is reduced to a constrained optimization problem, the solution of which is obtained by iterating a dynamic programming algorithm.</p> <p>Results</p> <p>In comparison with other methods, our algorithm reports blocks of larger average size. Nevertheless, the haplotype diversity within the blocks is captured by a small number of tagSNPs. Resampling HapMap haplotypes under a block-based model of recombination showed that our algorithm is robust in reproducing the same partitioning for recombinant samples. Our algorithm performed better than previously reported models in a case-control association study aimed at mapping a single locus trait, based on simulation results that were evaluated by a block-based statistical test. Compared to methods of haplotype block partitioning, we performed best on detection of recombination hotspots.</p> <p>Conclusion</p> <p>Our proposed method divides chromosomes into the regions within which allelic associations of SNP pairs are maximized. This approach presents a native design for dimension reduction in genome-wide association studies. Our results show that the pairwise allelic association of SNPs can describe various features of genomic variation, in particular recombination hotspots.</p

Effect of nesiritide in patients with acute decompensated heart failure

Background Nesiritide is approved in the United States for early relief of dyspnea in patients with acute heart failure. Previous meta-analyses have raised questions regarding renal toxicity and the mortality associated with this agent. Methods We randomly assigned 7141 patients who were hospitalized with acute heart failure to receive either nesiritide or placebo for 24 to 168 hours in addition to standard care. Coprimary end points were the change in dyspnea at 6 and 24 hours, as measured on a 7-point Likert scale, and the composite end point of rehospitalization for heart failure or death within 30 days. Results Patients randomly assigned to nesiritide, as compared with those assigned to placebo, more frequently reported markedly or moderately improved dyspnea at 6 hours (44.5% vs. 42.1%, P = 0.03) and 24 hours (68.2% vs. 66.1%, P = 0.007), but the prespecified level for significance (P≤0.005 for both assessments or P≤0.0025 for either) was not met. The rate of rehospitalization for heart failure or death from any cause within 30 days was 9.4% in the nesiritide group versus 10.1% in the placebo group (absolute difference, −0.7 percentage points; 95% confidence interval [CI], −2.1 to 0.7; P = 0.31). There were no significant differences in rates of death from any cause at 30 days (3.6% with nesiritide vs. 4.0% with placebo; absolute difference, −0.4 percentage points; 95% CI, −1.3 to 0.5) or rates of worsening renal function, defined by more than a 25% decrease in the estimated glomerular filtration rate (31.4% vs. 29.5%; odds ratio, 1.09; 95% CI, 0.98 to 1.21; P = 0.11). Conclusions Nesiritide was not associated with an increase or a decrease in the rate of death and rehospitalization and had a small, nonsignificant effect on dyspnea when used in combination with other therapies. It was not associated with a worsening of renal function, but it was associated with an increase in rates of hypotension. On the basis of these results, nesiritide cannot be recommended for routine use in the broad population of patients with acute heart failure. (Funded by Scios; ClinicalTrials.gov number, NCT00475852.

Effect of nesiritide in patients with acute decompensated heart failure.

BACKGROUND: Nesiritide is approved in the United States for early relief of dyspnea in patients with acute heart failure. Previous meta-analyses have raised questions regarding renal toxicity and the mortality associated with this agent. METHODS: We randomly assigned 7141 patients who were hospitalized with acute heart failure to receive either nesiritide or placebo for 24 to 168 hours in addition to standard care. Coprimary end points were the change in dyspnea at 6 and 24 hours, as measured on a 7-point Likert scale, and the composite end point of rehospitalization for heart failure or death within 30 days. RESULTS: Patients randomly assigned to nesiritide, as compared with those assigned to placebo, more frequently reported markedly or moderately improved dyspnea at 6 hours (44.5% vs. 42.1%, P=0.03) and 24 hours (68.2% vs. 66.1%, P=0.007), but the prespecified level for significance (P≤0.005 for both assessments or P≤0.0025 for either) was not met. The rate of rehospitalization for heart failure or death from any cause within 30 days was 9.4% in the nesiritide group versus 10.1% in the placebo group (absolute difference, -0.7 percentage points; 95% confidence interval [CI], -2.1 to 0.7; P=0.31). There were no significant differences in rates of death from any cause at 30 days (3.6% with nesiritide vs. 4.0% with placebo; absolute difference, -0.4 percentage points; 95% CI, -1.3 to 0.5) or rates of worsening renal function, defined by more than a 25% decrease in the estimated glomerular filtration rate (31.4% vs. 29.5%; odds ratio, 1.09; 95% CI, 0.98 to 1.21; P=0.11). CONCLUSIONS: Nesiritide was not associated with an increase or a decrease in the rate of death and rehospitalization and had a small, nonsignificant effect on dyspnea when used in combination with other therapies. It was not associated with a worsening of renal function, but it was associated with an increase in rates of hypotension. On the basis of these results, nesiritide cannot be recommended for routine use in the broad population of patients with acute heart failure. (Funded by Scios; ClinicalTrials.gov number, NCT00475852.