Dealing with substantial heterogeneity in Cochrane reviews. Cross-sectional study

<p>Abstract</p> <p>Background</p> <p>Dealing with heterogeneity in meta-analyses is often tricky, and there is only limited advice for authors on what to do. We investigated how authors addressed different degrees of heterogeneity, in particular whether they used a fixed effect model, which assumes that all the included studies are estimating the same true effect, or a random effects model where this is not assumed.</p> <p>Methods</p> <p>We sampled randomly 60 Cochrane reviews from 2008, which presented a result in its first meta-analysis with substantial heterogeneity (I²greater than 50%, i.e. more than 50% of the variation is due to heterogeneity rather than chance). We extracted information on choice of statistical model, how the authors had handled the heterogeneity, and assessed the methodological quality of the reviews in relation to this.</p> <p>Results</p> <p>The distribution of heterogeneity was rather uniform in the whole I²interval, 50-100%. A fixed effect model was used in 33 reviews (55%), but there was no correlation between I²and choice of model (P = 0.79). We considered that 20 reviews (33%), 16 of which had used a fixed effect model, had major problems. The most common problems were: use of a fixed effect model and lack of rationale for choice of that model, lack of comment on even severe heterogeneity and of reservations and explanations of its likely causes. The problematic reviews had significantly fewer included trials than other reviews (4.3 vs. 8.0, P = 0.024). The problems became less pronounced with time, as those reviews that were most recently updated more often used a random effects model.</p> <p>Conclusion</p> <p>One-third of Cochrane reviews with substantial heterogeneity had major problems in relation to their handling of heterogeneity. More attention is needed to this issue, as the problems we identified can be essential for the conclusions of the reviews.</p

Sulfonylurea versus metformin monotherapy in patients with type 2 diabetes:a Cochrane systematic review and meta-analysis of randomized clinical trials and trial sequential analysis

BACKGROUND: Guidelines recommend metformin as the first-line oral treatment for type 2 diabetes. We conducted a systematic review to assess whether the use of second- and third-generation sulfonylurea agents is associated with benefits and harms in terms of patient-important outcomes compared with metformin. METHODS: We searched several electronic databases and other sources for randomized clinical trials published to August 2011. We included trials that compared sulfonylurea versus metformin monotherapy among patients 18 years or older with type 2 diabetes and that had an intervention period of at least 24 weeks. We assessed risk of bias and extracted data related to interventions and outcomes. The risk of random errors was assessed by trial sequential analysis. RESULTS: We included 14 trials (4560 participants). All trials were judged to be at high risk of bias. Data on patient-important outcomes were sparse. Compared with metformin, sulfonylurea did not significantly affect all-cause mortality (relative risk [RR] 0.98, 95% confidence interval [CI] 0.61 to 1.58) or cardiovascular mortality (RR 1.47, 95% CI 0.54 to 4.01). Sulfonylurea significantly decreased the risk of nonfatal macrovascular outcomes (RR 0.67, 95% CI 0.48 to 0.93). However, the definition of this outcome varied among trials, and trial sequential analysis showed that more trials are needed before reliable conclusions can be drawn. No differences between sulfonylurea and metformin were found for change in fasting blood glucose level or glycosylated hemoglobin concentration in the random-effects model. Sulfonylurea resulted in greater weight gain compared with metformin, a finding confirmed in the trial sequential analysis. Significantly more patients in the sulfonylurea arm than in the metformin arm had mild hypoglycemia (RR 2.95, 95% CI 2.13 to 4.07) and severe hypoglycemia (RR 5.64, 95% CI 1.22 to 26.00). INTERPRETATION: Some evidence suggests that, compared with metformin, second- and third-generation sulfonylureas may not affect all-cause or cardiovascular mortality but may decrease the risk of nonfatal macrovascular outcomes among patients with type 2 diabetes. They may also increase the risk of hypoglycemia. In general, the available data were too few and inconsistent to provide firm evidence concerning patient-important outcomes in relation to the benefits and harms of sulfonylurea versus metformin monotherapy