Evolution of Heterogeneity (I²) Estimates and Their 95% Confidence Intervals in Large Meta-Analyses

<div><h3>Background</h3><p>Assessment of heterogeneity is essential in systematic reviews and meta-analyses of clinical trials. The most commonly used heterogeneity measure, <em>I²</em>, provides an estimate of the proportion of variability in a meta-analysis that is explained by differences between the included trials rather than by sampling error. Recent studies have raised concerns about the reliability of <em>I²</em> estimates, due to their dependence on the precision of included trials and time-dependent biases. Authors have also advocated use of 95% confidence intervals (CIs) to express the uncertainty associated with <em>I²</em> estimates. However, no previous studies have explored how many trials and events are required to ensure stable and reliable <em>I²</em> estimates, or how 95% CIs perform as evidence accumulates.</p> <h3>Methodology/Principal Findings</h3><p>To assess the stability and reliability of <em>I²</em> estimates and their 95% CIs, in relation to the cumulative number of trials and events in meta-analysis, we looked at 16 large Cochrane meta-analyses - each including a sufficient number of trials and events to reliably estimate <em>I²</em> - and monitored the <em>I²</em> estimates and their 95% CIs for each year of publication. In 10 of the 16 meta-analyses, the <em>I²</em> estimates fluctuated more than 40% over time. The median number of events and trials required before the cumulative <em>I²</em> estimates stayed within +/−20% of the final <em>I²</em> estimate was 467 and 11. No major fluctuations were observed after 500 events and 14 trials. The 95% confidence intervals provided good coverage over time.</p> <h3>Conclusions/Significance</h3><p><em>I²</em> estimates need to be interpreted with caution when the meta-analysis only includes a limited number of events or trials. Confidence intervals for <em>I²</em> estimates provide good coverage as evidence accumulates, and are thus valuable for reflecting the uncertainty associated with estimating <em>I²</em>.</p> </div

Presents the evolution of the cumulative <i>I²</i> estimates and their associated 95% confidence intervals (CIs) over the accumulation of trials in meta-analyses (1) to (8).

<p>The cumulative <i>I²</i> are represented by the dot-dashed line ( ), and their associated cumulative 95% CIs are represented by the dotted lines ( ).</p

Presents the evolution of the cumulative <i>I²</i> estimates and their associated 95% confidence intervals (CIs) over the accumulation of events in meta-analyses (1) to (8).

<p>The cumulative <i>I²</i> are represented by the dot-dashed line ( ), and their associated cumulative 95% CIs are represented by the dotted lines ( ).</p

Fluctuation spans of <i>I²</i> values and the number of trials and events required to become stabile.

<p>Fluctuation spans of <i>I²</i> values and the number of trials and events required to become stabile.</p

Presents the evolution of the cumulative <i>I²</i> estimates and their associated 95% confidence intervals (CIs) over the accumulation of trials in meta-analyses (9) to (16).

<p>The cumulative <i>I²</i> are represented by the dot-dashed line ( ), and their associated cumulative 95% CIs are represented by the dotted lines ( ).</p

Characteristics of the 16 included meta-analyses.

<p>Characteristics of the 16 included meta-analyses.</p