Why are different estimates of the effective reproductive number so different? A case study on COVID-19 in Germany

The effective reproductive number R$_t$ has taken a central role in the scientific, political, and public discussion during the COVID-19 pandemic, with numerous real-time estimates of this quantity routinely published. Disagreement between estimates can be substantial and may lead to confusion among decision-makers and the general public. In this work, we compare different estimates of the national-level effective reproductive number of COVID-19 in Germany in 2020 and 2021. We consider the agreement between estimates from the same method but published at different time points (within-method agreement) as well as retrospective agreement across eight different approaches (between-method agreement). Concerning the former, estimates from some methods are very stable over time and hardly subject to revisions, while others display considerable fluctuations. To evaluate between-method agreement, we reproduce the estimates generated by different groups using a variety of statistical approaches, standardizing analytical choices to assess how they contribute to the observed disagreement. These analytical choices include the data source, data pre-processing, assumed generation time distribution, statistical tuning parameters, and various delay distributions. We find that in practice, these auxiliary choices in the estimation of R$_t$ may affect results at least as strongly as the selection of the statistical approach. They should thus be communicated transparently along with the estimates

Why are different estimates of the effective reproductive number so different? A case study on COVID-19 in Germany.

The effective reproductive number Rt has taken a central role in the scientific, political, and public discussion during the COVID-19 pandemic, with numerous real-time estimates of this quantity routinely published. Disagreement between estimates can be substantial and may lead to confusion among decision-makers and the general public. In this work, we compare different estimates of the national-level effective reproductive number of COVID-19 in Germany in 2020 and 2021. We consider the agreement between estimates from the same method but published at different time points (within-method agreement) as well as retrospective agreement across eight different approaches (between-method agreement). Concerning the former, estimates from some methods are very stable over time and hardly subject to revisions, while others display considerable fluctuations. To evaluate between-method agreement, we reproduce the estimates generated by different groups using a variety of statistical approaches, standardizing analytical choices to assess how they contribute to the observed disagreement. These analytical choices include the data source, data pre-processing, assumed generation time distribution, statistical tuning parameters, and various delay distributions. We find that in practice, these auxiliary choices in the estimation of Rt may affect results at least as strongly as the selection of the statistical approach. They should thus be communicated transparently along with the estimates

Comparison of uncertainty intervals after standardization of analytical choices.

The figure shows 95% uncertainty intervals corresponding to Fig 6, Step 4.</p

Methodological characteristics and parameterizations of the compared estimation approaches.

The table follows the structure of Sections 2.1–2.5. The consensus model is introduced in Section 4.1 By conditional distribution of Xt we refer to the distribution of new cases Xt in formulation (1) or (2). The concept of “revision due to smoothing” is discussed in Section 3.3.</p

Comparison of 95% uncertainty intervals of the Cori method (consensus settings) with a Poisson (dark) and negative binomial distribution (light).

The uncertainty intervals under the Poisson distribution are hardly discernible from the line representing the point estimate.</p

Step-by-step alignment of analytical choices to the consensus specifications.

The left column shows the resulting Rt estimates for a subset of the considered time period. The right column shows the mean absolute differences between point estimates obtained from the different approaches. In the bottom panel all considered aspects other than the estimation method (incl. data pre-processing) are aligned. Note that the two top rows we use wider y-axis limits to accommodate the Ilmenau estimates.</p

Case incidence time series used by different research groups.

To enhance visibility we only display the period January through June 2021 (data version: November 23, 2021).</p