Comparison of uncertainty intervals after standardization of analytical choices.

The figure shows 95% uncertainty intervals corresponding to Fig 6, Step 4.</p

Methodological characteristics and parameterizations of the compared estimation approaches.

The table follows the structure of Sections 2.1–2.5. The consensus model is introduced in Section 4.1 By conditional distribution of Xt we refer to the distribution of new cases Xt in formulation (1) or (2). The concept of “revision due to smoothing” is discussed in Section 3.3.</p

Overlay of different <i>R</i>_<i>t</i> estimates.

Estimates for the effective reproductive number of COVID-19 in Germany published by eight different research teams on July 10, 2021 (July 11, 2021, for HZI). Top: point estimates (only available for the last 15 weeks for epiforecasts); bottom: 95% uncertainty intervals (not available for HZI).</p

Case incidence time series used by different research groups.

To enhance visibility we only display the period January through June 2021 (data version: November 23, 2021).</p

Scatter plot of mean generation time and corresponding standard deviation used by different research groups.

The red rhombus represents a “consensus value” chosen for further analysis, see Section 4.1. epiforecasts accounted for uncertainty in the generation time distribution by assuming independent normal priors for the mean and standard deviation; we illustrate the respective 95% uncertainty intervals by a cross. For context, we also show values used by public health agencies of other European countries. In the Netherlands (due to the transition to the Omicron variant) and Austria (due to a data update) the parameterization was revised. For details and references see Section B in S1 Text.</p

Overview of the groups who regularly published <i>R</i>_<i>t</i> estimates for Germany during the COVID-19 pandemic.

Descriptions of the respective methodology are provided in Section 2.2. Note that web domains provided in footnotes may be discontinued at some point; the links to the repositories provided in S1 Text, Section A, are likely to be more stable.</p

<i>R</i>_<i>t</i> estimates published between October 1, 2020, and December 10, 2020, and a consolidated estimate published 6 months later (epiforecasts: 15 weeks later).

Note that different time periods are used for Ilmenau and globalrt as these were not operated during the period shown for the other models. The consolidated ETH intervals are wider than those issued in real time due to a revision of methodology. The line type represents the label assigned to the estimate by the respective team: solid: “estimate”, dashed: “estimate based on partial data”, dotted: “forecast”. Shaded areas show 95% uncertainty intervals.</p

Comparison of 95% uncertainty intervals of the Cori method (consensus settings) with a Poisson (dark) and negative binomial distribution (light).

The uncertainty intervals under the Poisson distribution are hardly discernible from the line representing the point estimate.</p

Step-by-step alignment of analytical choices to the consensus specifications.

The left column shows the resulting Rt estimates for a subset of the considered time period. The right column shows the mean absolute differences between point estimates obtained from the different approaches. In the bottom panel all considered aspects other than the estimation method (incl. data pre-processing) are aligned. Note that the two top rows we use wider y-axis limits to accommodate the Ilmenau estimates.</p

Temporal coherence of <i>R</i>_<i>t</i> estimates.

Panels: A Proportion of 95% uncertainty intervals issued in real-time which contained the consolidated estimate. B Mean width of 95%-uncertainty intervals (unavailable for HZI, who only published point estimates). C Mean absolute difference of the real-time and consolidated estimates. D Same as C, but signed rather than absolute differences. E Proportions of cases in which real-time and consolidated point estimates disagree on whether Rt > 1. F Same as D, but with a tolerance region [0.97, 1.03], i.e., only instances where real-time and consolidated estimates are on different sides of this interval are counted. All indicators are shown as a function of the time between the target date (as stated by the teams) and the publication date. Averages refer to the period October 1, 2020—July 22, 2021 (see Fig F in S1 Text for exact periods during which methods were operated). The consolidated estimate corresponds to the one published 70 days after the respective target date. For ETH two additional lines are included in the top row differentiating between intervals obtained from the old procedure before January 26, 2021 (n = 95), and from the new bootstrap approach afterward (n = 171; see model description in Section 2.2).</p