Context-dependent representation of within- and between-model uncertainty: Aggregating probabilistic predictions in infectious disease epidemiology

Probabilistic predictions support public health planning and decision making, especially in infectious disease emergencies. Aggregating outputs from multiple models yields more robust predictions of outcomes and associated uncertainty. While the selection of an aggregation method can be guided by retrospective performance evaluations, this is not always possible. For example, if predictions are conditional on assumptions about how the future will unfold (e.g. possible interventions), these assumptions may never materialize, precluding any direct comparison between predictions and observations. Here, we summarize literature on aggregating probabilistic predictions, illustrate various methods for infectious disease predictions via simulation, and present a strategy for choosing an aggregation method when empirical validation cannot be used. We focus on the linear opinion pool (LOP) and Vincent average, common methods that make different assumptions about between-prediction uncertainty. We contend that assumptions of the aggregation method should align with a hypothesis about how uncertainty is expressed within and between predictions from different sources. The LOP assumes that between-prediction uncertainty is meaningful and should be retained, while the Vincent average assumes that between-prediction uncertainty is akin to sampling error and should not be preserved. We provide an R package for implementation. Given the rising importance of multi-model infectious disease hubs, our work provides useful guidance on aggregation and a deeper understanding of the benefits and risks of different approaches

Impacts of Zika emergence in Latin America on endemic dengue transmission

In 2015 and 2016, Zika virus (ZIKV) swept through dengue virus (DENV) endemic areas of Latin America. These viruses are of the same family, share a vector and may interact competitively or synergistically through human immune responses. We examine dengue incidence from Brazil and Colombia before, during, and after the Zika epidemic. We find evidence that dengue incidence was atypically low in 2017 in both countries. We investigate whether subnational Zika incidence is associated with changes in dengue incidence and find mixed results. Using simulations with multiple assumptions of interactions between DENV and ZIKV, we find cross-protection suppresses incidence of dengue following Zika outbreaks and low periods of dengue incidence are followed by resurgence. Our simulations suggest correlations in DENV and ZIKV reproduction numbers could complicate associations between ZIKV incidence and post-ZIKV DENV incidence and that periods of low dengue incidence are followed by large increases in dengue incidence

Collaborative Hubs: Making the Most of Predictive Epidemic Modeling

The COVID-19 pandemic has made it clear that epidemic models play an important role in how governments and the public respond to infectious disease crises. Early in the pandemic, models were used to estimate the true number of infections. Later, they estimated key parameters, generated short-term forecasts of outbreak trends, and quantified possible effects of interventions on the unfolding epidemic. In contrast to the coordinating role played by major national or international agencies in weather-related emergencies, pandemic modeling efforts were initially scattered across many research institutions. Differences in modeling approaches led to contrasting results, contributing to confusion in public perception of the pandemic. Efforts to coordinate modeling efforts in so-called “hubs” have provided governments, healthcare agencies, and the public with assessments and forecasts that reflect the consensus in the modeling community. This has been achieved by openly synthesizing uncertainties across different modeling approaches and facilitating comparisons between them

The US COVID-19 and Influenza Scenario Modeling Hubs: Delivering long-term projections to guide policy

Between December 2020 and April 2023, the COVID-19 Scenario Modeling Hub (SMH) generated operational multi-month projections of COVID-19 burden in the US to guide pandemic planning and decision-making in the context of high uncertainty. This effort was born out of an attempt to coordinate, synthesize and effectively use the unprecedented amount of predictive modeling that emerged throughout the COVID-19 pandemic. Here we describe the history of this massive collective research effort, the process of convening and maintaining an open modeling hub active over multiple years, and attempt to provide a blueprint for future efforts. We detail the process of generating 17 rounds of scenarios and projections at different stages of the COVID-19 pandemic, and disseminating results to the public health community and lay public. We also highlight how SMH was expanded to generate influenza projections during the 2022–23 season. We identify key impacts of SMH results on public health and draw lessons to improve future collaborative modeling efforts, research on scenario projections, and the interface between models and policy

Impact of SARS-CoV-2 vaccination of children ages 5–11 years on COVID-19 disease burden and resilience to new variants in the United States, November 2021–March 2022: A multi-model study

Background: The COVID-19 Scenario Modeling Hub convened nine modeling teams to project the impact of expanding SARS-CoV-2 vaccination to children aged 5–11 years on COVID-19 burden and resilience against variant strains. Methods: Teams contributed state- and national-level weekly projections of cases, hospitalizations, and deaths in the United States from September 12, 2021 to March 12, 2022. Four scenarios covered all combinations of 1) vaccination (or not) of children aged 5–11 years (starting November 1, 2021), and 2) emergence (or not) of a variant more transmissible than the Delta variant (emerging November 15, 2021). Individual team projections were linearly pooled. The effect of childhood vaccination on overall and age-specific outcomes was estimated using meta-analyses. Findings: Assuming that a new variant would not emerge, all-age COVID-19 outcomes were projected to decrease nationally through mid-March 2022. In this setting, vaccination of children 5–11 years old was associated with reductions in projections for all-age cumulative cases (7.2%, mean incidence ratio [IR] 0.928, 95% confidence interval [CI] 0.880–0.977), hospitalizations (8.7%, mean IR 0.913, 95% CI 0.834–0.992), and deaths (9.2%, mean IR 0.908, 95% CI 0.797–1.020) compared with scenarios without childhood vaccination. Vaccine benefits increased for scenarios including a hypothesized more transmissible variant, assuming similar vaccine effectiveness. Projected relative reductions in cumulative outcomes were larger for children than for the entire population. State-level variation was observed. Interpretation: Given the scenario assumptions (defined before the emergence of Omicron), expanding vaccination to children 5–11 years old would provide measurable direct benefits, as well as indirect benefits to the all-age U.S. population, including resilience to more transmissible variants. Funding: Various (see acknowledgments)

Evaluation of the US COVID-19 Scenario Modeling Hub for informing pandemic response under uncertainty

Our ability to forecast epidemics far into the future is constrained by the many complexities of disease systems. Realistic longer-term projections may, however, be possible under well-defined scenarios that specify the future state of critical epidemic drivers. Since December 2020, the U.S. COVID-19 Scenario Modeling Hub (SMH) has convened multiple modeling teams to make months ahead projections of SARS-CoV-2 burden, totaling nearly 1.8 million national and state-level projections. Here, we find SMH performance varied widely as a function of both scenario validity and model calibration. We show scenarios remained close to reality for 22 weeks on average before the arrival of unanticipated SARS-CoV-2 variants invalidated key assumptions. An ensemble of participating models that preserved variation between models (using the linear opinion pool method) was consistently more reliable than any single model in periods of valid scenario assumptions, while projection interval coverage was near target levels. SMH projections were used to guide pandemic response, illustrating the value of collaborative hubs for longer-term scenario projections

Synergistic interventions to control COVID-19:Mass testing and isolation mitigates reliance on distancing

Stay-at-home orders and shutdowns of non-essential businesses are powerful, but socially costly, tools to control the pandemic spread of SARS-CoV-2. Mass testing strategies, which rely on widely administered frequent and rapid diagnostics to identify and isolate infected individuals, could be a potentially less disruptive management strategy, particularly where vaccine access is limited. In this paper, we assess the extent to which mass testing and isolation strategies can reduce reliance on socially costly non-pharmaceutical interventions, such as distancing and shutdowns. We develop a multi-compartmental model of SARS-CoV-2 transmission incorporating both preventative non-pharmaceutical interventions (NPIs) and testing and isolation to evaluate their combined effect on public health outcomes. Our model is designed to be a policy-guiding tool that captures important realities of the testing system, including constraints on test administration and non-random testing allocation. We show how strategic changes in the characteristics of the testing system, including test administration, test delays, and test sensitivity, can reduce reliance on preventative NPIs without compromising public health outcomes in the future. The lowest NPI levels are possible only when many tests are administered and test delays are short, given limited immunity in the population. Reducing reliance on NPIs is highly dependent on the ability of a testing program to identify and isolate unreported, asymptomatic infections. Changes in NPIs, including the intensity of lockdowns and stay at home orders, should be coordinated with increases in testing to ensure epidemic control; otherwise small additional lifting of these NPIs can lead to dramatic increases in infections, hospitalizations and deaths. Importantly, our results can be used to guide ramp-up of testing capacity in outbreak settings, allow for the flexible design of combined interventions based on social context, and inform future cost-benefit analyses to identify efficient pandemic management strategies

Strategic testing approaches for targeted disease monitoring can be used to inform pandemic decision-making

More than 1.6 million Servere Acute Respiratory Syndrome Cronovirus 2(SARS-COV-2)tests were administered daily in the United States at the peak of the epidemic, with a significant focus on individual treatment. Here, we show that objective-driven, strategic sampling designs and analyses can maximize information gain at the population level, which is necessary to increase situational awareness and predict, prepare for, and respond to a pandemic, while also continuing to inform individual treatment. By focusing on specific objectives such as individual treatment or disease prediction and control (e.g., via the collection of population- level statistics to inform lockdown measures or vaccine rollout) and drawing from the literature on capture-recapture methods to deal with nonrandom sampling and testing errors, we illustrate how public health objectives can be achieved even with limited test availability when testing programs are designed a priori to meet those objectives. © 2021 Public Library of Science. All rights reserved