Association of Simulated COVID-19 Vaccination and Nonpharmaceutical Interventions With Infections, Hospitalizations, and Mortality

IMPORTANCE Vaccination against SARS-CoV-2 has the potential to significantly reduce transmission and COVID-19 morbidity and mortality. The relative importance of vaccination strategies and nonpharmaceutical interventions (NPIs) is not well understood. OBJECTIVE To assess the association of simulated COVID-19 vaccine efficacy and coverage scenarios with and without NPIs with infections, hospitalizations, and deaths. DESIGN, SETTING, AND PARTICIPANTS An established agent-based decision analytical model was used to simulate COVID-19 transmission and progression from March 24, 2020, to September 23, 2021. The model simulated COVID-19 spread in North Carolina, a US state of 10.5 million people. A network of 1 017 720 agents was constructed from US Census data to represent the statewide population. EXPOSURES Scenarios of vaccine efficacy (50% and 90%), vaccine coverage (25%, 50%, and 75% at the end of a 6-month distribution period), and NPIs (reduced mobility, school closings, and use of face masks) maintained and removed during vaccine distribution. MAIN OUTCOMES AND MEASURES Risks of infection from the start of vaccine distribution and risk differences comparing scenarios. Outcome means and SDs were calculated across replications. RESULTS In the worst-case vaccination scenario (50% efficacy, 25%coverage), a mean (SD) of 2 231 134 (117 867) new infections occurred after vaccination began with NPIs removed, and a mean (SD) of 799 949 (60 279) new infections occurred with NPIs maintained during 11 months. In contrast, in the best-case scenario (90% efficacy, 75%coverage), a mean (SD) of 527 409 (40 637) new infections occurred with NPIs removed and a mean (SD) of 450 575 (32 716) new infections occurred with NPIs maintained. With NPIs removed, lower efficacy (50%) and higher coverage (75%) reduced infection risk by a greater magnitude than higher efficacy (90%) and lower coverage (25%) compared with theworst-case scenario (mean [SD] absolute risk reduction, 13%[1%] and 8%[1%], respectively). CONCLUSIONS AND RELEVANCE Simulation outcomes suggest that removing NPIs while vaccines are distributed may result in substantial increases in infections, hospitalizations, and deaths. Furthermore, as NPIs are removed, higher vaccination coverage with less efficacious vaccines can contribute to a larger reduction in risk of SARS-CoV-2 infection compared with more efficacious vaccines at lower coverage. These findings highlight the need for well-resourced and coordinated efforts to achieve high vaccine coverage and continued adherence to NPIs before many prepandemic activities can be resumed

Can vaccine prioritization reduce disparities in COVID-19 burden for historically marginalized populations?

SARS-CoV-2 vaccination strategies were designed to reduce COVID-19 mortality, morbidity, and health inequities. To assess the impact of vaccination strategies on disparities in COVID-19 burden among historically marginalized populations (HMPs), e.g. Black race and Hispanic ethnicity, we used an agent-based simulation model, populated with census-tract data from North Carolina. We projected COVID-19 deaths, hospitalizations, and cases from 2020 July 1 to 2021 December 31, and estimated racial/ethnic disparities in COVID-19 outcomes. We modeled 2-stage vaccination prioritization scenarios applied to sub-groups including essential workers, older adults (65+), adults with high-risk health conditions, HMPs, or people in low-income tracts. Additionally, we estimated the effects of maximal uptake (100% for HMP vs. 100% for everyone), and distribution to only susceptible people. We found strategies prioritizing essential workers, then older adults led to the largest mortality and case reductions compared to no prioritization. Under baseline uptake scenarios, the age-adjusted mortality for HMPs was higher (e.g. 33.3%-34.1% higher for the Black population and 13.3%-17.0% for the Hispanic population) compared to the White population. The burden on HMPs decreased only when uptake was increased to 100% in HMPs; however, the Black population still had the highest relative mortality rate even when targeted distribution strategies were employed. If prioritization schemes were not paired with increased uptake in HMPs, disparities did not improve. The vaccination strategies publicly outlined were insufficient, exacerbating disparities between racial and ethnic groups. Strategies targeted to increase vaccine uptake among HMPs are needed to ensure equitable distribution and minimize disparities in outcomes

The United States COVID-19 Forecast Hub dataset

Academic researchers, government agencies, industry groups, and individuals have produced forecasts at an unprecedented scale during the COVID-19 pandemic. To leverage these forecasts, the United States Centers for Disease Control and Prevention (CDC) partnered with an academic research lab at the University of Massachusetts Amherst to create the US COVID-19 Forecast Hub. Launched in April 2020, the Forecast Hub is a dataset with point and probabilistic forecasts of incident cases, incident hospitalizations, incident deaths, and cumulative deaths due to COVID-19 at county, state, and national, levels in the United States. Included forecasts represent a variety of modeling approaches, data sources, and assumptions regarding the spread of COVID-19. The goal of this dataset is to establish a standardized and comparable set of short-term forecasts from modeling teams. These data can be used to develop ensemble models, communicate forecasts to the public, create visualizations, compare models, and inform policies regarding COVID-19 mitigation. These open-source data are available via download from GitHub, through an online API, and through R packages

COVSIM: A stochastic agent-based COVID-19 SIMulation model for North Carolina

We document the evolution and use of the stochastic agent-based COVID-19 simulation model (COVSIM) to study the impact of population behaviors and public health policy on disease spread within age, race/ethnicity, and urbanicity subpopulations in North Carolina. We detail the methodologies used to model the complexities of COVID-19, including multiple agent attributes (i.e., age, race/ethnicity, high-risk medical status), census tract-level interaction network, disease state network, agent behavior (i.e., masking, pharmaceutical intervention (PI) uptake, quarantine, mobility), and variants. We describe its uses outside of the COVID-19 Scenario Modeling Hub (CSMH), which has focused on the interplay of nonpharmaceutical and pharmaceutical interventions, equitability of vaccine distribution, and supporting local county decision-makers in North Carolina. This work has led to multiple publications and meetings with a variety of local stakeholders. When COVSIM joined the CSMH in January 2022, we found it was a sustainable way to support new COVID-19 challenges and allowed the group to focus on broader scientific questions. The CSMH has informed adaptions to our modeling approach, including redesigning our high-performance computing implementation

Scenario design for infectious disease projections: Integrating concepts from decision analysis and experimental design

Across many fields, scenario modeling has become an important tool for exploring long-term projections and how they might depend on potential interventions and critical uncertainties, with relevance to both decision makers and scientists. In the past decade, and especially during the COVID-19 pandemic, the field of epidemiology has seen substantial growth in the use of scenario projections. Multiple scenarios are often projected at the same time, allowing important comparisons that can guide the choice of intervention, the prioritization of research topics, or public communication. The design of the scenarios is central to their ability to inform important questions. In this paper, we draw on the fields of decision analysis and statistical design of experiments to propose a framework for scenario design in epidemiology, with relevance also to other fields. We identify six different fundamental purposes for scenario designs (decision making, sensitivity analysis, situational awareness, horizon scanning, forecasting, and value of information) and discuss how those purposes guide the structure of scenarios. We discuss other aspects of the content and process of scenario design, broadly for all settings and specifically for multi-model ensemble projections. As an illustrative case study, we examine the first 17 rounds of scenarios from the U.S. COVID-19 Scenario Modeling Hub, then reflect on future advancements that could improve the design of scenarios in epidemiological settings

Evaluation of the US COVID-19 Scenario Modeling Hub for informing pandemic response under uncertainty

Abstract Our ability to forecast epidemics far into the future is constrained by the many complexities of disease systems. Realistic longer-term projections may, however, be possible under well-defined scenarios that specify the future state of critical epidemic drivers. Since December 2020, the U.S. COVID-19 Scenario Modeling Hub (SMH) has convened multiple modeling teams to make months ahead projections of SARS-CoV-2 burden, totaling nearly 1.8 million national and state-level projections. Here, we find SMH performance varied widely as a function of both scenario validity and model calibration. We show scenarios remained close to reality for 22 weeks on average before the arrival of unanticipated SARS-CoV-2 variants invalidated key assumptions. An ensemble of participating models that preserved variation between models (using the linear opinion pool method) was consistently more reliable than any single model in periods of valid scenario assumptions, while projection interval coverage was near target levels. SMH projections were used to guide pandemic response, illustrating the value of collaborative hubs for longer-term scenario projections

Comparison between the assumed and observed annual uptake of COVID-19 reformulated vaccines by US state.

Solid lines represent the assumed state-level annual uptake of reformulated vaccines by age group, projected to follow the uptake patterns for the first booster dose (authorized in September 2021). Dots indicate the monthly observed uptake as of February 24, 2024, sourced from the CDC website. Each age group is represented by a different color. (TIF)</p

Comparison between the assumed and observed annual uptake of COVID-19 reformulated vaccines at the national level in the United States.

(A) Solid lines represent the assumed national-level annual uptake of reformulated vaccines by age group, projected to follow the uptake patterns for the first booster dose (authorized in September 2021). Dashed lines indicate the empirically observed uptake as of February 24, 2024, sourced from the CDC website, covering Puerto Rico and the Virgin Islands which are not accounted for in the assumed national-level uptake. Each age group is represented by a different color. (B) Observed and assumed annual uptake of reformulated vaccines among individuals aged 18 and over at the national level. Each color represents a different vaccine coverage data. (TIF)</p

Quantile-quantile (QQ) plot for assessing the performance of models regarding cumulative COVID-19 hospitalizations and deaths in the United States.

The actual coverage of each model, regarding cumulative hospitalizations and deaths as of December 16, 2023, is plotted against its expected coverage. Coverage measures the percentage of observations that fall within a given prediction interval (e.g., for a 90% prediction interval, expected coverage is 90%). Coverage was calculated across all locations and projection weeks. The dashed lines represent the expected relationship (expected coverage is equal to actual coverage), where a line below indicates models are overconfident (actual coverage is less than expected coverage), and above the line means models are underconfident (actual coverage is more than expected coverage). The black solid lines depict the ensemble model, while each colored line represents contributing individual models. Following the CDC recommendation for reformulated vaccines (published on September 12, 2023), only scenarios with vaccination recommendations to all individuals were included. (TIF)</p

Relationship between prevented COVID-19 hospitalizations and assumed vaccine coverage in individuals aged 65 and above across US states.

The relationship between the cumulative difference in COVID-19 hospitalizations for the next 2 years (April 2023–April 2025) under different vaccination recommendations and assumed vaccine uptake among those aged 65 and above (65+) in each US state: (A and B) vaccination of all compared to no vaccination and (C and D) vaccination of 65+, compared to no vaccination. The x-axis represents the assumed vaccine coverage among 65+ at saturation considering the higher severity in 65+ (likely to have the most significant contribution to decreasing hospitalizations). Dots in each panel correspond to individual US states. COVID-19, Coronavirus Disease 2019.</p