Near-term forecasts of influenza-like illness: An evaluation of autoregressive time series approaches

Seasonal influenza in the United States is estimated to cause 9-35 million illnesses annually, with resultant economic burden amounting to $47-$150 billion. Reliable real-time forecasts of influenza can help public health agencies better manage these outbreaks. Here, we investigate the feasibility of three autoregressive methods for near-term forecasts: an Autoregressive Integrated Moving Average (ARIMA) model with time-varying order; an ARIMA model fit to seasonally adjusted incidence rates (ARIMA-STL); and a feed-forward autoregressive artificial neural network with a single hidden layer (AR-NN). We generated retrospective forecasts for influenza incidence one to four weeks in the future at US National and 10 regions in the US during 5 influenza seasons. We compared the relative accuracy of the point and probabilistic forecasts of the three models with respect to each other and in relation to two large external validation sets that each comprise at least 20 other models. Both the probabilistic and point forecasts of AR-NN were found to be more accurate than those of the other two models overall. An additional sub-analysis found that the three models benefitted considerably from the use of search trends based 'nowcast' as a proxy for surveillance data, and these three models with use of nowcasts were found to be the highest ranked models in both validation datasets. When the nowcasts were withheld, the three models remained competitive relative to models in the validation sets. The difference in accuracy among the three models, and relative to models of the validation sets, was found to be largely statistically significant. Our results suggest that autoregressive models even when not equipped to capture transmission dynamics can provide reasonably accurate near-term forecasts for influenza. Existing support in open-source libraries make them suitable non-naïve baselines for model comparison studies and for operational forecasts in resource constrained settings where more sophisticated methods may not be feasible

Investigating associations between COVID-19 mortality and population-level health and socioeconomic indicators in the United States: A modeling study

Background: With the availability of multiple Coronavirus Disease 2019 (COVID-19) vaccines and the predicted shortages in supply for the near future, it is necessary to allocate vaccines in a manner that minimizes severe outcomes, particularly deaths. To date, vaccination strategies in the United States have focused on individual characteristics such as age and occupation. Here, we assess the utility of population-level health and socioeconomic indicators as additional criteria for geographical allocation of vaccines. Methods and findings: County-level estimates of 14 indicators associated with COVID-19 mortality were extracted from public data sources. Effect estimates of the individual indicators were calculated with univariate models. Presence of spatial autocorrelation was established using Moran's I statistic. Spatial simultaneous autoregressive (SAR) models that account for spatial autocorrelation in response and predictors were used to assess (i) the proportion of variance in county-level COVID-19 mortality that can explained by identified health/socioeconomic indicators (R2); and (ii) effect estimates of each predictor. Adjusting for case rates, the selected indicators individually explain 24%-29% of the variability in mortality. Prevalence of chronic kidney disease and proportion of population residing in nursing homes have the highest R2. Mortality is estimated to increase by 43 per thousand residents (95% CI: 37-49; p < 0.001) with a 1% increase in the prevalence of chronic kidney disease and by 39 deaths per thousand (95% CI: 34-44; p < 0.001) with 1% increase in population living in nursing homes. SAR models using multiple health/socioeconomic indicators explain 43% of the variability in COVID-19 mortality in US counties, adjusting for case rates. R2 was found to be not sensitive to the choice of SAR model form. Study limitations include the use of mortality rates that are not age standardized, a spatial adjacency matrix that does not capture human flows among counties, and insufficient accounting for interaction among predictors. Conclusions: Significant spatial autocorrelation exists in COVID-19 mortality in the US, and population health/socioeconomic indicators account for a considerable variability in county-level mortality. In the context of vaccine rollout in the US and globally, national and subnational estimates of burden of disease could inform optimal geographical allocation of vaccines

Reappraising the utility of Google Flu Trends

Estimation of influenza-like illness (ILI) using search trends activity was intended to supplement traditional surveillance systems, and was a motivation behind the development of Google Flu Trends (GFT). However, several studies have previously reported large errors in GFT estimates of ILI in the US. Following recent release of time-stamped surveillance data, which better reflects real-time operational scenarios, we reanalyzed GFT errors. Using three data sources—GFT: an archive of weekly ILI estimates from Google Flu Trends; ILIf: fully-observed ILI rates from ILINet; and, ILIp: ILI rates available in real-time based on partial reporting—five influenza seasons were analyzed and mean square errors (MSE) of GFT and ILIp as estimates of ILIf were computed. To correct GFT errors, a random forest regression model was built with ILI and GFT rates from the previous three weeks as predictors. An overall reduction in error of 44% was observed and the errors of the corrected GFT are lower than those of ILIp. An 80% reduction in error during 2012/13, when GFT had large errors, shows that extreme failures of GFT could have been avoided. Using autoregressive integrated moving average (ARIMA) models, one- to four-week ahead forecasts were generated with two separate data streams: ILIp alone, and with both ILIp and corrected GFT. At all forecast targets and seasons, and for all but two regions, inclusion of GFT lowered MSE. Results from two alternative error measures, mean absolute error and mean absolute proportional error, were largely consistent with results from MSE. Taken together these findings provide an error profile of GFT in the US, establish strong evidence for the adoption of search trends based 'nowcasts' in influenza forecast systems, and encourage reevaluation of the utility of this data source in diverse domains

Subregional Nowcasts of Seasonal Influenza Using Search Trends

Background: Limiting the adverse effects of seasonal influenza outbreaks at state or city level requires close monitoring of localized outbreaks and reliable forecasts of their progression. Whereas forecasting models for influenza or influenza-like illness (ILI) are becoming increasingly available, their applicability to localized outbreaks is limited by the nonavailability of real-time observations of the current outbreak state at local scales. Surveillance data collected by various health departments are widely accepted as the reference standard for estimating the state of outbreaks, and in the absence of surveillance data, nowcast proxies built using Web-based activities such as search engine queries, tweets, and access of health-related webpages can be useful. Nowcast estimates of state and municipal ILI were previously published by Google Flu Trends (GFT); however, validations of these estimates were seldom reported. Objective: The aim of this study was to develop and validate models to nowcast ILI at subregional geographic scales. Methods: We built nowcast models based on autoregressive (autoregressive integrated moving average; ARIMA) and supervised regression methods (Random forests) at the US state level using regional weighted ILI and Web-based search activity derived from Google's Extended Trends application programming interface. We validated the performance of these methods using actual surveillance data for the 50 states across six seasons. We also built state-level nowcast models using state-level estimates of ILI and compared the accuracy of these estimates with the estimates of the regional models extrapolated to the state level and with the nowcast estimates published by GFT. Results: Models built using regional ILI extrapolated to state level had a median correlation of 0.84 (interquartile range: 0.74-0.91) and a median root mean square error (RMSE) of 1.01 (IQR: 0.74-1.50), with noticeable variability across seasons and by state population size. Model forms that hypothesize the availability of timely state-level surveillance data show significantly lower errors of 0.83 (0.55-0.23). Compared with GFT, the latter model forms have lower errors but also lower correlation. Conclusions: These results suggest that the proposed methods may be an alternative to the discontinued GFT and that further improvements in the quality of subregional nowcasts may require increased access to more finely resolved surveillance data

Type- and Subtype-Specific Influenza Forecast

Prediction of the growth and decline of infectious disease incidence has advanced considerably in recent years. As these forecasts improve, their public health utility should increase, particularly as interventions are developed that make explicit use of forecast information. It is the task of the research community to increase the content and improve the accuracy of these infectious disease predictions. Presently, operational real-time forecasts of total influenza incidence are produced at the municipal and state level in the United States. These forecasts are generated using ensemble simulations depicting local influenza transmission dynamics, which have been optimized prior to forecast with observations of influenza incidence and data assimilation methods. Here, we explore whether forecasts targeted to predict influenza by type and subtype during 2003-2015 in the United States were more or less accurate than forecasts targeted to predict total influenza incidence. We found that forecasts separated by type/subtype generally produced more accurate predictions and, when summed, produced more accurate predictions of total influenza incidence. These findings indicate that monitoring influenza by type and subtype not only provides more detailed observational content but supports more accurate forecasting. More accurate forecasting can help officials better respond to and plan for current and future influenza activity

Differential COVID‐19 case positivity in New York City neighborhoods: Socioeconomic factors and mobility

Background: New York City (NYC) has been one of the hotspots of the COVID-19 pandemic in the United States. By the end of April 2020, close to 165 000 cases and 13 000 deaths were reported in the city with considerable variability across the city's ZIP codes. Objectives: In this study, we examine: (a) the extent to which the variability in ZIP code-level case positivity can be explained by aggregate markers of socioeconomic status (SES) and daily change in mobility; and (b) the extent to which daily change in mobility independently predicts case positivity. Methods: COVID-19 case positivity by ZIP code was modeled using multivariable linear regression with generalized estimating equations to account for within-ZIP clustering. Daily case positivity was obtained from NYC Department of Health and Mental Hygiene and measures of SES were based on data from the American Community Survey. Changes in human mobility were estimated using anonymized aggregated mobile phone location systems. Results: Our analysis indicates that the socioeconomic markers considered together explained 56% of the variability in case positivity through April 1 and their explanatory power decreased to 18% by April 30. Changes in mobility during this time period are not likely to be acting as a mediator of the relationship between ZIP-level SES and case positivity. During the middle of April, increases in mobility were independently associated with decreased case positivity. Conclusions: Together, these findings present evidence that heterogeneity in COVID-19 case positivity during NYC's spring outbreak was largely driven by residents' SES

Improved forecasts of influenza-associated hospitalization rates with Google Search Trends

Reliable forecasts of influenza-associated hospitalizations during seasonal outbreaks can help health systems better prepare for patient surges. Within the USA, public health surveillance systems collect and distribute near real-time weekly hospitalization rates, a key observational metric that makes real-time forecast of this outcome possible. In this paper, we describe a method to forecast hospitalization rates using a population level transmission model in combination with a data assimilation technique. Using this method, we generated retrospective forecasts of hospitalization rates for five age groups and the overall population during five seasons in the USA and quantified forecast accuracy for both near-term and seasonal targets. Additionally, we describe methods to correct for under-reporting of hospitalization rates (backcast) and to estimate hospitalization rates from publicly available online search trends data (nowcast). Forecasts based on surveillance rates alone were reasonably accurate in predicting peak hospitalization rates (within ± 25% of the actual peak rate, three weeks before peak). The error in predicting rates one to four weeks ahead, remained constant for the duration of the seasons, even during periods of increased influenza incidence. An improvement in forecast quality across all age groups, seasons and targets was observed when backcasts and nowcasts supplemented surveillance data. These results suggest that the model-inference framework can provide reasonably accurate real-time forecasts of influenza hospitalizations; backcasts and nowcasts offer a way to improve system tolerance to observational errors

Evaluation of mechanistic and statistical methods in forecasting influenza-like illness

A variety of mechanistic and statistical methods to forecast seasonal influenza have been proposed and are in use; however, the effects of various data issues and design choices (statistical versus mechanistic methods, for example) on the accuracy of these approaches have not been thoroughly assessed. Here, we compare the accuracy of three forecasting approaches-a mechanistic method, a weighted average of two statistical methods and a super-ensemble of eight statistical and mechanistic models-in predicting seven outbreak characteristics of seasonal influenza during the 2016-2017 season at the national and 10 regional levels in the USA. For each of these approaches, we report the effects of real time under- and over-reporting in surveillance systems, use of non-surveillance proxies of influenza activity and manual override of model predictions on forecast quality. Our results suggest that a meta-ensemble of statistical and mechanistic methods has better overall accuracy than the individual methods. Supplementing surveillance data with proxy estimates generally improves the quality of forecasts and transient reporting errors degrade the performance of all three approaches considerably. The improvement in quality from ad hoc and post-forecast changes suggests that domain experts continue to possess information that is not being sufficiently captured by current forecasting approaches

Characteristics and Outcomes among Older HIV-Positive Adults Enrolled in HIV Programs in Four Sub-Saharan African Countries

Limited information exists on adults ≥50 years receiving HIV care in sub-Saharan Africa. Using routinely-collected longitudinal patient-level data among 391,111 adults ≥15 years enrolling in HIV care from January 2005–December 2010 and 184,689 initiating ART, we compared characteristics and outcomes between older (≥50 years) and younger adults at 199 clinics in Kenya, Mozambique, Rwanda, and Tanzania. We calculated proportions over time of newly enrolled and active adults receiving HIV care and initiating ART who were ≥50 years; cumulative incidence of loss to follow-up (LTF) and recorded death one year after enrollment and ART initiation, and CD4+ response following ART initiation. From 2005–2010, the percentage of adults ≥50 years newly enrolled in HIV care remained stable at 10%, while the percentage of adults ≥50 years newly initiating ART (10% [2005]-12% [2010]), active in follow-up (10% [2005]-14% (2010]), and active on ART (10% [2005]-16% [2010]) significantly increased. One year after enrollment, older patients had significantly lower incidence of LTF (33.1% vs. 32.6%[40–49 years], 40.5%[25–39 years], and 56.3%[15–24 years]; p-value<0.0001), but significantly higher incidence of recorded death (6.0% vs. 5.0% [40–49 years], 4.1% [25–39 years], and 2.8% [15–24 years]; p-valve<0.0001). LTF was lower after vs. before ART initiation for all ages, with older adults experiencing less LTF than younger adults. Among 85,763 ART patients with baseline and follow-up CD4+ counts, adjusted average 12-month CD4+ response for older adults was 20.6 cells/mm3 lower than for adults 25–39 years of age (95% CI: 17.1–24.1). The proportion of patients who are ≥50 years has increased over time and been driven by aging of the existing patient population. Older patients experienced less LTF, higher recorded mortality and less robust CD4+ response after ART initiation. Increased programmatic attention on older adults receiving HIV care in sub-Saharan Africa is warranted

Estimating the infection-fatality risk of SARS-CoV-2 in New York City during the spring 2020 pandemic wave: a model-based analysis

Background As the COVID-19 pandemic continues to unfold, the infection-fatality risk (ie, risk of death among all infected individuals including those with asymptomatic and mild infections) is crucial for gauging the burden of death due to COVID-19 in the coming months or years. Here, we estimate the infection-fatality risk of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in New York City, NY, USA, the first epidemic centre in the USA, where the infection-fatality risk remains unclear. Methods In this model-based analysis, we developed a meta-population network model-inference system to estimate the underlying SARS-CoV-2 infection rate in New York City during the 2020 spring pandemic wave using available case, mortality, and mobility data. Based on these estimates, we further estimated the infection-fatality risk for all ages overall and for five age groups (<25, 25–44, 45–64, 65–74, and ≥75 years) separately, during the period March 1 to June 6, 2020 (ie, before the city began a phased reopening). Findings During the period March 1 to June 6, 2020, 205 639 people had a laboratory-confirmed infection with SARS-CoV-2 and 21 447 confirmed and probable COVID-19-related deaths occurred among residents of New York City. We estimated an overall infection-fatality risk of 1·39% (95% credible interval 1·04–1·77) in New York City. Our estimated infection-fatality risk for the two oldest age groups (65–74 and ≥75 years) was much higher than the younger age groups, with a cumulative estimated infection-fatality risk of 0·116% (0·0729–0·148) for those aged 25–44 years and 0·939% (0·729–1·19) for those aged 45–64 years versus 4·87% (3·37–6·89) for those aged 65–74 years and 14·2% (10·2–18·1) for those aged 75 years and older. In particular, weekly infection-fatality risk was estimated to be as high as 6·72% (5·52–8·01) for those aged 65–74 years and 19·1% (14·7–21·9) for those aged 75 years and older. Interpretation Our results are based on more complete ascertainment of COVID-19-related deaths in New York City than other places and thus probably reflect the true higher burden of death due to COVID-19 than that previously reported elsewhere. Given the high infection-fatality risk of SARS-CoV-2, governments must account for and closely monitor the infection rate and population health outcomes and enact prompt public health responses accordingly as the COVID-19 pandemic unfolds