Machine learning to refine decision making within a syndromic surveillance service

Background: Worldwide, syndromic surveillance is increasingly used for improved and timely situational awareness and early identification of public health threats. Syndromic data streams are fed into detection algorithms, which produce statistical alarms highlighting potential activity of public health importance. All alarms must be assessed to confirm whether they are of public health importance. In England, approximately 100 alarms are generated daily and, although their analysis is formalised through a risk assessment process, the process requires notable time, training, and maintenance of an expertise base to determine which alarms are of public health importance. The process is made more complicated by the observation that only 0.1% of statistical alarms are deemed to be of public health importance. Therefore, the aims of this study were to evaluate machine learning as a tool for computer-assisted human decision-making when assessing statistical alarms. Methods: A record of the risk assessment process was obtained from Public Health England for all 67505 statistical alarms between August 2013 and October 2015. This record contained information on the characteristics of the alarm (e.g. size, location). We used three Bayesian classifiers- naïve Bayes, tree-augmented naïve Bayes and Multinets - to examine the risk assessment record in England with respect to the final ‘Decision’ outcome made by an epidemiologist of ‘Alert’, ‘Monitor’ or ‘No-action’. Two further classifications based upon tree-augmented naïve Bayes and Multinets were implemented to account for the predominance of ‘No-action’ outcomes. Results: The attributes of each individual risk assessment were linked to the final decision made by an epidemiologist, providing confidence in the current process. The naïve Bayesian classifier performed best, correctly classifying 51.5% of ‘Alert’ outcomes. If the ‘Alert’ and ‘Monitor’ actions are combined then performance increases to 82.6% correctly classified. We demonstrate how a decision support system based upon a naïve Bayes classifier could be operationalised within an operational syndromic surveillance system. Conclusions: Within syndromic surveillance systems, machine learning techniques have the potential to make risk assessment following statistical alarms more automated, robust, and rigorous. However, our results also highlight the importance of specialist human input to the process

Can syndromic surveillance help forecast winter hospital bed pressures in England?

BACKGROUND: Health care planners need to predict demand for hospital beds to avoid deterioration in health care. Seasonal demand can be affected by respiratory illnesses which in England are monitored using syndromic surveillance systems. Therefore, we investigated the relationship between syndromic data and daily emergency hospital admissions. METHODS: We compared the timing of peaks in syndromic respiratory indicators and emergency hospital admissions, between 2013 and 2018. Furthermore, we created forecasts for daily admissions and investigated their accuracy when real-time syndromic data were included. RESULTS: We found that syndromic indicators were sensitive to changes in the timing of peaks in seasonal disease, especially influenza. However, each year, peak demand for hospital beds occurred on either 29th or 30th December, irrespective of the timing of syndromic peaks. Most forecast models using syndromic indicators explained over 70% of the seasonal variation in admissions (adjusted R square value). Forecast errors were reduced when syndromic data were included. For example, peak admissions for December 2014 and 2017 were underestimated when syndromic data were not used in models. CONCLUSION: Due to the lack of variability in the timing of the highest seasonal peak in hospital admissions, syndromic surveillance data do not provide additional early warning of timing. However, during atypical seasons syndromic data did improve the accuracy of forecast intensity

Using emergency department syndromic surveillance to investigate the impact of a national vaccination program: A retrospective observational study

BackgroundRotavirus infection is a common cause of gastroenteritis in children worldwide, with a high mortality burden in developing countries, particularly during the first two years of life. Rotavirus vaccination was introduced into the United Kingdom childhood vaccination schedule in July 2013, with high coverage (>90%) achieved by June 2016. We used an emergency department (ED) syndromic surveillance system to assess the impact of the rotavirus vaccination programme, specifically through the demonstration of any immediate and continuing impact on ED gastroenteritis visits in England.MethodsThis retrospective, observational study used syndromic surveillance data collected from 3 EDs in the two years before (July 2011-June 2013) and 3 years post (July 2013-June 2016) introduction of rotavirus vaccination. The weekly levels of ED visits for gastroenteritis (by age group and in total) during the period before rotavirus vaccination was first described alongside the findings of laboratory surveillance of rotavirus during the same period. An interrupted time-series analysis was then performed to demonstrate the impact of rotavirus vaccination introduction on gastroenteritis ED visit levels.ResultsDuring the two years before vaccine introduction ED visits for gastroenteritis in total and for the 0-4 years age group were seen to rise and fall in line with the seasonal rotavirus increases reported by laboratory surveillance. ED gastroenteritis visits by young children were lower in the three years following introduction of rotavirus vaccination (reduced from 8% of visits to 6% of visits). These attendance levels in young children (0-4years) remained higher than in older age groups, however the previously large seasonal increases in children were greatly reduced, from peaks of 16% to 3-10% of ED visits per week.ConclusionsED syndromic surveillance demonstrated a reduction in gastroenteritis visits following rotavirus vaccine introduction. This work establishes ED syndromic surveillance as a platform for rapid impact assessment of future vaccine programmes

The influence of a major sporting event upon emergency department attendances; A retrospective cross-national European study

Major sporting events may influence attendance levels at hospital emergency departments (ED). Previous research has focussed on the impact of single games, or wins/losses for specific teams/countries, limiting wider generalisations. Here we explore the impact of the Euro 2016 football championships on ED attendances across four participating nations (England, France, Northern Ireland, Wales), using a single methodology. Match days were found to have no significant impact upon daily ED attendances levels. Focussing upon hourly attendances, ED attendances across all countries in the four hour pre-match period were statistically significantly lower than would be expected (OR 0.97, 95% CI 0.94–0.99) and further reduced during matches (OR 0.94, 95% CI 0.91–0.97). In the 4 hour post-match period there was no significant increase in attendances (OR 1.01, 95% CI 0.99–1.04). However, these impacts were highly variable between individual matches: for example in the 4 hour period following the final, involving France, the number of ED attendances in France increased significantly (OR 1.27, 95% CI 1.13–1.42). Overall our results indicate relatively small impacts of major sporting events upon ED attendances. The heterogeneity observed makes it difficult for health providers to predict how major sporting events may affect ED attendances but supports the future development of compatible systems in different countries to support cross-border public health surveillance

Comparison of statistical algorithms for daily syndromic surveillance aberration detection

Motivation: Public health authorities can provide more effective and timely interventions to protect populations during health events if they have effective multi-purpose surveillance systems. These systems rely on aberration detection algorithms to identify potential threats within large datasets. Ensuring the algorithms are sensitive, specific and timely is crucial for protecting public health. Here, we evaluate the performance of three detection algorithms extensively used for syndromic surveillance: the ‘rising activity, multilevel mixed effects, indicator emphasis’ (RAMMIE) method and the improved quasi-Poisson regression-based method known as ‘Farrington Flexible’ both currently used at Public Health England, and the ‘Early Aberration Reporting System’ (EARS) method used at the US Centre for Disease Control and Prevention. We model the wide range of data structures encountered within the daily syndromic surveillance systems used by PHE. We undertake extensive simulations to identify which algorithms work best across different types of syndromes and different outbreak sizes. We evaluate RAMMIE for the first time since its introduction. Performance metrics were computed and compared in the presence of a range of simulated outbreak types that were added to baseline data. Results: We conclude that amongst the algorithm variants that have a high specificity (i.e. ¿90%), Farrington Flexible has the highest sensitivity and specificity, whereas RAMMIE has the highest probability of outbreak detection and is the most timely, typically detecting outbreaks 2-3 days earlier. Availability and Implementation: R codes developed for this project are available through https://github.com/FelipeJColon/AlgorithmCompariso

Syndromic surveillance: two decades experience of sustainable systems – its people not just data!

Syndromic surveillance is a form of surveillance that generates information for public health action by collecting, analysing and interpreting routine health-related data on symptoms and clinical signs reported by patients and clinicians rather than being based on microbiologically or clinically confirmed cases. In England, a suite of national real-time syndromic surveillance systems (SSS) have been developed over the last 20 years, utilising data from a variety of health care settings (a telehealth triage system, general practice and emergency departments). The real-time systems in England have been used for early detection (e.g. seasonal influenza), for situational awareness (e.g. describing the size and demographics of the impact of a heatwave) and for reassurance of lack of impact on population health of mass gatherings (e.g. the London 2012 Olympic and Paralympic Games).We highlight the lessons learnt from running SSS, for nearly two decades, and propose questions and issues still to be addressed. We feel that syndromic surveillance is an example of the use of ‘big data’, but contend that the focus for sustainable and useful systems should be on the added value of such systems and the importance of people working together to maximise the value for the public health of syndromic surveillance services

A Methodological Framework for the Evaluation of Syndromic Surveillance Systems: A Case Study of England

Background: Syndromic surveillance complements traditional public health surveillance by collecting and analysing health indicators in near real time. The rationale of syndromic surveillance is that it may detect health threats faster than traditional surveillance systems permitting more timely, and hence potentially more effective public health action. The effectiveness of syndromic surveillance largely relies on the methods used to detect aberrations. Very few studies have evaluated the performance of syndromic surveillance systems and consequently little is known about the types of events that such systems can and cannot detect. Methods: We introduce a framework for the evaluation of syndromic surveillance systems that can be used in any setting based upon the use of simulated scenarios. For a range of scenarios this allows the time and probability of to be determined and uncertainty is fully incorporated. In addition, we demonstrate how such a framework can model the benefits of increases in the number of centres reporting syndromic data and also determine the minimum size of outbreaks that can or cannot be detected. Here, we demonstrate its utility using simulations of national influenza outbreaks and localised outbreaks of cryptosporidiosis. Results: Influenza outbreaks are consistently detected with larger outbreaks being detected in a more timely manner. Small cryptosporidiosis outbreaks (<1000 symptomatic individuals) are unlikely to be detected. We also demonstrate the advantages of having multiple syndromic data streams (e.g. emergency attendance data, telephone helpline data, general practice consultation data) as different streams are able to detect different types outbreaks with different efficacy (e.g. emergency attendance data are useful for the detection of pandemic influenza but not for outbreaks of cryptosporidiosis). We also highlight that for any one disease, the utility of data streams may vary geographically, and that the detection ability of syndromic surveillance varies seasonally (e.g. an influenza outbreak starting in July is detected sooner than one starting later in the year). We argue that our framework constitutes a useful tool for public health emergency preparedness in multiple settings. Conclusions: The proposed framework allows the exhaustive evaluation of any syndromic surveillance system and constitutes a useful tool for emergency preparedness and response

The Epidemiology of Chickenpox in England, 2016-2022: An Observational Study Using General Practitioner Consultations.

Chickenpox is a common childhood disease caused by varicella-zoster virus (VZV). VZV vaccination is not part of the UK childhood immunisation programme, but its potential inclusion is regularly assessed. It is therefore important to understand the ongoing burden of VZV in the community to inform vaccine policy decisions. General practitioner (GP) chickenpox consultations were studied from 1 September 2016 to 9 December 2022. Over the study period, the mean weekly chickenpox consultation rate per 100,000 population in England was 3.4, with a regular peak occurring between weeks 13 and 15. Overall, rates decreased over time, from a mean weekly rate of 5.5 in 2017 to 4.2 in 2019. The highest mean weekly rates were among children aged 1-4 years. There was no typical epidemic peak during the COVID-19 pandemic, but in 2022, rates were proportionally higher among children aged < 1 year old compared to pre-pandemic years. Chickenpox GP consultation rates decreased in England, continuing a longer-term decline in the community. The COVID-19 pandemic impacted rates, likely caused by the introduction of non-pharmaceutical interventions to prevent SARS-CoV-2 transmission. The lasting impact of the interruption of typical disease transmission remains to be seen, but it is important to monitor the chickenpox burden to inform decisions on vaccine programmes

Demographic and socioeconomic patterns in healthcare-seeking behaviour for respiratory symptoms in England: a comparison with non-respiratory symptoms and between three healthcare services.

OBJECTIVE: This study will analyse respiratory contacts to three healthcare services that capture more of the community disease burden than acute data sources, such as hospitalisations. The objective is to explore associations between contacts to these services and the patient's age, gender and deprivation. Results will be compared between healthcare services, and with non-respiratory contacts to explore how contacts differ by service and illness. It is crucial to investigate the sociodemographic patterns in healthcare-seeking behaviour to enable targeted public health interventions. DESIGN: Ecological study. SETTING: Surveillance of respiratory contacts to three healthcare services in England: telehealth helpline (NHS111); general practitioner in-hours (GPIH); and general practitioner out of hours unscheduled care (GPOOH). PARTICIPANTS: 13 million respiratory contacts to NHS111, GPIH and GPOOH. OUTCOME MEASURES: Respiratory contacts to NHS111, GPIH and GPOOH, and non-respiratory contacts to NHS111 and GPOOH. RESULTS: More respiratory contacts were observed for females, with 1.59, 1.73, and 1.95 times the rate of contacts to NHS111, GPOOH and GPIH, respectively. When compared with 15-44 year olds, there were 37.32, 18.66 and 6.21 times the rate of respiratory contacts to NHS111, GPOOH and GPIH in children <1 year. There were 1.75 and 2.70 times the rate of respiratory contacts in the most deprived areas compared with the least deprived to NHS111 and GPOOH. Elevated respiratory contacts were observed for males <5 years compared with females <5 years. Healthcare-seeking behaviours between respiratory and non-respiratory contacts were similar. CONCLUSION: When contacts to services that capture more of the disease burden are explored, the demographic patterns are similar to those described in the literature for acute systems. Comparable results were observed between respiratory and non-respiratory contacts suggesting that when a wider spectrum of disease is explored, sociodemographic factors may be the strongest influencers of healthcare-seeking behaviour

Machine learning forecasts for seasonal epidemic peaks: Lessons learnt from an atypical respiratory syncytial virus season.

Seasonal peaks in infectious disease incidence put pressures on health services. Therefore, early warning of the timing and magnitude of peak activity during seasonal epidemics can provide information for public health practitioners to take appropriate action. Whilst many infectious diseases have predictable seasonality, newly emerging diseases and the impact of public health interventions can result in unprecedented seasonal activity. We propose a Machine Learning process for generating short-term forecasts, where models are selected based on their ability to correctly forecast peaks in activity, and can be useful during atypical seasons. We have validated our forecasts using typical and atypical seasonal activity, using respiratory syncytial virus (RSV) activity during 2019-2021 as an example. During the winter of 2020/21 the usual winter peak in RSV activity in England did not occur but was 'deferred' until the Spring of 2021. We compare a range of Machine Learning regression models, with alternate models including different independent variables, e.g. with or without seasonality or trend variables. We show that the best-fitting model which minimises daily forecast errors is not the best model for forecasting peaks when the selection criterion is based on peak timing and magnitude. Furthermore, we show that best-fitting models for typical seasons contain different variables to those for atypical seasons. Specifically, including seasonality in models improves performance during typical seasons but worsens it for the atypical seasons