A simulation study comparing aberration detection algorithms for syndromic surveillance

BACKGROUND: The usefulness of syndromic surveillance for early outbreak detection depends in part on effective statistical aberration detection. However, few published studies have compared different detection algorithms on identical data. In the largest simulation study conducted to date, we compared the performance of six aberration detection algorithms on simulated outbreaks superimposed on authentic syndromic surveillance data. METHODS: We compared three control-chart-based statistics, two exponential weighted moving averages, and a generalized linear model. We simulated 310 unique outbreak signals, and added these to actual daily counts of four syndromes monitored by Public Health – Seattle and King County's syndromic surveillance system. We compared the sensitivity of the six algorithms at detecting these simulated outbreaks at a fixed alert rate of 0.01. RESULTS: Stratified by baseline or by outbreak distribution, duration, or size, the generalized linear model was more sensitive than the other algorithms and detected 54% (95% CI = 52%–56%) of the simulated epidemics when run at an alert rate of 0.01. However, all of the algorithms had poor sensitivity, particularly for outbreaks that did not begin with a surge of cases. CONCLUSION: When tested on county-level data aggregated across age groups, these algorithms often did not perform well in detecting signals other than large, rapid increases in case counts relative to baseline levels

A systematic review of cluster detection mechanisms in syndromic surveillance: Towards developing a framework of cluster detection mechanisms for EDMON system

Source at http://www.ep.liu.se/ecp/151/011/ecp18151011.pdf.Time lag in detecting disease outbreaks remains a threat to global health security. Currently, our research team is working towards a system called EDMON, which uses blood glucose level and other supporting parameters from people with type 1 diabetes, as indicator variables for outbreak detection. Therefore, this paper aims to pinpoint the state of the art cluster detection mechanism towards developing an efficient framework to be used in EDMON and other similar syndromic surveillance systems. Various challenges such as user mobility, privacy and confidentiality, geographical location estimation and other factors have been considered. To this end, we conducted a systematic review exploring different online scholarly databases. Considering peer reviewed journals and articles, literatures search was conducted between January and March 2018. Relevant literatures were identified using the title, keywords, and abstracts as a preliminary filter with the inclusion criteria and a full text review were done for literatures that were found to be relevant. A total of 28 articles were included in the study. The result indicates that various clustering and aberration detection algorithms have been developed and tested up to the task. In this regard, privacy preserving policies and high computational power requirement were found challenging since it restrict usage of specific locations for syndromic surveillance

What's unusual in online disease outbreak news?

Background: Accurate and timely detection of public health events of international concern is necessary to help support risk assessment and response and save lives. Novel event-based methods that use the World Wide Web as a signal source offer potential to extend health surveillance into areas where traditional indicator networks are lacking. In this paper we address the issue of systematically evaluating online health news to support automatic alerting using daily disease-country counts text mined from real world data using BioCaster. For 18 data sets produced by BioCaster, we compare 5 aberration detection algorithms (EARS C2, C3, W2, F-statistic and EWMA) for performance against expert moderated ProMED-mail postings. Results: We report sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), mean alerts/100 days and F1, at 95% confidence interval (CI) for 287 ProMED-mail postings on 18 outbreaks across 14 countries over a 366 day period. Results indicate that W2 had the best F1 with a slight benefit for day of week effect over C2. In drill down analysis we indicate issues arising from the granular choice of country-level modeling, sudden drops in reporting due to day of week effects and reporting bias. Automatic alerting has been implemented in BioCaster available from http://born.nii.ac.jp. Conclusions: Online health news alerts have the potential to enhance manual analytical methods by increasing throughput, timeliness and detection rates. Systematic evaluation of health news aberrations is necessary to push forward our understanding of the complex relationship between news report volumes and case numbers and to select the best performing features and algorithms

Disease surveillance using a hidden Markov model

<p>Abstract</p> <p>Background</p> <p>Routine surveillance of disease notification data can enable the early detection of localised disease outbreaks. Although hidden Markov models (HMMs) have been recognised as an appropriate method to model disease surveillance data, they have been rarely applied in public health practice. We aimed to develop and evaluate a simple flexible HMM for disease surveillance which is suitable for use with sparse small area count data and requires little baseline data.</p> <p>Methods</p> <p>A Bayesian HMM was designed to monitor routinely collected notifiable disease data that are aggregated by residential postcode. Semi-synthetic data were used to evaluate the algorithm and compare outbreak detection performance with the established Early Aberration Reporting System (EARS) algorithms and a negative binomial cusum.</p> <p>Results</p> <p>Algorithm performance varied according to the desired false alarm rate for surveillance. At false alarm rates around 0.05, the cusum-based algorithms provided the best overall outbreak detection performance, having similar sensitivity to the HMMs and a shorter average time to detection. At false alarm rates around 0.01, the HMM algorithms provided the best overall outbreak detection performance, having higher sensitivity than the cusum-based Methods and a generally shorter time to detection for larger outbreaks. Overall, the 14-day HMM had a significantly greater area under the receiver operator characteristic curve than the EARS C3 and 7-day negative binomial cusum algorithms.</p> <p>Conclusion</p> <p>Our findings suggest that the HMM provides an effective method for the surveillance of sparse small area notifiable disease data at low false alarm rates. Further investigations are required to evaluation algorithm performance across other diseases and surveillance contexts.</p

Multivariate syndromic surveillance for cattle diseases: Epidemic simulation and algorithm performance evaluation.

Multivariate Syndromic Surveillance (SyS) systems that simultaneously assess and combine information from different data sources are especially useful for strengthening surveillance systems for early detection of infectious disease epidemics. Despite the strong motivation for implementing multivariate SyS and there being numerous methods reported, the number of operational multivariate SyS systems in veterinary medicine is still very small. One possible reason is that assessing the performance of such surveillance systems remains challenging because field epidemic data are often unavailable. The objective of this study is to demonstrate a practical multivariate event detection method (directionally sensitive multivariate control charts) that can be easily applied in livestock disease SyS, using syndrome time series data from the Swiss cattle population as an example. We present a standardized method for simulating multivariate epidemics of different diseases using four diseases as examples: Bovine Virus Diarrhea (BVD), Infectious Bovine Rhinotracheitis (IBR), Bluetongue virus (BTV) and Schmallenberg virus (SV). Two directional multivariate control chart algorithms, Multivariate Exponentially Weighted Moving Average (MEWMA) and Multivariate Cumulative Sum (MCUSUM) were compared. The two algorithms were evaluated using 12 syndrome time series extracted from two Swiss national databases. The two algorithms were able to detect all simulated epidemics around 4.5 months after the start of the epidemic, with a specificity of 95%. However, the results varied depending on the algorithm and the disease. The MEWMA algorithm always detected epidemics earlier than the MCUSUM, and epidemics of IBR and SV were detected earlier than epidemics of BVD and BTV. Our results show that the two directional multivariate control charts are promising methods for combining information from multiple time series for early detection of subtle changes in time series from a population without producing an unreasonable amount of false alarms. The approach that we used for simulating multivariate epidemics is relatively easy to implement and could be used in other situations where real epidemic data are unavailable. We believe that our study results can support the implementation and assessment of multivariate SyS systems in animal health

Applying cusum-based methods for the detection of outbreaks of Ross River virus disease in Western Australia

<p>Abstract</p> <p>Background</p> <p>The automated monitoring of routinely collected disease surveillance data has the potential to ensure that important changes in disease incidence are promptly recognised. However, few studies have established whether the signals produced by automated monitoring methods correspond with events considered by epidemiologists to be of public health importance. This study investigates the correspondence between retrospective epidemiological evaluation of notifications of Ross River virus (RRv) disease in Western Australia, and the signals produced by two cumulative sum (cusum)-based automated monitoring methods.</p> <p>Methods</p> <p>RRv disease case notification data between 1991 and 2004 were assessed retrospectively by two experienced epidemiologists, and the timing of identified outbreaks was compared with signals generated from two different types of cusum-based automated monitoring algorithms; the three Early Aberration Reporting System (EARS) cusum algorithms (C1, C2 and C3), and a negative binomial cusum.</p> <p>Results</p> <p>We found the negative binomial cusum to have a significantly greater area under the receiver operator characteristic curve when compared with the EARS algorithms, suggesting that the negative binomial cusum has a greater level of agreement with epidemiological opinion than the EARS algorithms with respect to the existence of outbreaks of RRv disease, particularly at low false alarm rates. However, the performance of individual EARS and negative binomial cusum algorithms were not significantly different when timeliness was also incorporated into the area under the curve analyses.</p> <p>Conclusion</p> <p>Our retrospective analysis of historical data suggests that, compared with the EARS algorithms, the negative binomial cusum provides greater sensitivity for the detection of outbreaks of RRv disease at low false alarm levels, and decreased timeliness early in the outbreak period. Prospective studies are required to investigate the potential usefulness of these algorithms in practice.</p

K-CUSUM: Cluster Detection Mechanism in EDMON

Source at https://www.ep.liu.se/ecp/contents.asp?issue=161. The main goal of the EDMON (Electronic Disease Monitoring Network) project is to detect the spread of contagious diseases at the earliest possible moment, and potentially before people know that they have been infected. The results shall be visualized on real-time maps as well as presented in digital communication. In this paper, a hybrid of K-nearness Neighbor (KNN) and cumulative sum (CUSUM), known as K-CUSUM, were explored and implemented with a prototype approach. The KNN algorithm, which was implemented in the K- CUSUM, recorded 99.52% accuracy when it was tested with simulated dataset containing geolocation coordinates among other features and SckitLearn KNN algorithm achieved an accuracy of 93.81% when it was tested with the same dataset. After injection of spikes of known outbreaks in the simulated data, the CUSUM module was totally specific and sensitive by correctly identifying all outbreaks and non-outbreak clusters. Suitable methods for obtaining a balance point of anonymizing geolocation attributes towards obscuring the privacy and confidentiality of diabetes subjects’ trajectories while maintaining the data requirements for public good, in terms of disease surveillance, remains a challenge