Towards cross-lingual alerting for bursty epidemic events

Background: Online news reports are increasingly becoming a source for event based early warning systems that detect natural disasters. Harnessing the massive volume of information available from multilingual newswire presents as many challenges as opportunities due to the patterns of reporting complex spatiotemporal events. Results: In this article we study the problem of utilising correlated event reports across languages. We track the evolution of 16 disease outbreaks using 5 temporal aberration detection algorithms on text-mined events classified according to disease and outbreak country. Using ProMED reports as a silver standard, comparative analysis of news data for 13 languages over a 129 day trial period showed improved sensitivity, F1 and timeliness across most models using cross-lingual events. We report a detailed case study analysis for Cholera in Angola 2010 which highlights the challenges faced in correlating news events with the silver standard. Conclusions: The results show that automated health surveillance using multilingual text mining has the potential to turn low value news into high value alerts if informed choices are used to govern the selection of models and data sources. An implementation of the C2 alerting algorithm using multilingual news is available at the BioCaster portal http://born.nii.ac.jp/?page=globalroundup

What's unusual in online disease outbreak news?

Background: Accurate and timely detection of public health events of international concern is necessary to help support risk assessment and response and save lives. Novel event-based methods that use the World Wide Web as a signal source offer potential to extend health surveillance into areas where traditional indicator networks are lacking. In this paper we address the issue of systematically evaluating online health news to support automatic alerting using daily disease-country counts text mined from real world data using BioCaster. For 18 data sets produced by BioCaster, we compare 5 aberration detection algorithms (EARS C2, C3, W2, F-statistic and EWMA) for performance against expert moderated ProMED-mail postings. Results: We report sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), mean alerts/100 days and F1, at 95% confidence interval (CI) for 287 ProMED-mail postings on 18 outbreaks across 14 countries over a 366 day period. Results indicate that W2 had the best F1 with a slight benefit for day of week effect over C2. In drill down analysis we indicate issues arising from the granular choice of country-level modeling, sudden drops in reporting due to day of week effects and reporting bias. Automatic alerting has been implemented in BioCaster available from http://born.nii.ac.jp. Conclusions: Online health news alerts have the potential to enhance manual analytical methods by increasing throughput, timeliness and detection rates. Systematic evaluation of health news aberrations is necessary to push forward our understanding of the complex relationship between news report volumes and case numbers and to select the best performing features and algorithms

Enhanced health event detection and influenza surveillance using a joint Veterans Affairs and Department of Defense biosurveillance application

<p>Abstract</p> <p>Background</p> <p>The establishment of robust biosurveillance capabilities is an important component of the U.S. strategy for identifying disease outbreaks, environmental exposures and bioterrorism events. Currently, U.S. Departments of Defense (DoD) and Veterans Affairs (VA) perform biosurveillance independently. This article describes a joint VA/DoD biosurveillance project at North Chicago-VA Medical Center (NC-VAMC). The Naval Health Clinics-Great Lakes facility physically merged with NC-VAMC beginning in 2006 with the full merger completed in October 2010 at which time all DoD care and medical personnel had relocated to the expanded and remodeled NC-VAMC campus and the combined facility was renamed the Lovell Federal Health Care Center (FHCC). The goal of this study was to evaluate disease surveillance using a biosurveillance application which combined data from both populations.</p> <p>Methods</p> <p>A retrospective analysis of NC-VAMC/Lovell FHCC and other Chicago-area VAMC data was performed using the ESSENCE biosurveillance system, including one infectious disease outbreak (Salmonella/Taste of Chicago-July 2007) and one weather event (Heat Wave-July 2006). Influenza-like-illness (ILI) data from these same facilities was compared with CDC/Illinois Sentinel Provider and Cook County ESSENCE data for 2007-2008.</p> <p>Results</p> <p>Following consolidation of VA and DoD facilities in North Chicago, median number of visits more than doubled, median patient age dropped and proportion of females rose significantly in comparison with the pre-merger NC-VAMC facility. A high-level gastrointestinal alert was detected in July 2007, but only low-level alerts at other Chicago-area VAMCs. Heat-injury alerts were triggered for the merged facility in June 2006, but not at the other facilities. There was also limited evidence in these events that surveillance of the combined population provided utility above and beyond the VA-only and DoD-only components. Recorded ILI activity for NC-VAMC/Lovell FHCC was more pronounced in the DoD component, likely due to pediatric data in this population. NC-VAMC/Lovell FHCC had two weeks of ILI activity exceeding both the Illinois State and East North Central Regional baselines, whereas Hines VAMC had one and Jesse Brown VAMC had zero.</p> <p>Conclusions</p> <p>Biosurveillance in a joint VA/DoD facility showed potential utility as a tool to improve surveillance and situational awareness in an area with Veteran, active duty and beneficiary populations. Based in part on the results of this pilot demonstration, both agencies have agreed to support the creation of a combined VA/DoD ESSENCE biosurveillance system which is now under development.</p

Factors Influencing Performance of Internet-Based Biosurveillance Systems Used in Epidemic Intelligence for Early Detection of Infectious Diseases Outbreaks

Background: Internet-based biosurveillance systems have been developed to detect health threats using information available on the Internet, but system performance has not been assessed relative to end-user needs and perspectives. Method and Findings: Infectious disease events from the French Institute for Public Health Surveillance (InVS) weekly international epidemiological bulletin published in 2010 were used to construct the gold-standard official dataset. Data from six biosurveillance systems were used to detect raw signals (infectious disease events from informal Internet sources): Argus, BioCaster, GPHIN, HealthMap, MedISys and ProMED-mail. Crude detection rates (C-DR), crude sensitivity rates (C-Se) and intrinsic sensitivity rates (I-Se) were calculated from multivariable regressions to evaluate the systems’ performance (events detected compared to the gold-standard) 472 raw signals (Internet disease reports) related to the 86 events included in the gold-standard data set were retrieved from the six systems. 84 events were detected before their publication in the gold-standard. The type of sources utilised by the systems varied significantly (p,0001). I-Se varied significantly from 43% to 71% (p = 0001) whereas other indicators were similar (C-DR: p = 020; C-Se, p = 013). I-Se was significantly associated with individual systems, types of system, languages, regions of occurrence, and types of infectious disease. Conversely, no statistical difference of C-DR was observed after adjustment for other variables. Conclusion: Although differences could result from a biosurveillance system’s conceptual design, findings suggest that the combined expertise amongst systems enhances early detection performance for detection of infectious diseases. While all systems showed similar early detection performance, systems including human moderation were found to have a 53% higher I-Se (p = 00001) after adjustment for other variables. Overall, the use of moderation, sources, languages, regions of occurrence, and types of cases were found to influence system performance.JRC.G.2-Global security and crisis managemen

Anomaly Detection in Time Series: Theoretical and Practical Improvements for Disease Outbreak Detection

The automatic collection and increasing availability of health data provides a new opportunity for techniques to monitor this information. By monitoring pre-diagnostic data sources, such as over-the-counter cough medicine sales or emergency room chief complaints of cough, there exists the potential to detect disease outbreaks earlier than traditional laboratory disease confirmation results. This research is particularly important for a modern, highly-connected society, where the onset of disease outbreak can be swift and deadly, whether caused by a naturally occurring global pandemic such as swine flu or a targeted act of bioterrorism. In this dissertation, we first describe the problem and current state of research in disease outbreak detection, then provide four main additions to the field. First, we formalize a framework for analyzing health series data and detecting anomalies: using forecasting methods to predict the next day's value, subtracting the forecast to create residuals, and finally using detection algorithms on the residuals. The formalized framework indicates the link between the forecast accuracy of the forecast method and the performance of the detector, and can be used to quantify and analyze the performance of a variety of heuristic methods. Second, we describe improvements for the forecasting of health data series. The application of weather as a predictor, cross-series covariates, and ensemble forecasting each provide improvements to forecasting health data. Third, we describe improvements for detection. This includes the use of multivariate statistics for anomaly detection and additional day-of-week preprocessing to aid detection. Most significantly, we also provide a new method, based on the CuScore, for optimizing detection when the impact of the disease outbreak is known. This method can provide an optimal detector for rapid detection, or for probability of detection within a certain timeframe. Finally, we describe a method for improved comparison of detection methods. We provide tools to evaluate how well a simulated data set captures the characteristics of the authentic series and time-lag heatmaps, a new way of visualizing daily detection rates or displaying the comparison between two methods in a more informative way

Factors Influencing Performance of Internet-Based Biosurveillance Systems Used in Epidemic Intelligence for Early Detection of Infectious Diseases Outbreaks

Background: Internet-based biosurveillance systems have been developed to detect health threats using information available on the Internet, but system performance has not been assessed relative to end-user needs and perspectives. Method and Findings Infectious disease events from the French Institute for Public Health Surveillance (InVS) weekly international epidemiological bulletin published in 2010 were used to construct the gold-standard official dataset. Data from six biosurveillance systems were used to detect raw signals (infectious disease events from informal Internet sources): Argus, BioCaster, GPHIN, HealthMap, MedISys and ProMED-mail. Crude detection rates (C-DR), crude sensitivity rates (C-Se) and intrinsic sensitivity rates (I-Se) were calculated from multivariable regressions to evaluate the systems’ performance (events detected compared to the gold-standard) 472 raw signals (Internet disease reports) related to the 86 events included in the gold-standard data set were retrieved from the six systems. 84 events were detected before their publication in the gold-standard. The type of sources utilised by the systems varied significantly (p<0001). I-Se varied significantly from 43% to 71% (p = 0001) whereas other indicators were similar (C-DR: p = 020; C-Se, p = 013). I-Se was significantly associated with individual systems, types of system, languages, regions of occurrence, and types of infectious disease. Conversely, no statistical difference of C-DR was observed after adjustment for other variables. Conclusion: Although differences could result from a biosurveillance system's conceptual design, findings suggest that the combined expertise amongst systems enhances early detection performance for detection of infectious diseases. While all systems showed similar early detection performance, systems including human moderation were found to have a 53% higher I-Se (p = 00001) after adjustment for other variables. Overall, the use of moderation, sources, languages, regions of occurrence, and types of cases were found to influence system performance

Evaluation of Epidemic Intelligence Systems Integrated in the Early Alerting and Reporting Project for the Detection of A/H5N1 Influenza Events

Web-based expert systems dedicated to epidemic intelligence were developed to detect health threats. The Early Alerting and Reporting (EAR) project, launched under the Global Health Initiative, aimed at assessing the feasibility and opportunity of pooling seven of those expert systems. A qualitative survey was carried out with EAR participants to document epidemic intelligence strategies and to assess perceptions regarding the performance of participating systems. Timeliness and sensitivity were rated with high scores illustrating the overall perceived value of all systems while weaknesses were underlined especially in terms of representativeness, completeness and flexibility. These findings were corroborated by the quantitative analysis performed on signals potentially related to influenza A/H5N1 events which occurred in March 2010. For the six systems for which this information was available; the detection rate ranged from 31% to 38%, and increased to 72% when considering the virtual combined system. The positive predictive values (PPV) ranged from 3% to 24% and the F1-score ranged from 6% to 27%. These low scores point out false positive signals related to varying abilities of the systems to efficiently sort-out information and reduce background noise. For the seven systems sensitivity ranged from 38% to 72%. An average difference of 23% was observed between the sensitivities calculated for human cases and epizootics, underlining the difficulties to develop an efficient algorithm or a single pathology. The sensitivity increased to 93% when the virtual combined system was considered, clearly illustrating the systems’ complementarities. The average delay between the detection of the A/H5N1 events by the systems and their official reporting by WHO or OIE was 10.2 days (CI95%, 6.7; 13.8). This work illustrates the diversity in implemented epidemic intelligence activities, differences in systems designs and the potential added values and opportunities for synergy: between systems, between users and between systems and users.JRC.G.2-Global security and crisis managemen

Identification and characterization of diseases on social web

[no abstract

Syndromic surveillance using veterinary laboratory data : data pre-processing and algorithm performance evaluation

Diagnostic test orders to an animal laboratory were explored as a data source for monitoring trends in the incidence of clinical syndromes in cattle. Four years of real data and over 200 simulated outbreak signals were used to compare pre-processing methods that could remove temporal effects in the data, as well as temporal aberration detection algorithms that provided high sensitivity and specificity. Weekly differencing demonstrated solid performance in removing day-of-week effects, even in series with low daily counts. For aberration detection, the results indicated that no single algorithm showed performance superior to all others across the range of outbreak scenarios simulated. Exponentially weighted moving average charts and Holt-Winters exponential smoothing demonstrated complementary performance, with the latter offering an automated method to adjust to changes in the time series that will likely occur in the future. Shewhart charts provided lower sensitivity but earlier detection in some scenarios. Cumulative sum charts did not appear to add value to the system; however, the poor performance of this algorithm was attributed to characteristics of the data monitored. These findings indicate that automated monitoring aimed at early detection of temporal aberrations will likely be most effective when a range of algorithms are implemented in parallel

International Society for Disease Surveillance Conference 2011: Building the Future of Public Health Surveillance: Building the Future of Public Health Surveillance

Daniel Reidpath - ORCID: 0000-0002-8796-0420 https://orcid.org/0000-0002-8796-04204pubpub1117