Factors Influencing Performance of Internet-Based Biosurveillance Systems Used in Epidemic Intelligence for Early Detection of Infectious Diseases Outbreaks

Background: Internet-based biosurveillance systems have been developed to detect health threats using information available on the Internet, but system performance has not been assessed relative to end-user needs and perspectives. Method and Findings Infectious disease events from the French Institute for Public Health Surveillance (InVS) weekly international epidemiological bulletin published in 2010 were used to construct the gold-standard official dataset. Data from six biosurveillance systems were used to detect raw signals (infectious disease events from informal Internet sources): Argus, BioCaster, GPHIN, HealthMap, MedISys and ProMED-mail. Crude detection rates (C-DR), crude sensitivity rates (C-Se) and intrinsic sensitivity rates (I-Se) were calculated from multivariable regressions to evaluate the systems’ performance (events detected compared to the gold-standard) 472 raw signals (Internet disease reports) related to the 86 events included in the gold-standard data set were retrieved from the six systems. 84 events were detected before their publication in the gold-standard. The type of sources utilised by the systems varied significantly (p<0001). I-Se varied significantly from 43% to 71% (p = 0001) whereas other indicators were similar (C-DR: p = 020; C-Se, p = 013). I-Se was significantly associated with individual systems, types of system, languages, regions of occurrence, and types of infectious disease. Conversely, no statistical difference of C-DR was observed after adjustment for other variables. Conclusion: Although differences could result from a biosurveillance system's conceptual design, findings suggest that the combined expertise amongst systems enhances early detection performance for detection of infectious diseases. While all systems showed similar early detection performance, systems including human moderation were found to have a 53% higher I-Se (p = 00001) after adjustment for other variables. Overall, the use of moderation, sources, languages, regions of occurrence, and types of cases were found to influence system performance

Use of media and public-domain Internet sources for detection and assessment of plant health threats

Event-based biosurveillance is a recognized approach to early warning and situational awareness of emerging health threats. In this study, we build upon previous human and animal health work to develop a new approach to plant pest and pathogen surveillance. We show that monitoring public domain electronic media for indications and warning of epidemics and associated social disruption can provide information about the emergence and progression of plant pest infestation or disease outbreak. The approach is illustrated using a case study, which describes a plant pest and pathogen epidemic in China and Vietnam from February 2006 to December 2007, and the role of ducks in contributing to zoonotic virus spread in birds and humans. This approach could be used as a complementary method to traditional plant pest and pathogen surveillance to aid global and national plant protection officials and political leaders in early detection and timely response to significant biological threats to plant health, economic vitality, and social stability. This study documents the inter-relatedness of health in human, animal, and plant populations and emphasizes the importance of plant health surveillance

Evaluation of Epidemic Intelligence Systems Integrated in the Early Alerting and Reporting Project for the Detection of A/H5N1 Influenza Events

Peer reviewe

Measuring trends in hepatitis C testing with commercial laboratory data

ObjectiveUsing the two largest commercial laboratory data sources nationally, we estimated the annual rates of hepatitis C testing among individuals who were recommended to be tested (i.e., baby boomer cohort born between 1945 and 1965) by the CDC and United States Preventive Services Task Force. This panel will discuss strengths and weaknesses for monitoring hepatitis C testing using alternative data sources including self-reported data, insurance claims data, and laboratory testing data.IntroductionHepatitis C virus (HCV) infection is a leading cause of liver disease-related morbidity and mortality in the United States. Approximately 75% of people infected with chronic HCV were born between 1945 and 1965. Since 2012, the CDC has recommended one-time screening for chronic HCV infection for all persons in this birth cohort (baby boomers). The United States Preventive Services Task Force (USPSTF) subsequently made the same recommendation in June 2013. We estimated the rate of HCV testing between 2011 and 2017 among persons with commercial health insurance coverage and compared rates by birth cohort.MethodsHepatitis C virus testing data were obtained from Quest Diagnostics (Quest) and Laboratory Corporation of America (LabCorp), two large U.S. commercial laboratories serving clinicians and hospitals in all 50 U.S states and the District of Columbia. Analysis was based on de-identified person-level data from HCV antibody immunoassay tests ordered by clinicians in the U.S. between 2011 and in 2017 (with LabCorp data in 2017 limited to January through October). HCV antibody testing rates were calculated and defined as: the number of unique individuals who received their first HCV antibody test during a particular month per 100 unique individuals who had any laboratory test performed by the commercial laboratory during the same month, presented as an annual average (mean) testing rate. Persons born between 1945 and 1965 were classified as baby boomers and compared to persons born in all other years.ResultsIn 2011, prior to the CDC recommendation change, rates of HCV antibody testing relative to overall testing with each cohort were higher for the non-baby boomer cohort served by both Quest and LabCorp. In contrast, from 2012 thorugh2017, testing was more frequent among baby boomers than among non-baby boomers as a proportion of overall testing in each cohort. The rate of testing among baby boomers served by Quest rose from 1.7 per 100 test requests in 2011 to 3.8 per 100, an increase of 131%, while the rate of testing among non-baby boomers rose from 2.3 per 100 to 3.1 per 100, a 35% increase. Changes among patients served by LabCorp were nearly identical; a 132% increase among baby boomers (1.7 per 100 in 2011 to 4.0 per 100 in 2017) and a 31% increase among non-baby boomers (1.7 per 100 in 2011 to 3.2 per 100 in 2017).ConclusionsThis study demonstrates the utility of commercial laboratory data for assessing changes in HCV testing, as well as the potential impact of national recommendations supporting HCV testing of baby boomers. The study also highlights a prominent, the increase in HCV antibody testing in 2017 relative to 2011, prior to the recommendation change.

Hepatitis C testing trends among large commercially insured populations, 2011–2017

ObjectiveWe estimated the rate of hepatitis C testing between 2011 and 2017 among persons with commercial health insurance coverage and compared rates by birth cohort.IntroductionHepatitis C virus (HCV) infection is the most common blood-borne infection in the US, and a leading cause of liver-related morbidity and mortality. Approximately 3.5 million individuals in the US were estimated to have been living with hepatitis C in 2010, and approximately half of them were unaware that they were infected. Among HCV infected individuals, those born between 1945 and 1965 (usually referred to as the baby boomer cohort) represent approximately 75% of current cases. Because of the substantial burden of disease among this age group, CDC expanded its existing hepatitis C risk-based testing recommendations to include a one-time HCV antibody test for all persons born between 1945 and 1965. The United States Preventive Services Task Force (USPSTF) subsequently made the same recommendation in June 2013.DescriptionWe obtained data from the 2011–2017 IBM MarketScan® Commercial Claims and Encounters and Medicare Supplemental and Coordination of Benefits databases. These data consist of inpatient and outpatient service claims for persons with employer-sponsored health insurance coverage and their dependents. This analysis was restricted to adults 18 years of age and older with continuous enrollment in a commercial or Medicare Supplemental plan for at least one calendar year during the study period (a 45-day gap in coverage was allowed) who received outpatient prescription drug claims data feeds. Claims for hepatitis C antibody testing were identified using Current Procedural Terminology (CPT) codes (80074, 86803). We defined the annual hepatitis C testing rate as the number of patients with an HCV antibody test claim divided by the total number of study-eligible enrollees in a given calendar year. Testing rates were calculated for persons born between 1945 and 1965 and all other adults.There were 54,298,561 unique adults who were continuously enrolled for at least one calendar year during the study period. Among these, 4,629,040 (9%) had one or more inpatient or outpatient service claim with a CPT code for hepatitis C antibody testing during the study period. The overall estimated annual testing rate increased from 2.2% in 2011 to 5.3% in 2017. The testing rate increased from 1.7% to 7.8% among the 1945–1965 birth cohort and 2.5% to 4.0% in other birth cohorts. The average annual percent change in testing was 30.1% among the 1945–1965 birth cohort and 8.2% among other birth cohorts. Testing rate increased markedly (64.1%) between 2016 and 2017 in the 1945–1965 birth cohort, but not in other birth cohorts (7.7%).In this sample of individuals covered by commercial insurance, hepatitis C testing rates have increased slowly between 2011 and 2016. In 2017, there was a substantial increase in testing rates among the Baby Boomer cohort due most likely to an increase in awareness of CDC and USPSTF recommendations by both providers and individual patients associated with CDC health promotion efforts and increased marketing efforts by drug manufacturers. Efforts should continue to promote and increase the awareness of these recommendations and have people tested and treated for HCV.How the Moderator Intends to Engage the Audience in Discussions on the TopicThis panel will discuss strengths and weaknesses for monitoring hepatitis C testing using alternative data sources including self-reported data, insurance claims data, and laboratory testing data.

Event-based internet biosurveillance: relation to epidemiological observation

Abstract Background The World Health Organization (WHO) collects and publishes surveillance data and statistics for select diseases, but traditional methods of gathering such data are time and labor intensive. Event-based biosurveillance, which utilizes a variety of Internet sources, complements traditional surveillance. In this study we assess the reliability of Internet biosurveillance and evaluate disease-specific alert criteria against epidemiological data. Methods We reviewed and compared WHO epidemiological data and Argus biosurveillance system data for pandemic (H1N1) 2009 (April 2009 – January 2010) from 8 regions and 122 countries to: identify reliable alert criteria among 15 Argus-defined categories; determine the degree of data correlation for disease progression; and assess timeliness of Internet information. Results Argus generated a total of 1,580 unique alerts; 5 alert categories generated statistically significant (p  Conclusion Confirmed pandemic (H1N1) 2009 cases collected by Argus and WHO methods returned consistent results and confirmed the reliability and timeliness of Internet information. Disease-specific alert criteria provide situational awareness and may serve as proxy indicators to event progression and escalation in lieu of traditional surveillance data; alerts may identify early-warning indicators to another pandemic, preparing the public health community for disease events.</p

Trend analysis in hepatitis C testing, OptumLabs® Data Warehouse, 2011–2017

ObjectiveUsing administrative claims for privately insured and Medicare Advantage enrollees from a large, private, U.S. health plan, we estimated the prevalence of hepatitis C testing among individuals who were recommended to be tested (i.e., baby boomer cohort born between 1945 and 1965) by the CDC and United States Preventive Services Task Force. This panel will discuss strengths and weaknesses for monitoring hepatitis C testing using alternative data sources including self-reported data, insurance claims data, and laboratory testing data.IntroductionHepatitis C virus (HCV) infection is the most common blood-borne disease in the US and the leading cause of liver-related morbidity and mortality. Approximately 3.5 million individuals in the US were estimated to be living with HCV in 2010 and approximately half of them were unaware that they were currently infected. Among HCV infected individuals, those born between 1945 and 1965 (usually referred to as the baby boomer cohort) represents approximately 75% of current cases. Because of the substantial burden of disease among this age group, CDC expanded its existing HCV risk-based testing recommendations to include a one-time HCV antibody test for all persons born between 1945-1965. The United States Preventive Services Task Force (USPSTF) subsequently made the same recommendation in June 2013.MethodsWe obtained health plan enrollment information and claims data from the 2011 - 2017 OptumLabs® Data Warehouse, and utilized data from patients enrolled in either commercially insured programs or Medicare Advantage. We examined trends in HCV testing for the birth cohort born between 1945 and 1965 and compared their trend in testing to individuals who were not in the birth cohort. We developed two different estimates for HCV testing incidence in order to make comparisons to other commercial claims datasets. The denominator for both estimates was the number of adults continuously enrolled in one or more health plan(s) in a given calendar year (allowing up to a 45-day gap in coverage). The numerator for the first estimate was the number of people receiving any HCV related test in the current calendar year who had not received any HCV related test including HCV antibody test, HCV RNA test or HCV genotype test in the previous calendar years. The numerator for the second estimate was the number of people who were given an HCV antibody test (CPT: 86803 and 80074) in a given calendar year, irrespective of previous testing history.ResultsDuring the study period 2011 - 2017, there were 20,332,848 unique adults who met the inclusion criteria in the OptumLabs® data. Approximately 7.1 million (35.0%) of these individuals were born between 1945 and 1965. On average, there were approximately 2.8 million birth cohort enrollees for any given calendar year. For the birth cohort, the annual incidence of HCV testing was about 2% per year during the time period between 2008 and 2011 (data not shown). In general, between 2011 and 2017, the trends in testing rates were consistent across both estimation methods. Specifically for the birth cohort, the HCV testing rate increased substantially between 2012 and 2017, peaking in 2017 at 8.56% [95% CI: 8.53-8.59%] and 10.24% [95% CI: 10.21-10.27%]. The greatest increase occurred between 2016 and 2017 when the testing rate almost doubled. In contrast, for the non-birth cohort, the HCV testing rate started in 2012 at a rate similar to the birth cohort but did not increase in a similar fashion and did not see a substantial increase in HCV testing in 2016 or 2017.ConclusionsSince CDC and USPSTF recommended universal testing for the birth cohort in 2012 and 2013, respectively, hepatitis C testing rates have been increasing across all age groups. The rate of increase for the birth cohort was substantially greater than that for the non-birth cohort. CDC and USPSTF recommendations are likely a strong contributing factor impacting hepatitis C testing rates in the US. Efforts to promote hepatitis C testing should continue.

A machine-learning algorithm to identify hepatitis C in health insurance claims data

ObjectiveWe developed a machine learning-based algorithm to identify patients with chronic hepatitis C infection in health insurance claims data.IntroductionHepatitis C virus (HCV) infection is a leading cause of liver disease-related morbidity and mortality in the United States. Monitoring the burden of chronic HCV infection requires robust methods to identify patients with infection. Insurance claims data are a potentially rich source of information about disease burden, but often lack the laboratory results necessary to define chronic HCV infection. We developed a machine learning-based algorithm to identify patients with chronic HCV infection using health insurance claims alone and compared it a previously developed ICD-9 code-based algorithm.MethodsWe obtained insurance claims, demographics, enrollment information, and hepatitis C laboratory results from the IBM MarketScan® Commercial Claims and Encounters databases. We defined chronic HCV infection cases as a patient with one or more positive HCV RNA result and required controls to have a negative HCV antibody result and no positive HCV RNA or antibody results. Patients were required to be continuously enrolled in a health insurance plan during the six months before and after the first positive or negative test result (index date). Outpatient and inpatient insurance claims for the six months before and after the index date were included in the analyses. The study period spanned from 2011 to 2014.Subjects were randomly divided into a training sample (80%) and test (20%) sample. We trained a random forest classifier using age, sex, region, Charlson comorbidity index, and variables defining the presence and frequency of 67 ICD-9 diagnosis codes and CPT procedure codes related to HCV and liver disease. We up-weighted cases to account for the low prevalence of infection in our sample. We generated forests of 1,000 trees for all models. The initial model included all variables. Permutation-based variable importance scores from this initial model were used to select variables for the final model. The previously developed algorithm defined chronic HCV infection as either two claims with codes for chronic hepatitis infection &gt; 60 days apart after an HCV RNA test claim or three claims with codes for chronic HCV infection on different dates after an HCV RNA test claim. We compared the predicted classification to HCV laboratory result-defined classification and calculated percent agreement, Kappa, sensitivity, specificity, positive predictive value, and negative predictive value. We then applied the final classifier to all individuals continuously enrolled in commercial and/or Medicare supplemental insurance to estimate the prevalence of chronic HCV infection in this population in 2014. Analyses were performed in SAS version 9.4.ResultsWe identified 5,780 (5.6%) cases with chronic HCV infection and 97,831 controls with negative HCV test results. The training dataset consisted of 82,888 individuals with approximately six million inpatient and outpatient claims. The final model included 23 variables related to hepatitis C (e.g., number of HCV RNA test claims), liver disease (e.g., cirrhosis diagnosis code), and comorbidities. In the training dataset, percent agreement, Kappa, sensitivity, specificity, positive predictive value, and negative predictive value were 99.2%, 0.92, 92.3%, 99.6%, 93.2%, and 99.5%, respectively. The presence of a CPT code for HCV RNA testing had the highest variable importance score. The test dataset included 20,723 individuals with approximately 1.5 million inpatient and outpatient claims. In the test dataset, percent agreement, Kappa, sensitivity, specificity, positive predictive value, and negative predictive value for the final classifier were 98.9%, 0.89, 89.9%, 99.4%, 89.0%, and 99.4%, respectively. Percent agreement, Kappa, sensitivity, specificity, positive predictive value, and negative predictive value for the previously developed algorithm were 96.3%, 0.50, 35.0%, 99.9%, 96.7%, 96.3%, respectively. Among the 35.6 million individuals with continuous commercial and/or Medicare supplemental insurance in 2014, 317,932 (0.9%) were classified as having chronic HCV infection.ConclusionsOur machine learning-based algorithm was able to identify chronic hepatitis C cases in commercial health insurance claims data with relatively high estimates for percent agreement, Kappa, sensitivity, specificity, positive predictive value, and negative predictive. Future analyses and models will explore the ability of the algorithm to estimate the prevalence of HCV infection in different populations covered by different health plan types (e.g., commercial, Medicaid, Medicare, or no insurance) and for populations where laboratory testing data is not available or collected.