Enhancing Time-Series Detection Algorithms for Automated Biosurveillance

Algorithm modifications may improve sensitivity for detecting artificially added data

Summary of data reported to CDC's national automated biosurveillance system, 2008

<p>Abstract</p> <p>Background</p> <p>BioSense is the US national automated biosurveillance system. Data regarding chief complaints and diagnoses are automatically pre-processed into 11 broader syndromes (e.g., respiratory) and 78 narrower sub-syndromes (e.g., asthma). The objectives of this report are to present the types of illness and injury that can be studied using these data and the frequency of visits for the syndromes and sub-syndromes in the various data types; this information will facilitate use of the system and comparison with other systems.</p> <p>Methods</p> <p>For each major data source, we summarized information on the facilities, timeliness, patient demographics, and rates of visits for each syndrome and sub-syndrome.</p> <p>Results</p> <p>In 2008, the primary data sources were the 333 US Department of Defense, 770 US Veterans Affairs, and 532 civilian hospital emergency department facilities. Median times from patient visit to record receipt at CDC were 2.2 days, 2.0 days, and 4 hours for these sources respectively. Among sub-syndromes, we summarize mean 2008 visit rates in 45 infectious disease categories, 11 injury categories, 7 chronic disease categories, and 15 other categories.</p> <p>Conclusions</p> <p>We present a systematic summary of data that is automatically available to public health departments for monitoring and responding to emergencies.</p

Data Quality Improvements in National Syndromic Surveillance Program (NSSP) Data

ObjectiveReview the impact of applying regular data quality checks to assess completeness of core data elements that support syndromic surveillance.IntroductionThe National Syndromic Surveillance Program (NSSP) is a community focused collaboration among federal, state, and local public health agencies and partners for timely exchange of syndromic data. These data, captured in nearly real time, are intended to improve the nation's situational awareness and responsiveness to hazardous events and disease outbreaks. During CDC’s previous implementation of a syndromic surveillance system (BioSense 2), there was a reported lack of transparency and sharing of information on the data processing applied to data feeds, encumbering the identification and resolution of data quality issues. The BioSense Governance Group Data Quality Workgroup paved the way to rethink surveillance data flow and quality. Their work and collaboration with state and local partners led to NSSP redesigning the program’s data flow. The new data flow provided a ripe opportunity for NSSP analysts to study the data landscape (e.g., capturing of HL7 messages and core data elements), assess end-to-end data flow, and make adjustments to ensure all data being reported were processed, stored, and made accessible to the user community. In addition, NSSP extensively documented the new data flow, providing the transparency the community needed to better understand the disposition of facility data. Even with a new and improved data flow, data quality issues that were issues in the past, but went unreported, remained issues in the new data. However, these issues were now identified. The newly designed data flow provided opportunities to report and act on issues found in the data unlike previous versions. Therefore, an important component of the NSSP data flow was the implementation of regularly scheduled standard data quality checks, and release of standard data quality reports summarizing data quality findings.MethodsNSSP data was assessed for the national-level completeness of chief complaint and discharge diagnosis data. Completeness is the rate of non- null values (Batini et al., 2009). It was defined as the percent of visits (e.g., emergency department, urgent care center) with a non-null value found among the one or more records associated with the visit. National completeness rates for visits in 2016 were compared with completeness rates of visits in 2017 (a partial year including visits through August 2017). In addition, facility-level progress was quantified after scoring each facility based on the percent completeness change between 2016 and 2017. Legacy data processed prior to introducing the new NSSP data flow were not included in this assessment.ResultsNationally, the percent completeness of chief complaint for visits in 2016 was 82.06% (N=58,192,721), and the percent completeness of chief complaint for visits in 2017 was 87.15% (N=80,603,991). Of the 2,646 facilities that sent visits data in 2016 and 2017, 114 (4.31%) facilities showed an increase of at least 10% in chief complaint completeness in 2017 compared with 2016. As for discharge diagnosis, national results showed the percent completeness of discharge diagnosis for 2016 visits was 50.83% (N=36,048,334), and the percent completeness of discharge diagnosis for 2017 was 59.23% (N=54,776,310). Of the 2,646 facilities that sent data for visits in 2016 and 2017, 306 (11.56%) facilities showed more than a 10% increase in percent completeness of discharge diagnosis in 2017 compared with 2016.ConclusionsNationally, the percent completeness of chief complaint for visits in 2016 was 82.06% (N=58,192,721), and the percent completeness of chief complaint for visits in 2017 was 87.15% (N=80,603,991). Of the 2,646 facilities that sent visits data in 2016 and 2017, 114 (4.31%) facilities showed an increase of at least 10% in chief complaint completeness in 2017 compared with 2016. As for discharge diagnosis, national results showed the percent completeness of discharge diagnosis for 2016 visits was 50.83% (N=36,048,334), and the percent completeness of discharge diagnosis for 2017 was 59.23% (N=54,776,310). Of the 2,646 facilities that sent data for visits in 2016 and 2017, 306 (11.56%) facilities showed more than a 10% increase in percent completeness of discharge diagnosis in 2017 compared with 2016.ReferencesBatini, C., Cappiello. C., Francalanci, C. and Maurino, A. (2009) Methodologies for data quality assessment and improvement. ACM Comput. Surv., 41(3). 1-52

Use of Diagnosis Code in Mental Health Syndrome Definition

Objective: The objectives of this study are to (1) create a mental health syndrome definition for syndromic surveillance to monitor mental health-related ED visits in near real time; (2) examine whether CC data alone can accurately detect mental health related ED visits; and (3) assess the added value of using Dx data to detect mental health-related ED visits.Introduction: Between 2006 and 2013, the rate of emergency department (ED) visits related to mental and substance use disorders increased substantially. This increase was higher for mental disorders visits (55 percent for depression, anxiety or stress reactions and 52 percent for psychoses or bipolar disorders) than for substance use disorders (37 percent) visits [1]. This increasing number of ED visits by patients with mental disorders indicates a growing burden on the health-care delivery system. New methods of surveillance are needed to identify and understand these changing trends in ED utilization and affected underlying populations.Syndromic surveillance can be leveraged to monitor mental health-related ED visits in near real-time. ED syndromic surveillance systems primarily rely on patient chief complaints (CC) to monitor and detect health events. Some studies suggest that the use of ED discharge diagnoses data (Dx), in addition to or instead of CC, may improve sensitivity and specificity of case identification [2].Methods: We extracted a de-identified random sample of 50,000 ED visits with CC from the National Syndromic Surveillance Program (NSSP) for the period January 1—June 30, 2017. NSSP’s BioSense Platform receives ED data from &gt;4000 hospitals, representing about 55 percent of all ED visits in the country [3]. From this sample we extracted 22868 ED visits that included both CC and Dx data. We then applied our mental health syndrome case definition which comprised mental health-related keywords and ICD-9-CM and ICD-10-CM codes. We queried CC text for the words “stress,” “PTSD,” “anxiety,” “depression,” “clinical depression,” “manic depression,” “unipolar depression,” “agitated,” “nervousness,” “mental health,” “mental disorder,” “affective disorder,” “schizoaffective disorder,” “psycoaffective disorder,” “obsessive-compulsive disorder,” “mood disorder,” “bipolar disorder,” “schizotypal personality disorder,” “panic disorder,” “psychosis,” “paranoia,” “psych,” “manic,” “mania,” “hallucinating,” “hallucination,” “mental episode,” and “mental illness.” We queried Dx fields either for ICD-9- CM codes 295-296; 300, 311 or for ICD-10-CM codes F20-F48. The ICD-9- CM and ICD-10-CM codes used to identify mental health-related ED visits are based on the mental health disorders most frequently seen in EDs. Alcohol and substance use, suicide ideation, and suicide attempt were excluded from this study because they are included in alternate syndromes [2]. We manually reviewed the CC text to validate the search terms. Sensitivity, specificity, and positive predictive value will be calculated based on agreement of coding mental health against the human review of mental health visits.Based on our case definition, the sample of 22868 ED visits with CC and Dx data was further stratified into two groups: (1) mental health identified in either CC or Dx, and (2) no mental health identified in CC and Dx. Group 1 was further stratified into three groups: (a) mental health identified only in CC, (b) mental health identified in both CC and Dx, and (c) mental health identified only in Dx. The sample of 27132 ED visits with CC and no Dx data was further stratified into two groups: (1) mental health identified in CC, and (2) no mental health identified in CC (Figure).Results: Of the 50,000 sample of ED visits with CC data, 22868 visits had both CC and Dx data. Of the 22868 visits, we identified 1560 mental health-related ED visits using the mental health syndrome case definition. Of those visits, 241 were identified by a CC only, 226 were identified by both CC and Dx, and 1093 by a mental health-related Dx. Of the 27132 ED visits without Dx data, 421 had mental health identified in CC.Conclusions: Based on our preliminary analysis these findings suggest potential benefits of including Dx data in syndrome binning for mental health. Mental health terms are more likely to be found in Dx data than in the CC (1093 vs. 662). Using CC alone may underestimate the number of mental health-related ED visits. This study had several limitations. Not all facilities reporting to NSSP provide chief complaint data in the same manner, some provide CC as a drop down menu with predefined terms while others include the full text of CC. Not all records contained a Dx code which limited our ability to examine the added value of Dx code for that subset