Syndromic Surveillance for Influenzalike Illness in Ambulatory Care Setting

Detection algorithm using proxy data for a bioterrorism agent release and historical data for influenza was effective

Reliability and validity of EMS dispatch code-based categorization of emergency patients for syndromic surveillance.

A retrospective study involving the secondary analysis of public health surveillance records was undertaken to characterize the reliability and validity of an EMS dispatch data-based scheme for assigning emergency patients to surveillance syndromes in relation to two other schemes, one based on hospital ED clinicians\u27 manual categorization according to patients\u27 chief complaint and clinical presentation, and one based on ICD-9 coded hospital ED diagnoses. Comparisons of a sample of individual emergency patients\u27 syndrome assignments according to the EMS versus each of the two hospital categorization schemes were made by matching EMS run records to their corresponding emergency department patient encounter records. This new, linked dataset was analyzed to assess the level of agreement beyond chance between the three possible pairs of syndrome categorization schemes in assigning patients to a respiratory or non-respiratory syndrome and to a gastrointestinal or non-gastrointestinal syndrome. Cohen\u27s kappa statistics were used to measure chance-adjusted agreement between categorization schemes (raters). Z-tests and a chi-square-like test based on the variance of the kappa statistic were used to test the equivalence of kappa coefficients across syndromes, population subgroups and pairs of syndrome assignment schemes. The sensitivity, specificity, predictive value positive and predictive value negative of EMS dispatch and chief complaint-based categorization schemes were also calculated, using the ICD-9-coded ED diagnosis-based categorization scheme as the criterion standard. Comparisons of all performance characteristic (i.e. sensitivity, specificity, predictive value positive and predictive value negative) values were made across categorization schemes and surveillance syndromes to determine whether they were significantly different. The use of EMS dispatch codes for assigning emergency patients to surveillance syndromes was found to have limited but statistically significant reliability in relation to more commonly used syndrome grouping methods based on chief complaints or ICD-9 coded ED diagnoses. The reliability of EMS-based syndrome assignment varied significantly by syndrome, age group and comparison rater. When ICD-9 coded ED diagnosis-based grouping is taken as the criterion standard of syndrome definition, the validity of EMS-based syndrome assignment was limited but comparable to chief complaint-based assignment. The validity of EMS-based syndrome assignment varied significantly by syndrome

Exploratory analysis of methods for automated classification of laboratory test orders into syndromic groups in veterinary medicine

Background: Recent focus on earlier detection of pathogen introduction in human and animal populations has led to the development of surveillance systems based on automated monitoring of health data. Real- or near real-time monitoring of pre-diagnostic data requires automated classification of records into syndromes-syndromic surveillance-using algorithms that incorporate medical knowledge in a reliable and efficient way, while remaining comprehensible to end users. Methods: This paper describes the application of two of machine learning (Naïve Bayes and Decision Trees) and rule-based methods to extract syndromic information from laboratory test requests submitted to a veterinary diagnostic laboratory. Results: High performance (F1-macro = 0.9995) was achieved through the use of a rule-based syndrome classifier, based on rule induction followed by manual modification during the construction phase, which also resulted in clear interpretability of the resulting classification process. An unmodified rule induction algorithm achieved an F1-micro score of 0.979 though this fell to 0.677 when performance for individual classes was averaged in an unweighted manner (F1-macro), due to the fact that the algorithm failed to learn 3 of the 16 classes from the training set. Decision Trees showed equal interpretability to the rule-based approaches, but achieved an F1-micro score of 0.923 (falling to 0.311 when classes are given equal weight). A Naïve Bayes classifier learned all classes and achieved high performance (F1-micro = 0.994 and F1-macro =. 955), however the classification process is not transparent to the domain experts. Conclusion: The use of a manually customised rule set allowed for the development of a system for classification of laboratory tests into syndromic groups with very high performance, and high interpretability by the domain experts. Further research is required to develop internal validation rules in order to establish automated methods to update model rules without user input

MS

thesisThe early detection of infectious disease outbreaks is key to their management and initiation of mitigation strategies. This is true whether the disease is naturally occurring or due to intentional release as an act of terrorism. In recent times, this has become evident with the anthrax bioterrorism attacks of October 2001, the occurrence of emerging infections such as West Nile Virus and Severe Acute Respiratory Syndrome of the concern for a new pandemic of influenza based on H5N1 avian influenza. Public health surveillance efforts at the University of Utah have been place for several years and came to the forefront during the 2002 Winter Olympic Games. At that time, an electronic medical record-based system was developed and deployed to perform daily surveillance of patients visiting the clinics and emergency department of the University of Utah Health Care System. This effort was then followed by a detailed validation of the computer rules used in the surveillance system, with special emphasis on the early detection of central nervous system (CNS) syndromes such as meningitis and encephalitis. These syndromes are of importance to both emerging infections such as West Nile Virus and for NIH/CDC Category B threat agents such as Eastern and Western Equine Encephalitis. True CNS syndromes caused by infectious agents represent a small proportion of patients seen at the emergency department of a large tertiary hospital. "Reason for visit" chief complaint data were poor predictors for the early detection of CNS syndromes. Orders and early results from the laboratory testing of cerebro-spinal fluid were useful for the early detection of meningitis and encephalitis. Overall, computer-based surveillance methods have a role to play in the early detection of infectious diseases. In particular, this project has contributed to public health surveillance by moving the field beyond complaint data and has shown the validity of suing computer-based rules for the detection of meningitis and encephalitis

Assessing and improving the accuracy of surveillance case definitions using administrative data

BACKGROUND Keeping pace with the rapidly evolving demands of infectious disease monitoring requires constant advances in surveillance methodology and infrastructure. A promising new method is syndromic surveillance, where health department staff, assisted by automated data acquisition and statistical alerts, monitor health indicators in near real-time. Several syndromic surveillance systems use diagnoses in administrative databases. However, physician claim diagnoses are not audited, and the effect of diagnostic coding variation on surveillance case definitions is not known. Furthermore, syndromic surveillance systems are limited by high false-positive (FP) rates. Almost no effort has been made to reduce FP rates by improving the positive predictive value (PPV) of surveilled data. OBJECTIVES 1) To evaluate the feasibility of identifying syndrome cases using diagnoses in physician claims. 2) To assess the accuracy of syndrome definitions based on diagnoses in physician claims. 3) To identify physician, patient, encounter and billing characteristics associated with the PPV of syndrome definitions. METHODS & RESULTS STUDY 1: We focused on a subset of diagnoses from a single syndrome (respiratory). We compared cases and non-cases identified from physician claims to medical charts. A convenience sample of 9 Montreal-area family physicians participated. 3,526 visits among 729 patients were abstracted from medical charts and linked to physician claims. The sensitivity and PPV of physician claims for identifying respiratory infections were 0.49, 95%CI (0.45, 0.53) and 0.93, 95%CI (0.91, 0.94). This pilot work demonstrated the feasibility of the proposed method and contributed to planning a full-scale validation of several syndrome definitions. STUDY 2: We focused on 5 syndromes: fever, gastrointestinal, neurological, rash, and respiratory. We selected a random sample of 3,600 physicians practicing in the province of Quebec in 2005-2007, then a stratified random sample of 10 visits per physician from their claims. We obtained chart diagnoses for all sampled visits through double-blinded chart reviews. Sensitivity, specificity, PPV, and negative predictive value (NPV) of syndrome definitions based on diagnoses in physician claims were estimated by comparison to chart review. 1,098 (30.5%) physicians completed the chart review and 10,529 visits were validated. The sensitivity of syndrome definitions ranged from 0.11, 95%CI (0.10, 0.13) for fever to 0.44, 95%CI (0.41, 0.47) for respiratory syndrome. The specificity and NPV were high for all syndromes. The PPV ranged from 0.59, 95%CI (0.55, 0.64) for fever to 0.85, 95%CI (0.83, 0.88) for respiratory syndrome. STUDY 3: We focused on the 4,330 syndrome cases identified from the claims of the 1,098 physicians who participated in study 2. We estimated the association between claim-chart agreement and physician, patient, encounter and billing characteristics using multivariate logistic regression. The likelihood of the medical chart agreeing with the physician claim about the presence of a syndrome was higher when the physician had billed many visits for the same syndrome recently (RR per 10 visits, 1.05; 95%CI, 1.01-1.08), had a lower workload (RR per 10 claims, 0.93; 95%CI, 0.90-0.97), and when the patient was younger (RR per 5 years, 0.96; 95%CI, 0.94-0.97) and less socially deprived (RR most vs least deprived, 0.76; 95%CI, 0.60-0.95). CONCLUSIONS This was the first population-based validation of syndromic surveillance case definitions based on diagnoses in physician claims. We found that the sensitivity of syndrome definitions was low, the PPV was moderate to high, and the specificity and NPV were high. We identified several physician, patient, encounter and billing characteristics associated with the PPV of syndrome definitions, many of which are readily accessible to public health departments and could be used to reduce the FP rate of syndromic surveillance systems.CONTEXTE La surveillance des maladies infectieuses est un défi en constante évolution et un progrès continu au niveau des méthodes et des infrastructures est nécessaire pour répondre à la demande. Une nouvelle approche est la surveillance syndromique, où le personnel de santé publique, assisté de collecte automatisée de données et d'alertes statistiques, surveille des indicateurs de santé en temps quasi-réel. Plusieurs systèmes de surveillance syndromique s'appuient sur les diagnostics issus de bases de données administratives. Parce que ces codes de diagnostics ne font pas l'objet d'audits, l'effet de variations dans leur codage sur les définitions syndromiques demeure inconnu. OBJECTIFS 1) Évaluer la faisabilité d'identifier des syndromes à partir des diagnostics issus des services facturés par les médecins. 2) Évaluer l'exactitude de définitions syndromiques basées sur les diagnostics issus des services facturés par les médecins.3) Identifier les caractéristiques du médecin, du patient, de la rencontre médecin-patient et du mode de facturation associées au coefficient de prédiction positif (CPP) des définitions syndromiques. MÉTHODES & RÉSULTATS ÉTUDE 1: Cette étude a porté sur un seul syndrome (respiratoire). Nous avons comparés les cas positifs et négatifs identifiés à partir de la facturation, aux dossiers médicaux. Un échantillon de 9 médecins généralistes Montréalais a été utilisé. Les diagnostics de 3 526 visites effectuées par 729 patients ont été extraits des dossiers médicaux, et reliés à la facturation. La sensibilité et le CPP des diagnostics d'infection respiratoire issus de la facturation étaient 0.49 et 0.93. Cette étude de faisabilité a permis la planification d'une validation à grande-échelle de plusieurs définitions syndromiques. ÉTUDE 2: Cette étude a porté sur 5 syndromes: fièvre, gastro-intestinal, neurologique, cutané et respiratoire. Nous avons sélectionné aléatoirement 3600 médecins pratiquant au Québec en 2005-2007 et, parmi tous les services facturés, 10 visites par médecin. Pour chaque visite, le diagnostic du dossier médical a été obtenu grâce à une révision de dossier à double insu. La sensibilité, la spécificité, le CPP et le coefficient prédictif négatif (CPN) des définitions syndromiques basées sur les diagnostics issus de la facturation ont été estimés. 1098 (30.5%) médecins ont participé à l'étude et 10529 visites ont été validées. La sensibilité des définitions syndromiques variait de 0.11 pour la fièvre à 0.44 pour le syndrome respiratoire. La spécificité et le CPN étaient élevés pour tous les syndromes. Le CPP variait de 0.59 pour la fièvre à 0.85 pour le syndrome respiratoire. ÉTUDE 3: Nous avons restreint notre échantillon aux 4330 visites des 1098 médecins de l'étude 2 où le diagnostic de la facturation correspondait à l'un des syndromes. Nous avons utilisé une régression logistique multi-variée afin d'estimer l'association entre l'accord facturation-dossier et les caractéristiques du médecin, du patient, de la rencontre médecin-patient et du mode de facturation. La probabilité que le dossier médical confirme un syndrome présent selon la facturation était plus élevée lorsque le médecin avait facturé plusieurs visites pour le même syndrome récemment, avait une charge de travail moindre, et lorsque le patient était plus jeune et moins défavorisé socialement. CONCLUSIONS Cette étude a été la première validation à grande-échelle de définitions syndromiques basées sur les diagnostics issus des services facturés par les médecins. Nous avons découvert que la sensibilité de ces définitions est faible, le CPP varie de moyen à élevé, et la spécificité et le CPN sont élévés. Nous avons identifiés maintes caractéristiques du médecin, du patient, de la rencontre médecin-patient et du mode de facturation associées au CPP des définitions syndromiques, dont plusieurs sont accessibles aux agences de santé publique et pourraient être utilisées pour améliorer les systèmes de surveillance syndromique

SYNDROMIC SURVEILLANCE FOR THE EARLY DETECTION OF INFLUENZA OUTBREAKS

Syndromic surveillance is a new mechanism utilized to detect naturally occurring and bioterroristic outbreaks. The public health significance is its potential to alert public health to outbreaks earlier and allow a timelier public health response. It involves monitoring data that can be collected in near real-time to find anomalous data. Syndromic surveillance includes school and work absenteeism, over-the-counter drug sales, and hospital admissions data to name a few. This study is an assessment of an extension of the use of syndromic surveillance as an improvement to the traditional method to detect more routine public health problems, specifically, the detection of influenza outbreaks. The assessment involves the prediction of outbreaks in four areas during the period October 15, 2003 to March 31, 2004. The four areas studied included Allegheny County, Pennsylvania, Jefferson County, Kentucky, Los Angeles County, California, and Salt Lake County, Utah. Two aspects of community activity were used as the method for syndromic surveillance, over-the-counter pharmaceutical sales and hospital chief complaints. The over-the-counter sales encompassed a panel of six items including anti-diarrheal medication, anti-fever adult medication, anti-fever pediatric medication, cough and cold products, electrolytes, and thermometers. Additionally, two of the seven hospital chief complaints used in the RODS open source paradigm were monitored. These were constitutional and respiratory chief complaints. Application of standard statistical algorithms showed that the system was able to identify unusual activity several weeks prior to the time when the local health departments were able to identify an outbreak using the standard methods. The largest improvement in detection using syndromic surveillance occurred in Los Angeles where the outbreak was detected 52 days before the Centers for Disease Control had declared widespread activity for the state. In each county over-the-counter sales detected the outbreak sooner then hospital chief complaints, but the hospital chief complaints detect the outbreaks consistently across the various algorithms. More conclusive evidence regarding the possible improvement in outbreak detection with syndromic surveillance can be obtained once a longer time frame has passed to allow more historical data to accumulate. Conducting additional studies on influenza outbreaks in other jurisdictions would also be useful assessments

Biomedical informatics and translational medicine

Biomedical informatics involves a core set of methodologies that can provide a foundation for crossing the "translational barriers" associated with translational medicine. To this end, the fundamental aspects of biomedical informatics (e.g., bioinformatics, imaging informatics, clinical informatics, and public health informatics) may be essential in helping improve the ability to bring basic research findings to the bedside, evaluate the efficacy of interventions across communities, and enable the assessment of the eventual impact of translational medicine innovations on health policies. Here, a brief description is provided for a selection of key biomedical informatics topics (Decision Support, Natural Language Processing, Standards, Information Retrieval, and Electronic Health Records) and their relevance to translational medicine. Based on contributions and advancements in each of these topic areas, the article proposes that biomedical informatics practitioners ("biomedical informaticians") can be essential members of translational medicine teams

Syndromic surveillance: reports from a national conference, 2003

Overview of Syndromic Surveillance -- What is Syndromic Surveillance? -- Linking Better Surveillance to Better Outcomes -- Review of the 2003 National Syndromic Surveillance Conference - Lessons Learned and Questions To Be Answered -- -- System Descriptions -- New York City Syndromic Surveillance Systems -- Syndrome and Outbreak Detection Using Chief-Complaint Data - Experience of the Real-Time Outbreak and Disease Surveillance Project -- Removing a Barrier to Computer-Based Outbreak and Disease Surveillance - The RODS Open Source Project -- National Retail Data Monitor for Public Health Surveillance -- National Bioterrorism Syndromic Surveillance Demonstration Program -- Daily Emergency Department Surveillance System - Bergen County, New Jersey -- Hospital Admissions Syndromic Surveillance - Connecticut, September 2001-November 2003 -- BioSense - A National Initiative for Early Detection and Quantification of Public Health Emergencies -- Syndromic Surveillance at Hospital Emergency Departments - Southeastern Virginia -- -- Research Methods -- Bivariate Method for Spatio-Temporal Syndromic Surveillance -- Role of Data Aggregation in Biosurveillance Detection Strategies with Applications from ESSENCE -- Scan Statistics for Temporal Surveillance for Biologic Terrorism -- Approaches to Syndromic Surveillance When Data Consist of Small Regional Counts -- Algorithm for Statistical Detection of Peaks - Syndromic Surveillance System for the Athens 2004 Olympic Games -- Taming Variability in Free Text: Application to Health Surveillance -- Comparison of Two Major Emergency Department-Based Free-Text Chief-Complaint Coding Systems -- How Many Illnesses Does One Emergency Department Visit Represent? Using a Population-Based Telephone Survey To Estimate the Syndromic Multiplier -- Comparison of Office Visit and Nurse Advice Hotline Data for Syndromic Surveillance - Baltimore-Washington, D.C., Metropolitan Area, 2002 -- Progress in Understanding and Using Over-the-Counter Pharmaceuticals for Syndromic Surveillance -- -- Evaluation -- Evaluation Challenges for Syndromic Surveillance - Making Incremental Progress -- Measuring Outbreak-Detection Performance By Using Controlled Feature Set Simulations -- Evaluation of Syndromic Surveillance Systems - Design of an Epidemic Simulation Model -- Benchmark Data and Power Calculations for Evaluating Disease Outbreak Detection Methods -- Bio-ALIRT Biosurveillance Detection Algorithm Evaluation -- ESSENCE II and the Framework for Evaluating Syndromic Surveillance Systems -- Conducting Population Behavioral Health Surveillance by Using Automated Diagnostic and Pharmacy Data Systems -- Evaluation of an Electronic General-Practitioner-Based Syndromic Surveillance System -- National Symptom Surveillance Using Calls to a Telephone Health Advice Service - United Kingdom, December 2001-February 2003 -- Field Investigations of Emergency Department Syndromic Surveillance Signals - New York City -- Should We Be Worried? Investigation of Signals Generated by an Electronic Syndromic Surveillance System - Westchester County, New York -- -- Public Health Practice -- Public Health Information Network - Improving Early Detection by Using a Standards-Based Approach to Connecting Public Health and Clinical Medicine -- Information System Architectures for Syndromic Surveillance -- Perspective of an Emergency Physician Group as a Data Provider for Syndromic Surveillance -- SARS Surveillance Project - Internet-Enabled Multiregion Surveillance for Rapidly Emerging Disease -- Health Information Privacy and Syndromic Surveillance SystemsPapers from the second annual National Syndromic Surveillance Conference convened by the New York City Department of Health and Mental Hygiene, the New York Academy of Medicine, and the CDC in New York City during Oct. 23-24, 2003. Published as the September 24, 2004 supplement to vol. 53 of MMWR. Morbidity and mortality weekly report.1571461

An Investigation of the Public Health Informatics Research and Practice in the Past Fifteen Years from 2000 to 2014: A Scoping Review in MEDLINE

Objective: To examine the extent and nature of existing Public Health Informatics (PHI) studies in the past 15 years on MEDLINE. Methods: This thesis adopted the scientific scoping review methodology recommended by Arksey and O’Malley in 2005. It proceeded with the five main stages, which were: Stage I - identifying the research question; Stage II - identifying relevant studies; Stage III - study selection; Stage IV - charting the data; and Stage V - collating, summarizing, and reporting the results. Each methodological stage was carried out with the joint collaboration with the academic supervisor and a final result and conclusion were set forth. Results: The results of this study captured a total number of 486 articles in MEDLINE focused in PHI. Out of them, a majority belonged to the USA followed by the UK, Australia and Canada. Only about one fifth of the articles were from the rest of the world. Further, About 60% of the articles represented infectious disease monitoring, outbreak detection, and bio-terrorism surveillance. Furthermore, about 10% belonged to chronic disease monitoring; whereas public health policy system and research represented 40% of the total articles. The most frequently used information technology were electronic registry, website, and GIS. In contrast, mass media and mobile phones were among the least used technologies. Conclusion: Despite multiple research and discussions conducted in the past 15 years (starting from 2000), the PHI system requires further improvements in the application of modern PHT such as wireless devices, wearable devices, remote sensors, remote/ cloud computing etc. on various domains of PH, which were scarcely discussed or used in the available literature