Evaluation of natural language processing from emergency department computerized medical records for intra-hospital syndromic surveillance

<p>Abstract</p> <p>Background</p> <p>The identification of patients who pose an epidemic hazard when they are admitted to a health facility plays a role in preventing the risk of hospital acquired infection. An automated clinical decision support system to detect suspected cases, based on the principle of syndromic surveillance, is being developed at the University of Lyon's Hôpital de la Croix-Rousse. This tool will analyse structured data and narrative reports from computerized emergency department (ED) medical records. The first step consists of developing an application (UrgIndex) which automatically extracts and encodes information found in narrative reports. The purpose of the present article is to describe and evaluate this natural language processing system.</p> <p>Methods</p> <p>Narrative reports have to be pre-processed before utilizing the French-language medical multi-terminology indexer (ECMT) for standardized encoding. UrgIndex identifies and excludes syntagmas containing a negation and replaces non-standard terms (abbreviations, acronyms, spelling errors...). Then, the phrases are sent to the ECMT through an Internet connection. The indexer's reply, based on Extensible Markup Language, returns codes and literals corresponding to the concepts found in phrases. UrgIndex filters codes corresponding to suspected infections. Recall is defined as the number of relevant processed medical concepts divided by the number of concepts evaluated (coded manually by the medical epidemiologist). Precision is defined as the number of relevant processed concepts divided by the number of concepts proposed by UrgIndex. Recall and precision were assessed for respiratory and cutaneous syndromes.</p> <p>Results</p> <p>Evaluation of 1,674 processed medical concepts contained in 100 ED medical records (50 for respiratory syndromes and 50 for cutaneous syndromes) showed an overall recall of 85.8% (95% CI: 84.1-87.3). Recall varied from 84.5% for respiratory syndromes to 87.0% for cutaneous syndromes. The most frequent cause of lack of processing was non-recognition of the term by UrgIndex (9.7%). Overall precision was 79.1% (95% CI: 77.3-80.8). It varied from 81.4% for respiratory syndromes to 77.0% for cutaneous syndromes.</p> <p>Conclusions</p> <p>This study demonstrates the feasibility of and interest in developing an automated method for extracting and encoding medical concepts from ED narrative reports, the first step required for the detection of potentially infectious patients at epidemic risk.</p

An Investigation of the Public Health Informatics Research and Practice in the Past Fifteen Years from 2000 to 2014: A Scoping Review in MEDLINE

Objective: To examine the extent and nature of existing Public Health Informatics (PHI) studies in the past 15 years on MEDLINE. Methods: This thesis adopted the scientific scoping review methodology recommended by Arksey and O’Malley in 2005. It proceeded with the five main stages, which were: Stage I - identifying the research question; Stage II - identifying relevant studies; Stage III - study selection; Stage IV - charting the data; and Stage V - collating, summarizing, and reporting the results. Each methodological stage was carried out with the joint collaboration with the academic supervisor and a final result and conclusion were set forth. Results: The results of this study captured a total number of 486 articles in MEDLINE focused in PHI. Out of them, a majority belonged to the USA followed by the UK, Australia and Canada. Only about one fifth of the articles were from the rest of the world. Further, About 60% of the articles represented infectious disease monitoring, outbreak detection, and bio-terrorism surveillance. Furthermore, about 10% belonged to chronic disease monitoring; whereas public health policy system and research represented 40% of the total articles. The most frequently used information technology were electronic registry, website, and GIS. In contrast, mass media and mobile phones were among the least used technologies. Conclusion: Despite multiple research and discussions conducted in the past 15 years (starting from 2000), the PHI system requires further improvements in the application of modern PHT such as wireless devices, wearable devices, remote sensors, remote/ cloud computing etc. on various domains of PH, which were scarcely discussed or used in the available literature

Real-time classifiers from free-text for continuous surveillance of small animal disease

A wealth of information of epidemiological importance is held within unstructured narrative clinical records. Text mining provides computational techniques for extracting usable information from the language used to communicate between humans, including the spoken and written word. The aim of this work was to develop text-mining methodologies capable of rendering the large volume of information within veterinary clinical narratives accessible for research and surveillance purposes. The free-text records collated within the dataset of the Small Animal Veterinary Surveillance Network formed the development material and target of this work. The efficacy of pre-existent clinician-assigned coding applied to the dataset was evaluated and the nature of notation and vocabulary used in documenting consultations was explored and described. Consultation records were pre-processed to improve human and software readability, and software was developed to redact incidental identifiers present within the free-text. An automated system able to classify for the presence of clinical signs, utilising only information present within the free-text record, was developed with the aim that it would facilitate timely detection of spatio-temporal trends in clinical signs. Clinician-assigned main reason for visit coding provided a poor summary of the large quantity of information exchanged during a veterinary consultation and the nature of the coding and questionnaire triggering further obfuscated information. Delineation of the previously undocumented veterinary clinical sublanguage identified common themes and their manner of documentation, this was key to the development of programmatic methods. A rule-based classifier using logically-chosen dictionaries, sequential processing and data-masking redacted identifiers while maintaining research usability of records. Highly sensitive and specific free-text classification was achieved by applying classifiers for individual clinical signs within a context-sensitive scaffold, this permitted or prohibited matching dependent on the clinical context in which a clinical sign was documented. The mean sensitivity achieved within an unseen test dataset was 98.17 (74.47, 99.9)% and mean specificity 99.94 (77.1, 100.0)%. When used in combination to identify animals with any of a combination of gastrointestinal clinical signs, the sensitivity achieved was 99.44% (95% CI: 98.57, 99.78)% and specificity 99.74 (95% CI: 99.62, 99.83). This work illustrates the importance, utility and promise of free-text classification of clinical records and provides a framework within which this is possible whilst respecting the confidentiality of client and clinician

THE USE OF ELECTRONIC MEDICAL RECORDS BASED ON A PHYSICIAN DIAGNOSIS OF ASTHMA FOR COUNTY WIDE ASTHMA SURVEILLANCE

Allegheny County (AC) has limited information on asthma morbidity. In order to improve upon the sensitivity of asthma, a cross sectional study from January 1, 2002 through December 31, 2005 was conducted to determine whether the data received for emergency room visits from a large regional medical center might be a good predictor for quantifying asthma cases for surveillance. An electronic medical record (EMR) abstract using the Council for State and Territorial Epidemiology (CSTE) Asthma Surveillance case definition of an ICD 9 coded physician diagnosis for primary and secondary asthma (n= 18,284), and primary asthma (n = 5,100) were used to define asthma. The analysis used data from a subset of six hospitals from a large regional medical center covering approximately 60% of adult ED visits in AC that use electronic data for reporting. A secondary analysis of the physician diagnosed primary asthma cases (n= 180) was applied against the CSTE Clinical and Laboratory case definition. Statistical software was used to validate these data abstracted from the EMR. Once these data were validated for accuracy, a fourth dataset of any primary asthma emergency room visits (n= 10,183) were used to test the relationship between asthma morbidity and exposure to ozone. Recent studies have linked asthma hospitalizations in several cities to ozone action days. However, data on the effects of ozone as they relate to asthma emergency room (ER) visits have not been well studied. Electronic medical records from the six hospitals representing the large metropolitan medical center in Allegheny County, PA were obtained on individuals with asthma based on the ICD-9 discharge diagnosis of (493.0-493.9) for the respective time period. Data on ozone, PM2.5, and temperature were obtained for same period. A case crossover methodology using conditional logistic regression as the statistical estimator was conducted to assess the relationship between levels of ozone and PM 2.5 and increases in asthma ER visits. A time stratified sampling strategy was employed assuming a 3:1 case-control ratio.A total of 6,979 individuals were included in the study, with a mean age of 39.25 ±21.0. The mean ozone exposure for this period was 40.6 ppb (range: 0-126). The effect estimates for year-round data was greatest for a 2-day lag adjusted for temperature (OR= 1.02 (95% CI= 1.01-1.04) (p<.05). For each 10-ppb increase in 24-hour maximum ozone, a 2% increase was noted in asthma ER visits. These results indicate that asthma ED visits may be an additional source of information for use in environmental public health tracking

Artificial intelligence-based tools to control healthcare associated infections: A systematic review of the literature

Background: Healthcare-associated infections (HAIs) are the most frequent adverse events in healthcare and a global public health concern. Surveillance is the foundation for effective HAIs prevention and control. Manual surveillance is labor intensive, costly and lacks standardization. Artificial Intelligence (AI) and machine learning (ML) might support the development of HAI surveillance algorithms aimed at understanding HAIs risk factors, improve patient risk stratification, identification of transmission pathways, timely or real-time detection. Scant evidence is available on AI and ML implementation in the field of HAIs and no clear patterns emerges on its impact. Methods: We conducted a systematic review following the PRISMA guidelines to systematically retrieve, quantitatively pool and critically appraise the available evidence on the development, implementation, performance and impact of ML-based HAIs detection models. Results: Of 3445 identified citations, 27 studies were included in the review, the majority published in the US (n = 15, 55.6%) and on surgical site infections (SSI, n = 8, 29.6%). Only 1 randomized controlled trial was included. Within included studies, 17 (63%) ML approaches were classified as predictive and 10 (37%) as retrospective. Most of the studies compared ML algorithms' performance with non-ML logistic regression statistical algorithms, 18.5% compared different ML models' performance, 11.1% assessed ML algorithms' performance in comparison with clinical diagnosis scores, 11.1% with standard or automated surveillance models. Overall, there is moderate evidence that ML-based models perform equal or better as compared to non-ML approaches and that they reach relatively high-performance standards. However, heterogeneity amongst the studies is very high and did not dissipate significantly in subgroup analyses, by type of infection or type of outcome. Discussion: Available evidence mainly focuses on the development and testing of HAIs detection and prediction models, while their adoption and impact for research, healthcare quality improvement, or national surveillance purposes is still far from being explored

Can routinely collected electronic health data be used to develop novel healthcare associated infection surveillance tools?

Background: Healthcare associated infections (HCAI) pose a significant burden to health systems both within the UK and internationally. Surveillance is an essential component to any infection control programme, however traditional surveillance systems are time consuming and costly. Large amounts of electronic routine data are collected within the English NHS, yet these are not currently exploited for HCAI surveillance. Aim: To investigate whether routinely collected electronic hospital data can be exploited for HCAI surveillance within the NHS. Methods: This thesis made use of local linked electronic health data from Imperial College Healthcare NHS Trust, including information on patient admissions, discharges, diagnoses, procedures, laboratory tests, diagnostic imaging requests and traditional infection surveillance data. To establish the evidence base on surveillance and risks of HCAI, two literature reviews were carried out. Based on these, three types of innovative surveillance tools were generated and assessed for their utility and applicability. Results: The key findings were firstly the emerging importance of automated and syndromic surveillance in infection surveillance, but the lack of investigation and application of these tools within the NHS. Syndromic surveillance of surgical site infections was successful in coronary artery bypass graft patients; however it was an inappropriate methodology for caesarean section patients. Automated case detection of healthcare associated urinary tract infections, based on electronic microbiology data, demonstrated similar rates of infection to those recorded during a point prevalence survey. Routine administrative data demonstrated mixed utility in the creation of simplified risk scores or infection, with poorly performing risk models of surgical site infections but reasonable model fit for HCA UTI. Conclusion: Whilst in principle routine administrative data can be used to generate novel surveillance tools for healthcare associated infections; in reality it is not yet practical within the IT infrastructure of the NHS

Assessing and improving the accuracy of surveillance case definitions using administrative data

BACKGROUND Keeping pace with the rapidly evolving demands of infectious disease monitoring requires constant advances in surveillance methodology and infrastructure. A promising new method is syndromic surveillance, where health department staff, assisted by automated data acquisition and statistical alerts, monitor health indicators in near real-time. Several syndromic surveillance systems use diagnoses in administrative databases. However, physician claim diagnoses are not audited, and the effect of diagnostic coding variation on surveillance case definitions is not known. Furthermore, syndromic surveillance systems are limited by high false-positive (FP) rates. Almost no effort has been made to reduce FP rates by improving the positive predictive value (PPV) of surveilled data. OBJECTIVES 1) To evaluate the feasibility of identifying syndrome cases using diagnoses in physician claims. 2) To assess the accuracy of syndrome definitions based on diagnoses in physician claims. 3) To identify physician, patient, encounter and billing characteristics associated with the PPV of syndrome definitions. METHODS &amp; RESULTS STUDY 1: We focused on a subset of diagnoses from a single syndrome (respiratory). We compared cases and non-cases identified from physician claims to medical charts. A convenience sample of 9 Montreal-area family physicians participated. 3,526 visits among 729 patients were abstracted from medical charts and linked to physician claims. The sensitivity and PPV of physician claims for identifying respiratory infections were 0.49, 95%CI (0.45, 0.53) and 0.93, 95%CI (0.91, 0.94). This pilot work demonstrated the feasibility of the proposed method and contributed to planning a full-scale validation of several syndrome definitions. STUDY 2: We focused on 5 syndromes: fever, gastrointestinal, neurological, rash, and respiratory. We selected a random sample of 3,600 physicians practicing in the province of Quebec in 2005-2007, then a stratified random sample of 10 visits per physician from their claims. We obtained chart diagnoses for all sampled visits through double-blinded chart reviews. Sensitivity, specificity, PPV, and negative predictive value (NPV) of syndrome definitions based on diagnoses in physician claims were estimated by comparison to chart review. 1,098 (30.5%) physicians completed the chart review and 10,529 visits were validated. The sensitivity of syndrome definitions ranged from 0.11, 95%CI (0.10, 0.13) for fever to 0.44, 95%CI (0.41, 0.47) for respiratory syndrome. The specificity and NPV were high for all syndromes. The PPV ranged from 0.59, 95%CI (0.55, 0.64) for fever to 0.85, 95%CI (0.83, 0.88) for respiratory syndrome. STUDY 3: We focused on the 4,330 syndrome cases identified from the claims of the 1,098 physicians who participated in study 2. We estimated the association between claim-chart agreement and physician, patient, encounter and billing characteristics using multivariate logistic regression. The likelihood of the medical chart agreeing with the physician claim about the presence of a syndrome was higher when the physician had billed many visits for the same syndrome recently (RR per 10 visits, 1.05; 95%CI, 1.01-1.08), had a lower workload (RR per 10 claims, 0.93; 95%CI, 0.90-0.97), and when the patient was younger (RR per 5 years, 0.96; 95%CI, 0.94-0.97) and less socially deprived (RR most vs least deprived, 0.76; 95%CI, 0.60-0.95). CONCLUSIONS This was the first population-based validation of syndromic surveillance case definitions based on diagnoses in physician claims. We found that the sensitivity of syndrome definitions was low, the PPV was moderate to high, and the specificity and NPV were high. We identified several physician, patient, encounter and billing characteristics associated with the PPV of syndrome definitions, many of which are readily accessible to public health departments and could be used to reduce the FP rate of syndromic surveillance systems.CONTEXTE La surveillance des maladies infectieuses est un défi en constante évolution et un progrès continu au niveau des méthodes et des infrastructures est nécessaire pour répondre à la demande. Une nouvelle approche est la surveillance syndromique, où le personnel de santé publique, assisté de collecte automatisée de données et d'alertes statistiques, surveille des indicateurs de santé en temps quasi-réel. Plusieurs systèmes de surveillance syndromique s'appuient sur les diagnostics issus de bases de données administratives. Parce que ces codes de diagnostics ne font pas l'objet d'audits, l'effet de variations dans leur codage sur les définitions syndromiques demeure inconnu. OBJECTIFS 1) Évaluer la faisabilité d'identifier des syndromes à partir des diagnostics issus des services facturés par les médecins. 2) Évaluer l'exactitude de définitions syndromiques basées sur les diagnostics issus des services facturés par les médecins.3) Identifier les caractéristiques du médecin, du patient, de la rencontre médecin-patient et du mode de facturation associées au coefficient de prédiction positif (CPP) des définitions syndromiques. MÉTHODES &amp; RÉSULTATS ÉTUDE 1: Cette étude a porté sur un seul syndrome (respiratoire). Nous avons comparés les cas positifs et négatifs identifiés à partir de la facturation, aux dossiers médicaux. Un échantillon de 9 médecins généralistes Montréalais a été utilisé. Les diagnostics de 3 526 visites effectuées par 729 patients ont été extraits des dossiers médicaux, et reliés à la facturation. La sensibilité et le CPP des diagnostics d'infection respiratoire issus de la facturation étaient 0.49 et 0.93. Cette étude de faisabilité a permis la planification d'une validation à grande-échelle de plusieurs définitions syndromiques. ÉTUDE 2: Cette étude a porté sur 5 syndromes: fièvre, gastro-intestinal, neurologique, cutané et respiratoire. Nous avons sélectionné aléatoirement 3600 médecins pratiquant au Québec en 2005-2007 et, parmi tous les services facturés, 10 visites par médecin. Pour chaque visite, le diagnostic du dossier médical a été obtenu grâce à une révision de dossier à double insu. La sensibilité, la spécificité, le CPP et le coefficient prédictif négatif (CPN) des définitions syndromiques basées sur les diagnostics issus de la facturation ont été estimés. 1098 (30.5%) médecins ont participé à l'étude et 10529 visites ont été validées. La sensibilité des définitions syndromiques variait de 0.11 pour la fièvre à 0.44 pour le syndrome respiratoire. La spécificité et le CPN étaient élevés pour tous les syndromes. Le CPP variait de 0.59 pour la fièvre à 0.85 pour le syndrome respiratoire. ÉTUDE 3: Nous avons restreint notre échantillon aux 4330 visites des 1098 médecins de l'étude 2 où le diagnostic de la facturation correspondait à l'un des syndromes. Nous avons utilisé une régression logistique multi-variée afin d'estimer l'association entre l'accord facturation-dossier et les caractéristiques du médecin, du patient, de la rencontre médecin-patient et du mode de facturation. La probabilité que le dossier médical confirme un syndrome présent selon la facturation était plus élevée lorsque le médecin avait facturé plusieurs visites pour le même syndrome récemment, avait une charge de travail moindre, et lorsque le patient était plus jeune et moins défavorisé socialement. CONCLUSIONS Cette étude a été la première validation à grande-échelle de définitions syndromiques basées sur les diagnostics issus des services facturés par les médecins. Nous avons découvert que la sensibilité de ces définitions est faible, le CPP varie de moyen à élevé, et la spécificité et le CPN sont élévés. Nous avons identifiés maintes caractéristiques du médecin, du patient, de la rencontre médecin-patient et du mode de facturation associées au CPP des définitions syndromiques, dont plusieurs sont accessibles aux agences de santé publique et pourraient être utilisées pour améliorer les systèmes de surveillance syndromique

The use of emergency department electronic health data for syndromic surveillance to enhance public health surveillance programmes in England

Public health surveillance allows for the identification and monitoring of trends in human health. Syndromic surveillance is a relatively recent addition to these activities, offering the potential to monitor trends on a (near) real-time basis and is often more timely than may be possible through other, traditional, surveillance routes. Emergency department (ED) syndromic surveillance systems have been developed and successfully operated worldwide. The Public Health England Emergency Department Syndromic Surveillance System (EDSSS) was developed in preparation for the London 2012 Olympic and Paralympic Games and remains as a public health legacy of the Games. This thesis aimed to describe and provide evidence of how emergency department syndromic surveillance (as performed by EDSSS) provides additional benefit to public health surveillance and added value to emergency care services in England. Additionally the potential for further development and future improvements to public health surveillance is described. The EDSSS is shown here to have been successfully used to describe the impact of the rotavirus vaccine, indicating that EDSSS has the potential to be used for future rapid, stand alone, investigation of impact of vaccines in England. In the first cross-national study of its kind, the EDSSS (alongside OSCOUR, its counterpart in France) was successfully used to describe the changes in human health indicators during periods of poor air quality. In addition to reporting on both infectious and non-infectious disease, emergency department syndromic surveillance also successfully described the impacts of human behaviour on ED attendances. During the EURO 2016 football tournament ED attendances were found to differ from the expected during match periods, not only in France the host country, but also in the UK home nations where fans followed team progress from home. The EDSSS is also the first example of a syndromic surveillance system having input into the development of a standardised national dataset, which has been mandated across EDs in England. Primarily aimed to improve patient care and the wider workings of EDs, this improved data collection has resulted in improvements in the EDSSS itself, which was subsequently expanded from a small sentinel to truly national surveillance system. The standardisation of ED data collection and reporting, alongside improved geographical coverage and near real-time surveillance reporting, enabled rapid feedback on the impact of the COVID-19 pandemic on ED attendances in England. EDSSS described general trends in ED attendances, encompassing both infectious and non-infectious indicators, prompting the refinement of public health messaging, encouraging continued use of emergency care as required by the general public. The evidence presented in this thesis has demonstrated where the ED syndromic surveillance has added value for public health surveillance in England, utilising the system flexibility and timeliness of reporting. Successful collaborative working has provided the potential for future cross-system learning for further system development, as well as the ability to work at local, national and potentially international scales

Improving the Timeliness and Response to an Aerosolizedanthrax Attack in the Metropolitan Washington, D.C. Region

Our research was conducted to improve the timeliness, coordination, and communication during the detection, investigation and decision-making phases of the response to an aerosolized anthrax attack in the metropolitan Washington, DC, area with the goal of reducing casualties. Our research gathered information of the current response protocols through an extensive literature review and interviews with relevant officials and experts in order to identify potential problems that may exist in various steps of the detection, investigation, and response. Interviewing officials from private and government sector agencies allowed the development of a set of models of interactions and a communication network to identify discrepancies and redundancies that would elongate the delay time in initiating a public health response. In addition, we created a computer simulation designed to model an aerosol spread using weather patterns and population density to identify an estimated population of infected individuals within a target region depending on the virulence and dimensions of the weaponized spores. We developed conceptual models in order to design recommendations that would be presented to our collaborating contacts and agencies that would use such policy and analysis interventions to improve upon the overall response to an aerosolized anthrax attack, primarily through changes to emergency protocol functions and suggestions of technological detection and monitoring response to an aerosolized anthrax attack

Utilizing artificial intelligence in perioperative patient flow:systematic literature review

Abstract. The purpose of this thesis was to map the existing landscape of artificial intelligence (AI) applications used in secondary healthcare, with a focus on perioperative care. The goal was to find out what systems have been developed, and how capable they are at controlling perioperative patient flow. The review was guided by the following research question: How is AI currently utilized in patient flow management in the context of perioperative care? This systematic literature review examined the current evidence regarding the use of AI in perioperative patient flow. A comprehensive search was conducted in four databases, resulting in 33 articles meeting the inclusion criteria. Findings demonstrated that AI technologies, such as machine learning (ML) algorithms and predictive analytics tools, have shown somewhat promising outcomes in optimizing perioperative patient flow. Specifically, AI systems have proven effective in predicting surgical case durations, assessing risks, planning treatments, supporting diagnosis, improving bed utilization, reducing cancellations and delays, and enhancing communication and collaboration among healthcare providers. However, several challenges were identified, including the need for accurate and reliable data sources, ethical considerations, and the potential for biased algorithms. Further research is needed to validate and optimize the application of AI in perioperative patient flow. The contribution of this thesis is summarizing the current state of the characteristics of AI application in perioperative patient flow. This systematic literature review provides information about the features of perioperative patient flow and the clinical tasks of AI applications previously identified