Managing patient expectations at emergency department triage

Emergency departments (ED) overcrowding, long wait, and uncomfortable waiting room conditions may lower perceived quality of the patient experience and satisfaction. This study investigates the relationship between patient satisfaction and communication of expected wait times, at the point of triage. A pre-post (11/4/ 2008 – 2/5/2009) group design with convenience sample (n=1,209) of all discharge adult ED patients was utilized for this study. A static expected wait time model (i.e., average wait time + one standard deviation) based on time of the day, day of the week and triage levels was employed to communicating expected wait time at triage while an in-house survey with five-point Likert-scale patient satisfaction questions (satisfied with wait time in triage, informed about delays, and overall rating of ED visit) was administrated at the discharge desk. The communication of delays intervention was significant for only overall rating of ED, while binary communication status was significantly associated with all three patient satisfaction questions. The patients who didn’t receive any communication about delays, were between 1.42 to 5.48 times more likely to rate the three satisfaction questions lower than very good. With communication about delays, the percentage of patients responding very good and very poor/poor were 14.6% higher and 5.9% lower, respectively, for the satisfied with wait time in triage question. Although communication of delays intervention was not significant, the patients who received wait times information were significantly more satisfied. This indicates that patients are more likely to accept longer wait times provided their expectations are managed via communication. Future studies should explore technological solutions for communication of delays and operational improvement initiatives along with alignment of incentives for ED staff to further improve the patient experience. Experience Framework This article is associated with the Culture & Leadership lens of The Beryl Institute Experience Framework. (http://bit.ly/ExperienceFramework) Access other PXJ articles related to this lens. Access other resources related to this len

Utility of Natural Language Processing for Clinical Quality Measures Reporting

ObjectiveTo explain the utility of using an automated syndromic surveillanceprogram with advanced natural language processing (NLP) to improveclinical quality measures reporting for influenza immunization.IntroductionClinical quality measures (CQMs) are tools that help measure andtrack the quality of health care services. Measuring and reportingCQMs helps to ensure that our health care system is deliveringeffective, safe, efficient, patient-centered, equitable, and timely care.The CQM for influenza immunization measures the percentage ofpatients aged 6 months and older seen for a visit between October1 and March 31 who received (or reports previous receipt of) aninfluenza immunization. Centers for Disease Control and Preventionrecommends that everyone 6 months of age and older receive aninfluenza immunization every season, which can reduce influenza-related morbidity and mortality and hospitalizations.MethodsPatients at a large academic medical center who had a visit toan affiliated outpatient clinic during June 1 - 8, 2016 were initiallyidentified using their electronic medical record (EMR). The 2,543patients who were selected did not have documentation of influenzaimmunization in a discrete field of the EMR. All free text notes forthese patients between August 1, 2015 and March 31, 2016 wereretrieved and analyzed using the sophisticated NLP built withinGeographic Utilization of Artificial Intelligence in Real-Timefor Disease Identification and Alert Notification (GUARDIAN)– a syndromic surveillance program – to identify any mention ofinfluenza immunization. The goal was to identify additional cases thatmet the CQM measure for influenza immunization and to distinguishdocumented exceptions. The patients with influenza immunizationmentioned were further categorized by GUARDIAN NLP intoReceived, Recommended, Refused, Allergic, and Unavailable.If more than one category was applicable for a patient, they wereindependently counted in their respective categories. A descriptiveanalysis was conducted, along with manual review of a sample ofcases per each category.ResultsFor the 2,543 patients who did not have influenza immunizationdocumentation in a discrete field of the EMR, a total of 78,642 freetext notes were processed using GUARDIAN. Four hundred fiftythree (17.8%) patients had some mention of influenza immunizationwithin the notes, which could potentially be utilized to meet the CQMinfluenza immunization requirement. Twenty two percent (n=101)of patients mentioned already having received the immunizationwhile 34.7% (n=157) patients refused it during the study time frame.There were 27 patients with the mention of influenza immunization,who could not be differentiated into a specific category. The numberof patients placed into a single category of influenza immunizationwas 351 (77.5%), while 75 (16.6%) were classified into more thanone category. See Table 1.ConclusionsUsing GUARDIAN’s NLP can identify additional patients whomay meet the CQM measure for influenza immunization or whomay be exempt. This tool can be used to improve CQM reportingand improve overall influenza immunization coverage by using it toalert providers. Next steps involve further refinement of influenzaimmunization categories, automating the process of using the NLPto identify and report additional cases, as well as using the NLP forother CQMs.Table 1. Categorization of influenza immunization documentation within freetext notes of 453 patients using NL

Adaptation of GUARDIAN for Syndromic Surveillance During the NATO Summit

OBJECTIVE: To develop and implement a framework for special event surveillance using GUARDIAN, as well as document lessons learned post-event regarding design challenges and usability. INTRODUCTION: Special event driven syndromic surveillance is often initiated by public health departments with limited time for development of an automated surveillance framework, which can result in heavy reliance on frontline care providers and potentially miss early signs of emerging trends. To address timelines and reliability issues, automated surveillance system are required. METHODS: The North Atlantic Treaty Organization (NATO) summit was held in Chicago, IL, May 19–21, 2012. During the NATO summit, the Chicago Department of Public Health (CDPH) was charged with collecting and analyzing syndromic surveillance data from emergency department (ED) visits that may indicate a man-made or naturally occurring infectious disease threat. Ten days prior to the NATO summit surveillance period, Rush University Medical Center (RUMC) received a guidance document from CDPH outlining the syndromes for systematic surveillance, specifically febrile rash illness, localized cutaneous lesion, acute febrile respiratory illness, gastrointestinal illness, botulism-like illness, hemorrhagic illness, along with unexplained deaths or severe illness potentially due to infectious disease and cases due to toxins or suspected poisoning. RUMC researchers collected relevant ICD-9 codes for each syndrome category. GUARDIAN (1), an automated surveillance system, was programmed to scan patient charts and match free text using National Library of Medicine free-text term to unique medical concept, which were further mapped to relevant ICD-9 codes. The baselines were developed using ED patient data from 1/1/2010 to 12/31/2011. Statistical references were established for unsmoothed, 24 hour counts (Baseline = Average; Threshold = +2 standard deviations). During the NATO surveillance timeframe (May 13–26, 2012) automated results with prior reporting period’s counts, reference statistics, and charts were electronically sent to CDPH. In addition, ED charge nurses made manual surveillance reports by telephone at least daily. Open lines of communication were maintained between RUMC and CDPH during the event to discuss potential positive cases. In addition, a post-event debriefing was conducted to document lessons learned. RESULTS: The automated GUARDIAN surveillance reports not only provided timely counts of potentially positive cases for each syndrome but also provided trend analysis with baseline measures. The GUARDIAN User Interface was used to explain what data points could trigger positive cases. The Epic system was used to review patient charts, if further explanation was necessary. The observed counts never exceeded +2 standard deviations during the NATO surveillance period for any of the syndromes. 1. Quick turnaround time (∼ 10 days) from surveillance concept development to automated implementation using GUARDIAN. 2. Surveillance data was timely and reliable. 3. Additional statistical information was beneficial to put trends in context. 4. System may be too sensitive resulting in false alarms and additional investigative burden on public health departments. 5. Need for development of user-interfaces with drill down capabilities to patient level data. 6. Clinicians don’t necessarily utilize exact terminology used in ICD-9 codes which could result in undetected cases. CONCLUSIONS: This exercise successfully highlights rapid development and implementation of special event driven automated surveillance as well as collaborative approach between front-line entities such as emergency departments, surveillance researchers, and the department of public health. In addition, valuable lessons learned with potential solutions are documented for further refinements of such surveillance activities

A Novel Syndrome Definition Validation Approach for Rarely Occurring Diseases

OBJECTIVE: To develop and test a novel syndrome definition validation approach for rarely occurring diseases. INTRODUCTION: Early detection of rarely occurring but potentially harmful diseases such as bio-threat agents (e.g., anthrax), chemical agents (e.g., sarin), and naturally occurring diseases (e.g., meningitis) is critical for rapid initiation of treatment, infection control measures, and emergency response plans. To facilitate clinicians’ ability to detect these diseases, various syndrome definitions have been developed. Due to the rarity of these diseases, standard statistical methodologies for validating syndrome definitions are not applicable. METHODS: Syndrome definitions were developed by researchers for the Geographic Utilization of Artificial Intelligence in Real-Time for Disease Identification and Alert Notification (GUARDIAN) surveillance system (1 1. Partition of literature articles: Literature articles that described positive cases were randomly divided to generate detection (75% of articles) and testing (25% of articles) syndrome definitions. 2. Synthetic case generation: Syndrome definitions and associated statistical measures were reverse engineered using probability of occurrence and inverse Gaussian function to generate potentially infinite positive artificial cases. 3. Clinical filter application: To avoid clinically incompatible combinations of newly generated symptoms, rules based on clinically guided knowledge from emergency department (ED) physicians were applied. Steps 2 and 3 were repeated for both detection and testing syndrome definitions. 4. a. ED negative case sample: Detection syndrome definitions were tested using a random sample of negative ED cases. Knowledge gained through false positive cases was utilized to modify the surveillance algorithms and system thresholds. b. 10-fold cross-validation: Standard 10-fold cross-validation on detection articles of positive cases and ED negative cases was utilized to generate performance metrics. Suspected cases were reviewed by ED clinicians for threshold enhancement. c. Literature articles (n=1): The ability of syndrome definitions to correctly flag literature articles with n=1 case was documented. 5. a. Testing sample: Synthetic positive cases generated from the testing articles along with another set of ED negative cases were evaluated by the respective syndrome definition. Suspected cases were clinically evaluated. b. Literature articles (n=1): Similar to detection step 4c, articles with n=1 were tested using syndrome definitions. c. True positive samples: When available, true positive cases from an ED were identified and sent through the GUARDIAN system. 6. Multi-syndrome validation: A combined sample of positive cases of multiple syndromes and ED negative cases were evaluated for detection of individual syndromes among other similar syndromes. RESULTS: To demonstrate the validation approach, the anthrax syndrome definition was utilized. This syndrome definition was developed with 25 articles containing positive anthrax cases used for detection, and the remaining 11 articles used for testing. With a 10-fold cross validation of the detection phase, the initial results showed accuracy was 99.4% (false positive rate of 0.65% and false negative rate of 0.00%). The testing phase initial validation revealed 99.2% accuracy for the anthrax syndrome definition. CONCLUSIONS: Syndrome specific synthetic samples that are validated through clinical filters allowed the generation of an unlimited number of positive cases. Correct identification by GUARDIAN of these cases indicates robust and reliable syndrome definitions. Utilization of these cases, in conjunction with adherence to a methodological process, was the cornerstone of the GUARDIAN syndrome definition validation approach. The validation approach was successfully demonstrated on anthrax and can be applied to other bio-threat agents, chemical agents, and naturally occurring diseases

A Novel Syndrome Definition Validation Approach for Rarely Occurring Diseases

Syndrome definitions for rarely occuring but potentially harmful diseases were developed by researchers for the Geographic Utilization of Artificial Intelligence in Real-Time for Disease Identification and Alert Notification (GUARDIAN) surveillance system. The main steps for validation of the syndrome definitions are described

Utility of Natural Language Processing for Clinical Quality Measures Reporting

ObjectiveTo explain the utility of using an automated syndromic surveillanceprogram with advanced natural language processing (NLP) to improveclinical quality measures reporting for influenza immunization.IntroductionClinical quality measures (CQMs) are tools that help measure andtrack the quality of health care services. Measuring and reportingCQMs helps to ensure that our health care system is deliveringeffective, safe, efficient, patient-centered, equitable, and timely care.The CQM for influenza immunization measures the percentage ofpatients aged 6 months and older seen for a visit between October1 and March 31 who received (or reports previous receipt of) aninfluenza immunization. Centers for Disease Control and Preventionrecommends that everyone 6 months of age and older receive aninfluenza immunization every season, which can reduce influenza-related morbidity and mortality and hospitalizations.MethodsPatients at a large academic medical center who had a visit toan affiliated outpatient clinic during June 1 - 8, 2016 were initiallyidentified using their electronic medical record (EMR). The 2,543patients who were selected did not have documentation of influenzaimmunization in a discrete field of the EMR. All free text notes forthese patients between August 1, 2015 and March 31, 2016 wereretrieved and analyzed using the sophisticated NLP built withinGeographic Utilization of Artificial Intelligence in Real-Timefor Disease Identification and Alert Notification (GUARDIAN)– a syndromic surveillance program – to identify any mention ofinfluenza immunization. The goal was to identify additional cases thatmet the CQM measure for influenza immunization and to distinguishdocumented exceptions. The patients with influenza immunizationmentioned were further categorized by GUARDIAN NLP intoReceived, Recommended, Refused, Allergic, and Unavailable.If more than one category was applicable for a patient, they wereindependently counted in their respective categories. A descriptiveanalysis was conducted, along with manual review of a sample ofcases per each category.ResultsFor the 2,543 patients who did not have influenza immunizationdocumentation in a discrete field of the EMR, a total of 78,642 freetext notes were processed using GUARDIAN. Four hundred fiftythree (17.8%) patients had some mention of influenza immunizationwithin the notes, which could potentially be utilized to meet the CQMinfluenza immunization requirement. Twenty two percent (n=101)of patients mentioned already having received the immunizationwhile 34.7% (n=157) patients refused it during the study time frame.There were 27 patients with the mention of influenza immunization,who could not be differentiated into a specific category. The numberof patients placed into a single category of influenza immunizationwas 351 (77.5%), while 75 (16.6%) were classified into more thanone category. See Table 1.ConclusionsUsing GUARDIAN’s NLP can identify additional patients whomay meet the CQM measure for influenza immunization or whomay be exempt. This tool can be used to improve CQM reportingand improve overall influenza immunization coverage by using it toalert providers. Next steps involve further refinement of influenzaimmunization categories, automating the process of using the NLPto identify and report additional cases, as well as using the NLP forother CQMs.Table 1. Categorization of influenza immunization documentation within freetext notes of 453 patients using NL

Which Sections of Electronic Medical Records Are Most Relevant for Real-Time Surveillance of Influenza- like Illness?

This was a retrospective cross-sectional study of 100 emergency department positive influenza-like illness (ILI) patients at an academic medical center to investigate which section(s) of a patient's electronic medical record (EMR) contains the most relevant information for timely detection of ILI. The history of present illness and review of systems, followed by the nursing notes sections of the EMR were information rich and the most relevant sections for ILI surveillance for the study site

Adaptation of GUARDIAN for Syndromic Surveillance During the NATO Summit

During the NATO summit, the local health department was charged with collecting and analyzing syndromic surveillance data from emergency department (ED) visits that may indicate a man-made or naturally occurring infectious disease threat. GUARDIAN, an automated surveillance system, was programmed to conduct ED syndromic surveillance during the NATO summit. The automated GUARDIAN surveillance reports not only provided timely counts of potentially positive cases for each syndrome but also provided trend analysis with baseline measures

Natural Language Processing and Technical Challenges of Influenza-Like Illness Surveillance

Processing free-text clinical information in an electronic medical record may enhance surveillance systems for early identification of influenza-like illness outbreaks. However, processing clinical text using natural language processing (NLP) poses a challenge in preserving the semantics of the original information recorded. In this study, we discuss several NLP and technical issues as well as potential solutions for implementation in syndromic surveillance systems

Natural Language Processing and Technical Challenges of Influenza-Like Illness Surveillance

Processing free-text clinical information in an electronic medical record may enhance surveillance systems for early identification of influenza-like illness outbreaks. However, processing clinical text using natural language processing (NLP) poses a challenge in preserving the semantics of the original information recorded. In this study, we discuss several NLP and technical issues as well as potential solutions for implementation in syndromic surveillance systems