Performance of point-of-care severity scores to predict prognosis in patients admitted through the emergency department with COVID-19.

BACKGROUND: Identifying COVID-19 patients at the highest risk of poor outcomes is critical in emergency department (ED) presentation. Sepsis risk stratification scores can be calculated quickly for COVID-19 patients but have not been evaluated in a large cohort. OBJECTIVE: To determine whether well-known risk scores can predict poor outcomes among hospitalized COVID-19 patients. DESIGNS, SETTINGS, AND PARTICIPANTS: A retrospective cohort study of adults presenting with COVID-19 to 156 Hospital Corporation of America (HCA) Healthcare EDs, March 2, 2020, to February 11, 2021. INTERVENTION: Quick Sequential Organ Failure Assessment (qSOFA), Shock Index, National Early Warning System-2 (NEWS2), and quick COVID-19 Severity Index (qCSI) at presentation. MAIN OUTCOME AND MEASURES: The primary outcome was in-hospital mortality. Secondary outcomes included intensive care unit (ICU) admission, mechanical ventilation, and vasopressors receipt. Patients scored positive with qSOFA ≥ 2, Shock Index > 0.7, NEWS2 ≥ 5, and qCSI ≥ 4. Test characteristics and area under the receiver operating characteristics curves (AUROCs) were calculated. RESULTS: We identified 90,376 patients with community-acquired COVID-19 (mean age 64.3 years, 46.8% female). 17.2% of patients died in-hospital, 28.6% went to the ICU, 13.7% received mechanical ventilation, and 13.6% received vasopressors. There were 3.8% qSOFA-positive, 45.1% Shock Index-positive, 49.8% NEWS2-positive, and 37.6% qCSI-positive at ED-triage. NEWS2 exhibited the highest AUROC for in-hospital mortality (0.593, confidence interval [CI]: 0.588-0.597), ICU admission (0.602, CI: 0.599-0.606), mechanical ventilation (0.614, CI: 0.610-0.619), and vasopressor receipt (0.600, CI: 0.595-0.604). CONCLUSIONS: Sepsis severity scores at presentation have low discriminative power to predict outcomes in COVID-19 patients and are not reliable for clinical use. Severity scores should be developed using features that accurately predict poor outcomes among COVID-19 patients to develop more effective risk-based triage

A Systematic Approach to Manage Clinical Deterioration on Inpatient Units in the Health Care System

The transformation of health care delivery in the United States is accelerating at unbelievable speed. The acceleration is a result of many variables including health care reform as well as the covariation occurring with adjustments in regulations related to resident work hours. The evolving care delivery model has exposed a vulnerability of the health system, specifically in academic medical centers of the United States. Academic medical centers have established a care delivery model grounded and predicated in resident presence and performance. With changes in resident work expectations and reduced time spent in hospitals, an urgent need exists to evaluate and recreate a model of care that produces quality outcomes in an efficient, service driven organization. One potential care model that would stabilize organizations is infusion of APNs with the expanded skills and knowledge to instill practice continuity in the critical care environment. A Medicare demonstration project is proposed for funding an APN expanded role and alteration in the care delivery model. Formative and summative evaluation and impact of such an expanded practice role is included in the proposed project. An evolved partnership between the advanced practice nurse and physician will serve to fill some of the gap currently existing in the delivery system of today. As the complexity and acuity of the patients in the hospital escalates, innovation is demanded to ensure a care model that will foster achievement of the quality outcomes expected and deserved

Predicting out of intensive care unit cardiopulmonary arrest or death using electronic medical record data

BACKGROUND: Accurate, timely and automated identification of patients at high risk for severe clinical deterioration using readily available clinical information in the electronic medical record (EMR) could inform health systems to target scarce resources and save lives. METHODS: We identified 7,466 patients admitted to a large, public, urban academic hospital between May 2009 and March 2010. An automated clinical prediction model for out of intensive care unit (ICU) cardiopulmonary arrest and unexpected death was created in the derivation sample (50% randomly selected from total cohort) using multivariable logistic regression. The automated model was then validated in the remaining 50% from the total cohort (validation sample). The primary outcome was a composite of resuscitation events, and death (RED). RED included cardiopulmonary arrest, acute respiratory compromise and unexpected death. Predictors were measured using data from the previous 24 hours. Candidate variables included vital signs, laboratory data, physician orders, medications, floor assignment, and the Modified Early Warning Score (MEWS), among other treatment variables. RESULTS: RED rates were 1.2% of patient-days for the total cohort. Fourteen variables were independent predictors of RED and included age, oxygenation, diastolic blood pressure, arterial blood gas and laboratory values, emergent orders, and assignment to a high risk floor. The automated model had excellent discrimination (c-statistic=0.85) and calibration and was more sensitive (51.6% and 42.2%) and specific (94.3% and 91.3%) than the MEWS alone. The automated model predicted RED 15.9 hours before they occurred and earlier than Rapid Response Team (RRT) activation (5.7 hours prior to an event, p=0.003) CONCLUSION: An automated model harnessing EMR data offers great potential for identifying RED and was superior to both a prior risk model and the human judgment-driven RRT

Digital early warning scores in cardiac care settings: Mixed-methods research

The broad adoption of the National Early Warning Score (NEWS2) was formally endorsed for prediction of early deterioration across all settings. With current digitalisation of the Early Warning Score (EWS) through electronic health records (EHR) and automated patient monitoring, there is an excellent opportunity for facilitating and evaluating NEWS2 implementation. However, no evidence yet shows the success of such standardisation or digitalisation of EWS in cardiac care settings. Individuals with cardiovascular disease (CVD) have a significant risk of developing critical events, and CVD-related morbidity is a critical burden for health and social care. However, there is a gap in research evaluating the performance and implementation of EWS in cardiac settings and the role of digital solutions in the implementation and performance of EWS and clinicians' practice. This PhD aims to provide high-quality evidence on the effectiveness of NEWS2 in predicting worsening events in patients with CVD, the implementation of the digital NEWS2 in two healthcare settings, the experience of escalation of care during the COVID-19 pandemic, and the evaluation of EHR-integrated dashboard for auditing NEWS2 and clinicians' performance

A deep learning model for real-time mortality prediction in critically ill children

BACKGROUND: The rapid development in big data analytics and the data-rich environment of intensive care units together provide unprecedented opportunities for medical breakthroughs in the field of critical care. We developed and validated a machine learning-based model, the Pediatric Risk of Mortality Prediction Tool (PROMPT), for real-time prediction of all-cause mortality in pediatric intensive care units. METHODS: Utilizing two separate retrospective observational cohorts, we conducted model development and validation using a machine learning algorithm with a convolutional neural network. The development cohort comprised 1445 pediatric patients with 1977 medical encounters admitted to intensive care units from January 2011 to December 2017 at Severance Hospital (Seoul, Korea). The validation cohort included 278 patients with 364 medical encounters admitted to the pediatric intensive care unit from January 2016 to November 2017 at Samsung Medical Center. RESULTS: Using seven vital signs, along with patient age and body weight on intensive care unit admission, PROMPT achieved an area under the receiver operating characteristic curve in the range of 0.89-0.97 for mortality prediction 6 to 60 h prior to death. Our results demonstrated that PROMPT provided high sensitivity with specificity and outperformed the conventional severity scoring system, the Pediatric Index of Mortality, in predictive ability. Model performance was indistinguishable between the development and validation cohorts. CONCLUSIONS: PROMPT is a deep model-based, data-driven early warning score tool that can predict mortality in critically ill children and may be useful for the timely identification of deteriorating patients.ope

Deep Risk Prediction and Embedding of Patient Data: Application to Acute Gastrointestinal Bleeding

Acute gastrointestinal bleeding is a common and costly condition, accounting for over 2.2 million hospital days and 19.2 billion dollars of medical charges annually. Risk stratification is a critical part of initial assessment of patients with acute gastrointestinal bleeding. Although all national and international guidelines recommend the use of risk-assessment scoring systems, they are not commonly used in practice, have sub-optimal performance, may be applied incorrectly, and are not easily updated. With the advent of widespread electronic health record adoption, longitudinal clinical data captured during the clinical encounter is now available. However, this data is often noisy, sparse, and heterogeneous. Unsupervised machine learning algorithms may be able to identify structure within electronic health record data while accounting for key issues with the data generation process: measurements missing-not-at-random and information captured in unstructured clinical note text. Deep learning tools can create electronic health record-based models that perform better than clinical risk scores for gastrointestinal bleeding and are well-suited for learning from new data. Furthermore, these models can be used to predict risk trajectories over time, leveraging the longitudinal nature of the electronic health record. The foundation of creating relevant tools is the definition of a relevant outcome measure; in acute gastrointestinal bleeding, a composite outcome of red blood cell transfusion, hemostatic intervention, and all-cause 30-day mortality is a relevant, actionable outcome that reflects the need for hospital-based intervention. However, epidemiological trends may affect the relevance and effectiveness of the outcome measure when applied across multiple settings and patient populations. Understanding the trends in practice, potential areas of disparities, and value proposition for using risk stratification in patients presenting to the Emergency Department with acute gastrointestinal bleeding is important in understanding how to best implement a robust, generalizable risk stratification tool. Key findings include a decrease in the rate of red blood cell transfusion since 2014 and disparities in access to upper endoscopy for patients with upper gastrointestinal bleeding by race/ethnicity across urban and rural hospitals. Projected accumulated savings of consistent implementation of risk stratification tools for upper gastrointestinal bleeding total approximately $1 billion 5 years after implementation. Most current risk scores were designed for use based on the location of the bleeding source: upper or lower gastrointestinal tract. However, the location of the bleeding source is not always clear at presentation. I develop and validate electronic health record based deep learning and machine learning tools for patients presenting with symptoms of acute gastrointestinal bleeding (e.g., hematemesis, melena, hematochezia), which is more relevant and useful in clinical practice. I show that they outperform leading clinical risk scores for upper and lower gastrointestinal bleeding, the Glasgow Blatchford Score and the Oakland score. While the best performing gradient boosted decision tree model has equivalent overall performance to the fully connected feedforward neural network model, at the very low risk threshold of 99% sensitivity the deep learning model identifies more very low risk patients. Using another deep learning model that can model longitudinal risk, the long-short-term memory recurrent neural network, need for transfusion of red blood cells can be predicted at every 4-hour interval in the first 24 hours of intensive care unit stay for high risk patients with acute gastrointestinal bleeding. Finally, for implementation it is important to find patients with symptoms of acute gastrointestinal bleeding in real time and characterize patients by risk using available data in the electronic health record. A decision rule-based electronic health record phenotype has equivalent performance as measured by positive predictive value compared to deep learning and natural language processing-based models, and after live implementation appears to have increased the use of the Acute Gastrointestinal Bleeding Clinical Care pathway. Patients with acute gastrointestinal bleeding but with other groups of disease concepts can be differentiated by directly mapping unstructured clinical text to a common ontology and treating the vector of concepts as signals on a knowledge graph; these patients can be differentiated using unbalanced diffusion earth mover’s distances on the graph. For electronic health record data with data missing not at random, MURAL, an unsupervised random forest-based method, handles data with missing values and generates visualizations that characterize patients with gastrointestinal bleeding. This thesis forms a basis for understanding the potential for machine learning and deep learning tools to characterize risk for patients with acute gastrointestinal bleeding. In the future, these tools may be critical in implementing integrated risk assessment to keep low risk patients out of the hospital and guide resuscitation and timely endoscopic procedures for patients at higher risk for clinical decompensation

Implementation of an Innovative Early Warning System: Evidenced-based Strategies for Ensuring System-wide Nursing Adoption

Early deterioration in adult medical-surgical patients is associated with increased intensive care unit and hospital mortality (Goldhill, 2001). Failure to recognize deterioration is a preventable patient safety and quality issue. To address this problem, since 2013, Kaiser Permanente Northern California (KP NCAL) has piloted Advance Alert Monitor (AAM) at two hospitals. This early warning system employs a set of predictive models developed by the KP NCAL Division of Research, which automatically predicts patient deterioration within the next 12 hours based on a complex algorithm of laboratory and clinical data points. Improvements in mortality and length of stay have been realized at the two pilot hospitals. In anticipation of expansion to additional NCAL facilities, major changes to the AAM workflows and processes were developed that increased the sensitivity of the patients identified at risk for clinical deterioration, as well as the timeliness and clarity of clinical response. Expansion to two additional pilot hospitals using these revised processes rely on the evidence-based implementation strategies found in this Doctor of Nursing Practice project. This paper examines the planning, assessment, and implementation of early warning systems at two NCAL facilities using Rogers’ diffusion of innovation theory and Greenhalgh’s extension of Rogers’ theory. Key attributes need to be considered from a cultural and organizational perspective to both start and sustain an implementation. The success of AAM implementation is validated using specific outcome and process measures, including compliance with documentation and timeliness of workflows

Improved diagnosis and management of sepsis and bloodstream infection

Sepsis is a severe organ dysfunction triggered by infections, and a leading cause of hospitalization and death. Concurrent bloodstream infection (BSI) is common and around one third of sepsis patients have positive blood cultures. Prompt diagnosis and treatment is crucial, but there is a trade-off between the negative effects of over diagnosis and failure to recognize sepsis in time. The emerging crisis of antimicrobial resistance has made bacterial infections more difficult to treat, especially gram-negative pathogens such as Pseudomonas aeruginosa. The overall aim with this thesis was to improve diagnosis, assess the influence of time to antimicrobial treatment and explore prognostic bacterial virulence markers in sepsis and BSI. The papers are based on observational data from 7 cohorts of more than 100 000 hospital episodes. In addition, whole genome sequencing has been performed on approximately 800 invasive P. aeruginosa isolates collected from centers in Europe and Australia. Paper I showed that automated surveillance of sepsis incidence using the Sepsis-3 criteria is feasible in the non-ICU setting, with examples of how implementing this model generates continuous epidemiological data down to the ward level. This information can be used for directing resources and evaluating quality-of-care interventions. In Paper II, evidence is provided for using peripheral oxygen saturation (SpO2) to diagnose respiratory dysfunction in sepsis, proposing the novel thresholds 94% and 90% to get 1 and 2 SOFA points, respectively. This has important implications for improving sepsis diagnosis, especially when conventional arterial blood gas measurements are unavailable. Paper III verified that sepsis surveillance data can be utilized to develop machine learning screening tools to improve early identification of sepsis. A Bayesian network algorithm trained on routine electronic health record data predicted sepsis onset within 48 hours with better discrimination and earlier than conventional NEWS2 outside the ICU. The results suggested that screening may primarily be suited for the early admission period, which have broader implications also for other sepsis screening tools. Paper IV demonstrated that delays in antimicrobial treatment with in vitro pathogen coverage in BSI were associated with increased mortality after 12 hours from blood culture collection, but not at 1, 3, and 6 hours. This indicates a time window where clinicians should focus on the diagnostic workup, and proposes a target for rapid diagnostics of blood cultures. Finally, Paper V showed that the virulence genotype had some influence on mortality and septic shock in P. aeruginosa BSI, however, it was not a major prognostic determinant. Together these studies contribute to better understanding of the sepsis and BSI populations, and provide several suggestions to improve diagnosis and timing of treatment, with implications for clinical practice. Future works should focus on the implementation of sepsis surveillance, clinical trials of time to antimicrobial treatment and evaluating the prognostic importance of bacterial genotype data in larger populations from diverse infection sources and pathogens