Customized Reference Ranges for Laboratory Values Decrease False Positive Alerts in Intensive Care Unit Patients

<div><p>Background</p><p>Traditional electronic medical record (EMR) interfaces mark laboratory tests as abnormal based on standard reference ranges derived from healthy, middle-aged adults. This yields many false positive alerts with subsequent alert-fatigue when applied to complex populations like hospitalized, critically ill patients. Novel EMR interfaces using adjusted reference ranges customized for specific patient populations may ameliorate this problem.</p><p>Objective</p><p>To compare accuracy of abnormal laboratory value indicators in a novel vs traditional EMR interface.</p><p>Methods</p><p>Laboratory data from intensive care unit (ICU) patients consecutively admitted during a two-day period were recorded. For each patient, available laboratory results and the problem list were sent to two mutually blinded critical care experts, who marked the values about which they would like to be alerted. All disagreements were resolved by an independent super-reviewer. Based on this gold standard, we calculated and compared the sensitivity, specificity, positive and negative predictive values (PPV, NPV) of customized vs traditional abnormal value indicators.</p><p>Results</p><p>Thirty seven patients with a total of 1341 laboratory results were included. Experts’ agreement was fair (kappa = 0.39). Compared to the traditional EMR, custom abnormal laboratory value indicators had similar sensitivity (77% vs 85%, P = 0.22) and NPV (97.1% vs 98.6%, P = 0.06) but higher specificity (79% vs 61%, P<0.001) and PPV (28% vs 11%, P<0.001).</p><p>Conclusions</p><p>Reference ranges for laboratory values customized for an ICU population decrease false positive alerts. Disagreement among clinicians about which laboratory values should be indicated as abnormal limits the development of customized reference ranges.</p></div

Normal and abnormal Laboratory Values displayed by both Interfaces subclassified according to Gold Standard Judgment.

<p>Percentage of true positive (TP), false positive (FP), true negative (TN) and false negative (FN) values shown relative to the total number of laboratory values displayed by each interface as percent (number). Truly abnormal laboratory test results (TP) commonly signal health-care providers the need to take action with regards to their patients’ health status. Laboratory values falsely indicated as abnormal (FP) represent in this sense a distraction or “noise” clouding this important “signal”. While an abnormal value in the traditional interface reflects a true abnormality in roughly 1 out of 9 times this “signal-to-noise ratio” is 1 in 4 (i.e. more than twice as high) in the novel interface.</p

Studyflow and Results.

<p>Sensitivity, Specificity, Positive and Negative Predictive Values (PPV, NPV) are given as estimate (95%-Confidence Interval). Only specificity and negative predictive values differed significantly (for details see text).</p

Additional file 2: of Prompting with electronic checklist improves clinician performance in medical emergencies: a high-fidelity simulation study

Example of standardized clinical scenario. (DOCX 99 kb

Additional file 7: Figure S3. of Checklist for Early Recognition and Treatment of Acute Illness (CERTAIN): evolution of a content management system for point-of-care clinical decision support

Overview of customized content management system (for details see Table 2). (PDF 68 kb

Additional file 4: of Prompting with electronic checklist improves clinician performance in medical emergencies: a high-fidelity simulation study

Snapshot of CERTAIN tool. (DOCX 122 kb

Additional file 1: of Prompting with electronic checklist improves clinician performance in medical emergencies: a high-fidelity simulation study

Structure of the study design. (DOCX 46 kb

Data_Sheet_1_Development and usability testing of a patient digital twin for critical care education: a mixed methods study.DOCX

BackgroundDigital twins are computerized patient replicas that allow clinical interventions testing in silico to minimize preventable patient harm. Our group has developed a novel application software utilizing a digital twin patient model based on electronic health record (EHR) variables to simulate clinical trajectories during the initial 6 h of critical illness. This study aimed to assess the usability, workload, and acceptance of the digital twin application as an educational tool in critical care.MethodsA mixed methods study was conducted during seven user testing sessions of the digital twin application with thirty-five first-year internal medicine residents. Qualitative data were collected using a think-aloud and semi-structured interview format, while quantitative measurements included the System Usability Scale (SUS), NASA Task Load Index (NASA-TLX), and a short survey.ResultsMedian SUS scores and NASA-TLX were 70 (IQR 62.5–82.5) and 29.2 (IQR 22.5–34.2), consistent with good software usability and low to moderate workload, respectively. Residents expressed interest in using the digital twin application for ICU rotations and identified five themes for software improvement: clinical fidelity, interface organization, learning experience, serious gaming, and implementation strategies.ConclusionA digital twin application based on EHR clinical variables showed good usability and high acceptance for critical care education.</p

Additional file 2: of Development and validation of clinical performance assessment in simulated medical emergencies: an observational study

Electronic Supplement: Additional tables. (DOCX 22 kb

Additional file 3: of Prompting with electronic checklist improves clinician performance in medical emergencies: a high-fidelity simulation study

Satisfaction survey: a post-CERTAIN survey. (DOCX 33 kb