Rethinking Human-AI Collaboration in Complex Medical Decision Making: A Case Study in Sepsis Diagnosis

Today's AI systems for medical decision support often succeed on benchmark datasets in research papers but fail in real-world deployment. This work focuses on the decision making of sepsis, an acute life-threatening systematic infection that requires an early diagnosis with high uncertainty from the clinician. Our aim is to explore the design requirements for AI systems that can support clinical experts in making better decisions for the early diagnosis of sepsis. The study begins with a formative study investigating why clinical experts abandon an existing AI-powered Sepsis predictive module in their electrical health record (EHR) system. We argue that a human-centered AI system needs to support human experts in the intermediate stages of a medical decision-making process (e.g., generating hypotheses or gathering data), instead of focusing only on the final decision. Therefore, we build SepsisLab based on a state-of-the-art AI algorithm and extend it to predict the future projection of sepsis development, visualize the prediction uncertainty, and propose actionable suggestions (i.e., which additional laboratory tests can be collected) to reduce such uncertainty. Through heuristic evaluation with six clinicians using our prototype system, we demonstrate that SepsisLab enables a promising human-AI collaboration paradigm for the future of AI-assisted sepsis diagnosis and other high-stakes medical decision making.Comment: Under submission to CHI202

Intravenous Cetirizine Versus Intravenous Diphenhydramine for the Treatment of Acute Urticaria: A Phase III Randomized Controlled Noninferiority Trial

STUDY OBJECTIVE: Acute urticaria is a frequent presentation in emergency departments (EDs), urgent care centers, and other clinical arenas. Treatment options are limited if diphenhydramine is the only intravenous antihistamine offered because of its short duration of action and well-known adverse effects. We evaluate cetirizine injection, the first second-generation injectable antihistamine, for acute urticaria in this multicenter, randomized, noninferiority, phase 3 clinical trial. METHODS: Adult patients presenting to EDs and urgent care centers with acute urticaria requiring an intravenous antihistamine were randomized to either intravenous cetirizine 10 mg or intravenous diphenhydramine 50 mg. The primary endpoint was the 2-hour pruritus score change from baseline, with time spent in treatment center and rate of return to treatment centers as key secondary endpoints. Frequency of sedation and anticholinergic adverse effects were also recorded. RESULTS: Among 262 enrolled patients, the 2-hour pruritus score change from baseline for intravenous cetirizine was statistically noninferior to that for intravenous diphenhydramine (-1.6 versus -1.5; 95% confidence interval -0.1 to 0.3), and in favor of cetirizine. Treatment differences also favored cetirizine for mean time spent in treatment center (1.7 versus 2.1 hours; P=.005), return to treatment center (5.5% versus 14.1%; P=.02), lower change from baseline sedation score at 2 hours (0.1 versus 0.5; P=.03), and adverse event rate (3.9% versus 13.3%). CONCLUSION: Intravenous cetirizine is an effective alternative to intravenous diphenhydramine for treating acute urticaria, with benefits of less sedation, fewer adverse events, shorter time spent in treatment center, and lower rates of revisit to treatment center

Definitions of Urinary Tract Infection in Current Research: A Systematic Review

Defining urinary tract infection (UTI) is complex, as numerous clinical and diagnostic parameters are involved. In this systematic review, we aimed to gain insight into how UTI is defined across current studies. We included 47 studies, published between January 2019 and May 2022, investigating therapeutic or prophylactic interventions in adult patients with UTI. Signs and symptoms, pyuria, and a positive urine culture were required in 85%, 28%, and 55% of study definitions, respectively. Five studies (11%) required all 3 categories for the diagnosis of UTI. Thresholds for significant bacteriuria varied from 103 to 105 colony-forming units/mL. None of the 12 studies including acute cystitis and 2 of 12 (17%) defining acute pyelonephritis used identical definitions. Complicated UTI was defined by both host factors and systemic involvement in 9 of 14 (64%) studies. In conclusion, UTI definitions are heterogeneous across recent studies, highlighting the need for a consensus-based, research reference standard for UTI

Early High-Dose Vitamin D3 for Critically Ill, Vitamin D-Deficient Patients

BACKGROUND: Vitamin D deficiency is a common, potentially reversible contributor to morbidity and mortality among critically ill patients. The potential benefits of vitamin D supplementation in acute critical illness require further study. METHODS: We conducted a randomized, double-blind, placebo-controlled, phase 3 trial of early vitamin D3 supplementation in critically ill, vitamin D-deficient patients who were at high risk for death. Randomization occurred within 12 hours after the decision to admit the patient to an intensive care unit. Eligible patients received a single enteral dose of 540,000 IU of vitamin D3 or matched placebo. The primary end point was 90-day all-cause, all-location mortality. RESULTS: A total of 1360 patients were found to be vitamin D-deficient during point-of-care screening and underwent randomization. Of these patients, 1078 had baseline vitamin D deficiency (25-hydroxyvitamin D level,[50 nmol per liter]) confirmed by subsequent testing and were included in the primary analysis population. The mean day 3 level of 25-hydroxyvitamin D was 46.9±23.2 ng per milliliter (117±58 nmol per liter) in the vitamin D group and 11.4±5.6 ng per milliliter (28±14 nmol per liter) in the placebo group (difference, 35.5 ng per milliliter; 95% confidence interval [CI], 31.5 to 39.6). The 90-day mortality was 23.5% in the vitamin D group (125 of 531 patients) and 20.6% in the placebo group (109 of 528 patients) (difference, 2.9 percentage points; 95% CI, -2.1 to 7.9; P = 0.26). There were no clinically important differences between the groups with respect to secondary clinical, physiological, or safety end points. The severity of vitamin D deficiency at baseline did not affect the association between the treatment assignment and mortality. CONCLUSIONS: Early administration of high-dose enteral vitamin D3 did not provide an advantage over placebo with respect to 90-day mortality or other, nonfatal outcomes among critically ill, vitamin D-deficient patients. METHODS: We conducted a randomized, double-blind, placebo-controlled, phase 3 trial of early vitamin D3 supplementation in critically ill, vitamin D-deficient patients who were at high risk for death. Randomization occurred within 12 hours after the decision to admit the patient to an intensive care unit. Eligible patients received a single enteral dose of 540,000 IU of vitamin D3 or matched placebo. The primary end point was 90-day all-cause, all-location mortality

Quality of care in elder emergency department patients with pneumonia: a prospective cohort study

<p>Abstract</p> <p>Background</p> <p>The goals of the study were to assess the relationship between age and processes of care in emergency department (ED) patients admitted with pneumonia and to identify independent predictors of failure to meet recommended quality care measures.</p> <p>Methods</p> <p>This was a prospective cohort study of a pre-existing database undertaken at a university hospital ED in the Midwest. ED patients ≥18 years of age requiring admission for pneumonia, with no documented use of antibiotics in the 24 hours prior to ED presentation were included. Compliance with Pneumonia National Quality Measures was assessed including ED antibiotic administration, antibiotics within 4 hours, oxygenation assessment, and obtaining of blood cultures. Odds ratios were calculated for elders and non-elders. Logistic regression was used to identify independent predictors of process failure.</p> <p>Results</p> <p>One thousand, three hundred seventy patients met inclusion criteria, of which 560 were aged ≥65 years. In multiple variable logistic regression analysis, age ≥65 years was independently associated with receiving antibiotics in the ED (odds ratio [OR] = 2.03, 95% CI 1.28–3.21) and assessment of oxygenation (OR = 2.10, 95% CI, 1.18–3.32). Age had no significant impact on odds of receiving antibiotics within four hours of presentation (OR 1.10, 95% CI 0.84–1.43) or having blood cultures drawn (OR 1.02, 95%CI 0.78–1.32). Certain other patient characteristics were also independently associated with process failure.</p> <p>Conclusion</p> <p>Elderly patients admitted from the ED with pneumonia are more likely to receive antibiotics while in the ED and to have oxygenation assessed in the ED than younger patients. The independent association of certain patient characteristics with process failure provides an opportunity to further increase compliance with recommended quality measures in admitted patients diagnosed with pneumonia.</p

A prospective registry of emergency department patients admitted with infection

<p>Abstract</p> <p>Background</p> <p>Patients with infections account for a significant proportion of Emergency Department (ED) workload, with many hospital patients admitted with severe sepsis initially investigated and resuscitated in the ED. The aim of this registry is to systematically collect quality observational clinical and microbiological data regarding emergency patients admitted with infection, in order to explore in detail the microbiological profile of these patients, and to provide the foundation for a significant programme of prospective observational studies and further clinical research.</p> <p>Methods/design</p> <p>ED patients admitted with infection will be identified through daily review of the computerised database of ED admissions, and clinical information such as site of infection, physiological status in the ED, and components of management abstracted from patients' charts. This information will be supplemented by further data regarding results of investigations, microbiological isolates, and length of stay (LOS) from hospital electronic databases. Outcome measures will be hospital and intensive care unit (ICU) LOS, and mortality endpoints derived from a national death registry.</p> <p>Discussion</p> <p>This database will provide substantial insights into the characteristics, microbiological profile, and outcomes of emergency patients admitted with infections. It will become the nidus for a programme of research into compliance with evidence-based guidelines, optimisation of empiric antimicrobial regimens, validation of clinical decision rules and identification of outcome determinants. The detailed observational data obtained will provide a solid baseline to inform the design of further controlled trials planned to optimise treatment and outcomes for emergency patients admitted with infections.</p

Integrative omics to detect bacteremia in patients with febrile neutropenia

Background: Cancer chemotherapy-associated febrile neutropenia (FN) is a common condition that is deadly when bacteremia is present. Detection of bacteremia depends on culture, which takes days, and no accurate predictive tools applicable to the initial evaluation are available. We utilized metabolomics and transcriptomics to develop multivariable predictors of bacteremia among FN patients. Methods: We classified emergency department patients with FN and no apparent infection at presentation as bacteremic (cases) or not (controls), according to blood culture results. We assessed relative metabolite abundance in plasma, and relative expression of 2,560 immunology and cancer-related genes in whole blood. We used logistic regression to identify multivariable predictors of bacteremia, and report test characteristics of the derived predictors. Results: For metabolomics, 14 bacteremic cases and 25 non-bacteremic controls were available for analysis; for transcriptomics we had 7 and 22 respectively. A 5-predictor metabolomic model had an area under the receiver operating characteristic curve of 0.991 (95%CI: 0.972,1.000), 100% sensitivity, and 96% specificity for identifying bacteremia. Pregnenolone steroids were more abundant in cases and carnitine metabolites were more abundant in controls. A 3-predictor gene expression model had corresponding results of 0.961 (95%CI: 0.896,1.000), 100%, and 86%. Genes involved in innate immunity were differentially expressed. Conclusions: Classifiers derived from metabolomic and gene expression data hold promise as objective and accurate predictors of bacteremia among FN patients without apparent infection at presentation, and can provide insights into the underlying biology. Our findings should be considered illustrative, but may lay the groundwork for future biomarker development

Definitions of Urinary Tract Infection in Current Research: A Systematic Review

Defining urinary tract infection (UTI) is complex, as numerous clinical and diagnostic parameters are involved. In this systematic review, we aimed to gain insight into how UTI is defined across current studies. We included 47 studies, published between January 2019 and May 2022, investigating therapeutic or prophylactic interventions in adult patients with UTI. Signs and symptoms, pyuria, and a positive urine culture were required in 85%, 28%, and 55% of study definitions, respectively. Five studies (11%) required all 3 categories for the diagnosis of UTI. Thresholds for significant bacteriuria varied from 10 3 to 10 5 colony-forming units/mL. None of the 12 studies including acute cystitis and 2 of 12 (17%) defining acute pyelonephritis used identical definitions. Complicated UTI was defined by both host factors and systemic involvement in 9 of 14 (64%) studies. In conclusion, UTI definitions are heterogeneous across recent studies, highlighting the need for a consensus-based, research reference standard for UTI

Medical Student Professionalism Narratives: A Thematic Analysis and Interdisciplinary Comparative Investigation

<p>Abstract</p> <p>Background</p> <p>Professionalism development is influenced by the informal and hidden curriculum. The primary objective of this study was to better understand this experiential learning in the setting of the Emergency Department (ED). Secondarily, the study aimed to explore differences in the informal curriculum between Emergency Medicine (EM) and Internal Medicine (IM) clerkships.</p> <p>Methods</p> <p>A thematic analysis was conducted on 377 professionalism narratives from medical students completing a required EM clerkship from July 2008 through May 2010. The narratives were analyzed using established thematic categories from prior research as well as basic descriptive characteristics. Chi-square analysis was used to compare the frequency of thematic categories to prior research in IM. Finally, emerging themes not fully appreciated in the established thematic categories were created using grounded theory.</p> <p>Results</p> <p>Observations involving interactions between attending physician and patient were most abundant. The narratives were coded as positive 198 times, negative 128 times, and hybrid 37 times. The two most abundant narrative themes involved <it>manifesting respect </it>(36.9%) and <it>spending time </it>(23.7%). Both of these themes were statistically more likely to be noted by students on EM clerkships compared to IM clerkships. Finally, one new theme regarding <it>cynicism </it>emerged during analysis.</p> <p>Conclusions</p> <p>This analysis describes an informal curriculum that is diverse in themes. Student narratives suggest their clinical experiences to be influential on professionalism development. Medical students focus on different aspects of professionalism depending on clerkship specialty.</p

Comparison of balance assessment modalities in emergency department elders: a pilot cross-sectional observational study

<p>Abstract</p> <p>Background</p> <p>More than one-third of US adults 65 and over fall every year. These falls may cause serious injury including substantial long-term morbidity (due declines in activities of daily living) and death. The emergency department (ED) visit represents an opportunity for identifying high risk elders and potentially instituting falls-related interventions. The unique characteristic of the ED environment and patient population necessitate that risk-assessment modalities be validated in this specific setting. In order to better identify elders at risk of falls, we examined the relationship between patient-provided history of falling and two testing modalities (a balance plate system and the timed up-and-go [TUG] test) in elder emergency department (ED) patients.</p> <p>Methods</p> <p>We conducted a cross-sectional observational study of patients ≥ 60 years old being discharged from the ED. Patient history of falls in the past week, month, 6 months, and year was obtained. Balance plate center of pressure excursion (COP) measurements and TUG testing times were recorded. COP was recorded under four conditions: normal stability eyes open (NSEO) and closed (NSEC), and perturbed stability eyes open and closed. Correlation between TUG and COP scores was measured. Univariate logistic regression was used to identify the relationship between patient-provided falls history and the two testing modalities. Proportions, likelihood ratios, and receiver-operating-characteristic (ROC) curves for prediction of previous falls were reported.</p> <p>Results</p> <p>Fifty-three subjects were enrolled, 11% had fallen in the previous week and 42% in the previous year. There was no correlation between TUG and any balance plate measurements. In logistic regression, neither testing modality was associated with prior history of falls (<it>p </it>> 0.05 for all time periods). Balance plate NSEO and NSEC testing cutoffs could be identified which were 83% sensitive and had a negative likelihood ratio (LR-) of 0.3 for falls in the past week. TUG testing was not useful for falls in the past week, but performed best for more distant falls in the past month, 6 months, or year. TUG cutoffs with sensitivity over 80% and LR(-) of 0.17-0.32 could be identified for these time periods.</p> <p>Conclusion</p> <p>Over 40% of community-dwelling elder ED patients report a fall within the past year. Balance plate and TUG testing were feasibly conducted in an ED setting. There is no relationship between scores on balance plate and TUG testing in these patients. In regression analysis, neither modality was significantly associated with patient provided history of falls. These modalities should not be adopted for screening purposes in elders in the ED setting without validation in future studies or as part of multi-factorial risk assessment.</p