Investigating Unhealthy Alcohol Use As an Independent Risk Factor for Increased COVID-19 Disease Severity: Observational Cross-sectional Study

BACKGROUND: Unhealthy alcohol use (UAU) is known to disrupt pulmonary immune mechanisms and increase the risk of acute respiratory distress syndrome in patients with pneumonia; however, little is known about the effects of UAU on outcomes in patients with COVID-19 pneumonia. To our knowledge, this is the first observational cross-sectional study that aims to understand the effect of UAU on the severity of COVID-19. OBJECTIVE: We aim to determine if UAU is associated with more severe clinical presentation and worse health outcomes related to COVID-19 and if socioeconomic status, smoking, age, BMI, race/ethnicity, and pattern of alcohol use modify the risk. METHODS: In this observational cross-sectional study that took place between January 1, 2020, and December 31, 2020, we ran a digital machine learning classifier on the electronic health record of patients who tested positive for SARS-CoV-2 via nasopharyngeal swab or had two COVID-19 International Classification of Disease, 10th Revision (ICD-10) codes to identify patients with UAU. After controlling for age, sex, ethnicity, BMI, smoking status, insurance status, and presence of ICD-10 codes for cancer, cardiovascular disease, and diabetes, we then performed a multivariable regression to examine the relationship between UAU and COVID-19 severity as measured by hospital care level (ie, emergency department admission, emergency department admission with ventilator, or death). We used a predefined cutoff with optimal sensitivity and specificity on the digital classifier to compare disease severity in patients with and without UAU. Models were adjusted for age, sex, race/ethnicity, BMI, smoking status, and insurance status. RESULTS: Each incremental increase in the predicted probability from the digital alcohol classifier was associated with a greater odds risk for more severe COVID-19 disease (odds ratio 1.15, 95% CI 1.10-1.20). We found that patients in the unhealthy alcohol group had a greater odds risk to develop more severe disease (odds ratio 1.89, 95% CI 1.17-3.06), suggesting that UAU was associated with an 89% increase in the odds of being in a higher severity category. CONCLUSIONS: In patients infected with SARS-CoV-2, UAU is an independent risk factor associated with greater disease severity and/or death

Bias and fairness assessment of a natural language processing opioid misuse classifier: detection and mitigation of electronic health record data disadvantages across racial subgroups

OBJECTIVES: To assess fairness and bias of a previously validated machine learning opioid misuse classifier. MATERIALS & METHODS: Two experiments were conducted with the classifier's original (n = 1000) and external validation (n = 53 974) datasets from 2 health systems. Bias was assessed via testing for differences in type II error rates across racial/ethnic subgroups (Black, Hispanic/Latinx, White, Other) using bootstrapped 95% confidence intervals. A local surrogate model was estimated to interpret the classifier's predictions by race and averaged globally from the datasets. Subgroup analyses and post-hoc recalibrations were conducted to attempt to mitigate biased metrics. RESULTS: We identified bias in the false negative rate (FNR = 0.32) of the Black subgroup compared to the FNR (0.17) of the White subgroup. Top features included "heroin" and "substance abuse" across subgroups. Post-hoc recalibrations eliminated bias in FNR with minimal changes in other subgroup error metrics. The Black FNR subgroup had higher risk scores for readmission and mortality than the White FNR subgroup, and a higher mortality risk score than the Black true positive subgroup (P < .05). DISCUSSION: The Black FNR subgroup had the greatest severity of disease and risk for poor outcomes. Similar features were present between subgroups for predicting opioid misuse, but inequities were present. Post-hoc mitigation techniques mitigated bias in type II error rate without creating substantial type I error rates. From model design through deployment, bias and data disadvantages should be systematically addressed. CONCLUSION: Standardized, transparent bias assessments are needed to improve trustworthiness in clinical machine learning models

External validation of an opioid misuse machine learning classifier in hospitalized adult patients

BACKGROUND: Opioid misuse screening in hospitals is resource-intensive and rarely done. Many hospitalized patients are never offered opioid treatment. An automated approach leveraging routinely captured electronic health record (EHR) data may be easier for hospitals to institute. We previously derived and internally validated an opioid classifier in a separate hospital setting. The aim is to externally validate our previously published and open-source machine-learning classifier at a different hospital for identifying cases of opioid misuse. METHODS: An observational cohort of 56,227 adult hospitalizations was examined between October 2017 and December 2019 during a hospital-wide substance use screening program with manual screening. Manually completed Drug Abuse Screening Test served as the reference standard to validate a convolutional neural network (CNN) classifier with coded word embedding features from the clinical notes of the EHR. The opioid classifier utilized all notes in the EHR and sensitivity analysis was also performed on the first 24 h of notes. Calibration was performed to account for the lower prevalence than in the original cohort. RESULTS: Manual screening for substance misuse was completed in 67.8% (n = 56,227) with 1.1% (n = 628) identified with opioid misuse. The data for external validation included 2,482,900 notes with 67,969 unique clinical concept features. The opioid classifier had an AUC of 0.99 (95% CI 0.99-0.99) across the encounter and 0.98 (95% CI 0.98-0.99) using only the first 24 h of notes. In the calibrated classifier, the sensitivity and positive predictive value were 0.81 (95% CI 0.77-0.84) and 0.72 (95% CI 0.68-0.75). For the first 24 h, they were 0.75 (95% CI 0.71-0.78) and 0.61 (95% CI 0.57-0.64). CONCLUSIONS: Our opioid misuse classifier had good discrimination during external validation. Our model may provide a comprehensive and automated approach to opioid misuse identification that augments current workflows and overcomes manual screening barriers