Evaluation of Machine-Learning Algorithms for Predicting Opioid Overdose Risk Among Medicare Beneficiaries With Opioid Prescriptions

IMPORTANCE Current approaches to identifying individuals at high risk for opioid overdose target many patients who are not truly at high risk. OBJECTIVE To develop and validate a machine-learning algorithm to predict opioid overdose risk among Medicare beneficiaries with at least 1 opioid prescription. DESIGN, SETTING, AND PARTICIPANTS A prognostic study was conducted between September 1, 2017, and December 31, 2018. Participants (n = 560 057) included fee-for-service Medicare beneficiaries without cancer who filled 1 or more opioid prescriptions from January 1, 2011, to December 31, 2015. Beneficiaries were randomly and equally divided into training, testing, and validation samples. EXPOSURES Potential predictors (n = 268), including sociodemographics, health status, patterns of opioid use, and practitioner-level and regional-level factors, were measured in 3-month windows, starting 3 months before initiating opioids until loss of follow-up or the end of observation. MAIN OUTCOMES AND MEASURES Opioid overdose episodes from inpatient and emergency department claims were identified. Multivariate logistic regression (MLR), least absolute shrinkage and selection operator-type regression (LASSO), random forest (RF), gradient boosting machine (GBM), and deep neural network (DNN) were applied to predict overdose risk in the subsequent 3 months after initiation of treatment with prescription opioids. Prediction performance was assessed using the C statistic and other metrics (eg, sensitivity, specificity, and number needed to evaluate [NNE] to identify one overdose). The Youden index was used to identify the optimized threshold of predicted score that balanced sensitivity and specificity. RESULTS Beneficiaries in the training (n = 186 686), testing (n = 186 685), and validation (n = 186 686) samples had similar characteristics (mean [SD] age of 68.0 [14.5] years, and approximately 63% were female, 82% were white, 35% had disabilities, 41% were dual eligible, and 0.60% had at least 1 overdose episode). In the validation sample, the DNN (C statistic = 0.91; 95% CI, 0.88-0.93) and GBM (C statistic = 0.90; 95% CI, 0.87-0.94) algorithms outperformed the LASSO (C statistic = 0.84; 95% CI, 0.80-0.89), RF (C statistic = 0.80; 95% CI, 0.75-0.84), and MLR (C statistic = 0.75; 95% CI, 0.69-0.80) methods for predicting opioid overdose. At the optimized sensitivity and specificity, DNN had a sensitivity of 92.3%, specificity of 75.7%, NNE of 542, positive predictive value of 0.18%, and negative predictive value of 99.9%. The DNN classified patients into low-risk (76.2%[142 180] of the cohort), medium-risk (18.6%[34 579] of the cohort), and high-risk (5.2%[9747] of the cohort) subgroups, with only 1 in 10 000 in the low-risk subgroup having an overdose episode. More than 90% of overdose episodes occurred in the high-risk and medium-risk subgroups, although positive predictive values were low, given the rare overdose outcome. CONCLUSIONS AND RELEVANCE Machine-learning algorithms appear to perform well for risk prediction and stratification of opioid overdose, especially in identifying low-risk subgroups that have minimal risk of overdose.NIH/National Institute on Drug Abuse [R01DA044985]; Pharmaceutical Research and Manufacturers of America Foundation Research Starter AwardOpen access journalThis item from the UA Faculty Publications collection is made available by the University of Arizona with support from the University of Arizona Libraries. If you have questions, please contact us at [email protected]

Using machine learning to predict risk of incident opioid use disorder among fee-for-service Medicare beneficiaries: A prognostic study

Objective To develop and validate a machine-learning algorithm to improve prediction of incident OUD diagnosis among Medicare beneficiaries with >= 1 opioid prescriptions. Methods This prognostic study included 361,527 fee-for-service Medicare beneficiaries, without cancer, filling >= 1 opioid prescriptions from 2011-2016. We randomly divided beneficiaries into training, testing, and validation samples. We measured 269 potential predictors including socio-demographics, health status, patterns of opioid use, and provider-level and regional-level factors in 3-month periods, starting from three months before initiating opioids until development of OUD, loss of follow-up or end of 2016. The primary outcome was a recorded OUD diagnosis or initiating methadone or buprenorphine for OUD as proxy of incident OUD. We applied elastic net, random forests, gradient boosting machine, and deep neural network to predict OUD in the subsequent three months. We assessed prediction performance using C-statistics and other metrics (e.g., number needed to evaluate to identify an individual with OUD [NNE]). Beneficiaries were stratified into subgroups by risk-score decile. Results The training (n = 120,474), testing (n = 120,556), and validation (n = 120,497) samples had similar characteristics (age >= 65 years = 81.1%; female = 61.3%; white = 83.5%; with disability eligibility = 25.5%; 1.5% had incident OUD). In the validation sample, the four approaches had similar prediction performances (C-statistic ranged from 0.874 to 0.882); elastic net required the fewest predictors (n = 48). Using the elastic net algorithm, individuals in the top decile of risk (15.8% [n = 19,047] of validation cohort) had a positive predictive value of 0.96%, negative predictive value of 99.7%, and NNE of 104. Nearly 70% of individuals with incident OUD were in the top two deciles (n = 37,078), having highest incident OUD (36 to 301 per 10,000 beneficiaries). Individuals in the bottom eight deciles (n = 83,419) had minimal incident OUD (3 to 28 per 10,000). Conclusions Machine-learning algorithms improve risk prediction and risk stratification of incident OUD in Medicare beneficiaries.Open access journalThis item from the UA Faculty Publications collection is made available by the University of Arizona with support from the University of Arizona Libraries. If you have questions, please contact us at [email protected]

Facial Fractures Have Similar Outcomes When Managed by Either Otolaryngology or Plastic Surgery: Encounters From a Single Level I Trauma Center.

Study designRetrospective cohort.ObjectiveTraumatic facial fractures (FFs) often require specialty consultation with Plastic Surgery (PS) or Otolaryngology (ENT); however, referral patterns are often non-standardized and institution specific. Therefore, we sought to compare management patterns and outcomes between PS and ENT, hypothesizing no difference in operative rates, complications, or mortality.MethodsWe performed a retrospective analysis of patients with FFs at a single Level I trauma center from 2014 to 2017. Patients were compared by consulting service: PS vs. ENT. Chi-square and Mann-Whitney-U tests were performed.ResultsOf the 755 patients with FFs, 378 were consulted by PS and 377 by ENT. There was no difference in demographic data (P > 0.05). Patients managed by ENT received a longer mean course of antibiotics (9.4 vs 7.0 days, P = 0.008) and had a lower rate of open reduction internal fixation (ORIF) (9.8% vs. 15.3%, P = 0.017), compared to PS patients. No difference was observed in overall operative rate (15.1% vs. 19.8%), use of computed tomography (CT) imaging (99% vs. 99%), time to surgery (65 vs. 55 hours, P = 0.198), length of stay (LOS) (4 vs. 4 days), 30-day complication rate (10.6% vs. 7.1%), or mortality (4.5% vs. 2.6%) (all P > 0.05).ConclusionOur study demonstrated similar baseline characteristics, operative rates, complications, and mortality between FFs patients who had consultation by ENT and PS. This supports the practice of allowing both ENT and PS to care for trauma FFs patients, as there appears to be similar standardized care and outcomes. Future studies are needed to evaluate the generalizability of our findings

Integrating human services and criminal justice data with claims data to predict risk of opioid overdose among Medicaid beneficiaries: A machine-learning approach.

Health system data incompletely capture the social risk factors for drug overdose. This study aimed to improve the accuracy of a machine-learning algorithm to predict opioid overdose risk by integrating human services and criminal justice data with health claims data to capture the social determinants of overdose risk. This prognostic study included Medicaid beneficiaries (n = 237,259) in Allegheny County, Pennsylvania enrolled between 2015 and 2018, randomly divided into training, testing, and validation samples. We measured 290 potential predictors (239 derived from Medicaid claims data) in 30-day periods, beginning with the first observed Medicaid enrollment date during the study period. Using a gradient boosting machine, we predicted a composite outcome (i.e., fatal or nonfatal opioid overdose constructed using medical examiner and claims data) in the subsequent month. We compared prediction performance between a Medicaid claims only model to one integrating human services and criminal justice data with Medicaid claims (i.e., integrated model) using several metrics (e.g., C-statistic, number needed to evaluate [NNE] to identify one overdose). Beneficiaries were stratified into risk-score decile subgroups. The samples (training = 79,087, testing = 79,086, validation = 79,086) had similar characteristics (age = 38±18 years, female = 56%, white = 48%, having at least one overdose = 1.7% during study period). Using the validation sample, the integrated model slightly improved on the Medicaid claims only model (C-statistic = 0.885; 95%CI = 0.877-0.892 vs. C-statistic = 0.871; 95%CI = 0.863-0.878), with small corresponding improvements in the NNE and positive predictive value. Nine of the top 30 most important predictors in the integrated model were human services and criminal justice variables. Using the integrated model, approximately 70% of individuals with overdoses were members of the top risk decile (overdose rates in the subsequent month = 47/10,000 beneficiaries). Few individuals in the bottom 9 deciles had overdose episodes (0-12/10,000). Machine-learning algorithms integrating claims and social service and criminal justice data modestly improved opioid overdose prediction among Medicaid beneficiaries for a large U.S. county heavily affected by the opioid crisis

Identifying Strategies for Effective Telemedicine Use in Intensive Care Units

Telemedicine, the use of audiovisual technology to provide health care from a remote location, is increasingly used in intensive care units (ICUs). However, studies evaluating the impact of ICU telemedicine show mixed results, with some studies demonstrating improved patient outcomes, while others show limited benefit or even harm. Little is known about the mechanisms that influence variation in ICU telemedicine effectiveness, leaving providers without guidance on how to best use this potentially transformative technology. The Contributors to Effective Critical Care Telemedicine (ConnECCT) study aims to fill this knowledge gap by identifying the clinical and organizational factors associated with variation in ICU telemedicine effectiveness, as well as exploring the clinical contexts and provider perceptions of ICU telemedicine use and its impact on patient outcomes, using a range of qualitative methods. In this report, we describe the study protocol, data collection methods, and planned future analyses of the ConnECCT study. Over the course of 1 year, the study team visited purposefully sampled health systems across the United States that have adopted telemedicine. Data collection methods included direct observations, interviews, focus groups, and artifact collection. Data were collected at the ICUs that provide in-person critical care as well as at the supporting telemedicine units. Iterative thematic content analysis will be used to identify and define key constructs related to telemedicine effectiveness and describe the relationship between them. Ultimately, the study results will provide a framework for more effective implementation of ICU telemedicine, leading to improved clinical outcomes for critically ill patients