ODD: A Benchmark Dataset for the NLP-based Opioid Related Aberrant Behavior Detection

Opioid related aberrant behaviors (ORAB) present novel risk factors for opioid overdose. Previously, ORAB have been mainly assessed by survey results and by monitoring drug administrations. Such methods however, cannot scale up and do not cover the entire spectrum of aberrant behaviors. On the other hand, ORAB are widely documented in electronic health record notes. This paper introduces a novel biomedical natural language processing benchmark dataset named ODD, for ORAB Detection Dataset. ODD is an expert-annotated dataset comprising of more than 750 publicly available EHR notes. ODD has been designed to identify ORAB from patients' EHR notes and classify them into nine categories; 1) Confirmed Aberrant Behavior, 2) Suggested Aberrant Behavior, 3) Opioids, 4) Indication, 5) Diagnosed opioid dependency, 6) Benzodiapines, 7) Medication Changes, 8) Central Nervous System-related, and 9) Social Determinants of Health. We explored two state-of-the-art natural language processing (NLP) models (finetuning pretrained language models and prompt-tuning approaches) to identify ORAB. Experimental results show that the prompt-tuning models outperformed the finetuning models in most cateogories and the gains were especially higher among uncommon categories (Suggested aberrant behavior, Diagnosed opioid dependency and Medication change). Although the best model achieved the highest 83.92% on area under precision recall curve, uncommon classes (Suggested Aberrant Behavior, Diagnosed Opioid Dependence, and Medication Change) still have a large room for performance improvement.Comment: Under revie

Associations Between Natural Language Processing (NLP) Enriched Social Determinants of Health and Suicide Death among US Veterans

Importance: Social determinants of health (SDOH) are known to be associated with increased risk of suicidal behaviors, but few studies utilized SDOH from unstructured electronic health record (EHR) notes. Objective: To investigate associations between suicide and recent SDOH, identified using structured and unstructured data. Design: Nested case-control study. Setting: EHR data from the US Veterans Health Administration (VHA). Participants: 6,122,785 Veterans who received care in the US VHA between October 1, 2010, and September 30, 2015. Exposures: Occurrence of SDOH over a maximum span of two years compared with no occurrence of SDOH. Main Outcomes and Measures: Cases of suicide deaths were matched with 4 controls on birth year, cohort entry date, sex, and duration of follow-up. We developed an NLP system to extract SDOH from unstructured notes. Structured data, NLP on unstructured data, and combining them yielded six, eight and nine SDOH respectively. Adjusted odds ratios (aORs) and 95% confidence intervals (CIs) were estimated using conditional logistic regression. Results: In our cohort, 8,821 Veterans committed suicide during 23,725,382 person-years of follow-up (incidence rate 37.18/100,000 person-years). Our cohort was mostly male (92.23%) and white (76.99%). Across the five common SDOH as covariates, NLP-extracted SDOH, on average, covered 80.03% of all SDOH occurrences. All SDOH, measured by structured data and NLP, were significantly associated with increased risk of suicide. The SDOH with the largest effects was legal problems (aOR=2.66, 95% CI=.46-2.89), followed by violence (aOR=2.12, 95% CI=1.98-2.27). NLP-extracted and structured SDOH were also associated with suicide. Conclusions and Relevance: NLP-extracted SDOH were always significantly associated with increased risk of suicide among Veterans, suggesting the potential of NLP in public health studies.Comment: Submitted to JAMA Network Ope

Randomized Controlled Trial of Fish Oil and Montelukast and Their Combination on Airway Inflammation and Hyperpnea-Induced Bronchoconstriction

Both fish oil and montelukast have been shown to reduce the severity of exercise-induced bronchoconstriction (EIB). The purpose of this study was to compare the effects of fish oil and montelukast, alone and in combination, on airway inflammation and bronchoconstriction induced by eucapnic voluntary hyperpnea (EVH) in asthmatics. In this model of EIB, twenty asthmatic subjects with documented hyperpnea-induced bronchoconstriction (HIB) entered a randomized double-blind trial. All subjects entered on their usual diet (pre-treatment, n = 20) and then were randomly assigned to receive either one active 10 mg montelukast tablet and 10 placebo fish oil capsules (n = 10) or one placebo montelukast tablet and 10 active fish oil capsules totaling 3.2 g EPA and 2.0 g DHA (n = 10) taken daily for 3-wk. Thereafter, all subjects (combination treatment; n = 20) underwent another 3-wk treatment period consisting of a 10 mg active montelukast tablet or 10 active fish oil capsules taken daily. While HIB was significantly inhibited (p0.017) between treatment groups; percent fall in forced expiratory volume in 1-sec was −18.4±2.1%, −9.3±2.8%, −11.6±2.8% and −10.8±1.7% on usual diet (pre-treatment), fish oil, montelukast and combination treatment respectively. All three treatments were associated with a significant reduction (p0.017) in these biomarkers between treatments. While fish oil and montelukast are both effective in attenuating airway inflammation and HIB, combining fish oil with montelukast did not confer a greater protective effect than either intervention alone. Fish oil supplementation should be considered as an alternative treatment for EIB

Impact of electronic consultation (e-consult) on timeliness and guideline concordance of workups leading to thyroid nodule fine-needle aspiration biopsy

OBJECTIVE: Electronic consultations (e-consults) are commonly used to obtain endocrinology input on clinical questions without a face-to-face visit, but there are sparse data on quality of care resulting from e-consults for specific conditions. We examined workups resulting in thyroid nodule fine-needle aspiration (FNA) biopsy to examine whether endocrinology e-consults were more timely and similarly guideline concordant as compared to endocrinology face-to-face visits, and whether endocrinology e-consults were more guideline concordant as compared to workups without endocrinology input. METHODS: We conducted a retrospective chart review of 302 thyroid FNA biopsies conducted in the Veterans Affairs (VA) health system between 5/1/2017 and 2/4/2020 (e-consult, N=99; face-to-face visit, N=100; no endocrinology input, N=103). We used t-tests to compare timeliness and chi-square tests to compare the proportion of guideline-concordant workups. We used multivariable linear and logistic models to control for demographic factors. RESULTS: FNAs preceded by endocrinology e-consult had more timely workups compared to those preceded by endocrinology face-to-face visits in terms of days elapsed between referral and FNA biopsy [geometric mean [95% confidence interval]: 44.7 [37.2, 53.7] days vs. 61.7 [52.5, 72.4] days, t(195)=2.55, P=0.01]. The difference in the summary measure of guideline concordance across groups was not statistically significant (P=0.38). CONCLUSION: E-consults were faster than face-to-face consults and similar guideline concordant as compared to both face-to-face consults and no endocrinology input for workups resulting in FNA. Decisions about appropriate use of e-consults for thyroid nodules should account for these data while also considering potential benefits of direct patient-endocrinologist interaction for complex situations

Patient characteristics associated with oral anticoagulation control: results of the Veterans AffaiRs Study to Improve Anticoagulation (VARIA)

BACKGROUND: In patients receiving oral anticoagulation, improved control can reduce adverse outcomes such as stroke and major hemorrhage. However, little is known about patient-level predictors of anticoagulation control. OBJECTIVES: To identify patient-level predictors of oral anticoagulation control in the outpatient setting. PATIENTS/METHODS: We studied 124,619 patients who received oral anticoagulation from the Veterans Health Administration from October 2006 to September 2008. The outcome was anticoagulation control, summarized using percentage of time in therapeutic International Normalized Ratio range (TTR). Data were divided into inception (first 6 months of therapy; 39,447 patients) and experienced (any time thereafter; 104,505 patients). Patient-level predictors of TTR were examined by multivariable regression. RESULTS: Mean TTRs were 48% for inception management and 61% for experienced management. During inception, important predictors of TTR included hospitalizations (the expected TTR was 7.3% lower for those with two or more hospitalizations than for the non-hospitalized), receipt of more medications (16 or more medications predicted a 4.3% lower than for patients with 0-7 medications), alcohol abuse (-4.6%), cancer (-3.1%), and bipolar disorder (-2.9%). During the experienced period, important predictors of TTR included hospitalizations (four or more hospitalizations predicted 9.4% lower TTR), more medications (16 or more medications predicted 5.1% lower TTR), alcohol abuse (-5.4%), female sex (- 2.9%), cancer (-2.7%), dementia (-2.6%), non-alcohol substance abuse (-2.4%), and chronic liver disease (-2.3%). CONCLUSIONS: Some patients receiving oral anticoagulation therapy are more challenging to maintain within the therapeutic range than others. Our findings can be used to identify patients who require closer attention or innovative management strategies to maximize benefit and minimize harm from oral anticoagulation therapy

Risk-Adjusted Percent Time in Therapeutic Range as a Quality Indicator for Outpatient Oral Anticoagulation: Results of the Veterans Affairs Study To Improve Anticoagulation (VARIA)

Background- Oral anticoagulation is safer and more effective when patients receive high-quality care. However, there have been no prior efforts to measure quality of oral anticoagulation care or to risk adjust it to ensure credible comparisons. Our objective was to profile site performance in the Veterans Health Administration (VA) using risk-adjusted percent time in therapeutic range (TTR). Methods and Results- We included 124 551 patients who received outpatient oral anticoagulation from 100 VA sites of care for indications other than valvular heart disease from October 1, 2006, to September 30, 2008. We calculated TTR for each patient and mean TTR for each site of care. Expected TTR was calculated for each patient and each site based on the variables in the risk adjustment model, which included demographics, comorbid conditions, medications, and hospitalizations. Mean TTR for the entire sample was 58%. Site-observed TTR varied from 38% to 69% or from poor to excellent. Site-expected TTR varied from 54% to 62%. Site risk-adjusted performance ranged from 18% below expected to 12% above expected. Risk adjustment did not alter performance rankings for many sites, but for other sites, it made an important difference. For example, the site ranked 27th of 100 before risk adjustment was one of the best (risk-adjusted rank, 7). Risk-adjusted site rankings were consistent from year to year (correlation between years, 0.89). Conclusions- Risk-adjusted TTR can be used to profile the quality of outpatient oral anticoagulation in a large, integrated health system. This measure can serve as the basis for quality measurement and quality improvement efforts

Prompt Repeat Testing After Out-of-Range INR Values: A Quality Indicator for Anticoagulation Care

Background- Improved control of oral anticoagulation reduces adverse events. A program of quality measurement is needed for oral anticoagulation. The interval until the next test after an out-of-range International Normalized Ratio (INR) value (the follow-up interval ) could serve as a process of care measure. Methods and Results- We studied 104,451 patients cared for by 100 anticoagulation clinics in the Veterans Health Administration (VA). For each site, we computed the average follow-up interval after low (≤1.5) or or high (≥4.0) INR. Our outcome was each site\u27s average anticoagulation control, measured by percent time in therapeutic range (TTR); 59 837 patients (57%) contributed to the low INR analysis, 37 697 (36%) contributed to the high INR analysis, and all patients contributed to the dependent variable (mean site TTR). After a low INR, site mean follow-up interval ranged from 10 to 24 days. Longer follow-up intervals were associated with worse site-level control (1.04% lower for each additional day, P\u3c0.001). After a high INR, site mean follow-up interval ranged from 6 to 18 days, with longer follow-up intervals associated with worse site-level control (1.12% lower for each additional day, P\u3c0.001). These relationships were somewhat attenuated but still highly statistically significant when the proportion of INR values in-range was used as the dependent variable rather than TTR. Conclusions- Prompt repeat testing after out-of-range INR values is associated with better anticoagulation control at the site level and could be an important part of a quality improvement effort for oral anticoagulation

Re-Examining the Recommended Follow-Up Interval after an In-Range INR Value: Results from the Veterans AffaiRs Study to Improve Anticoagulation (VARIA)

BACKGROUND: Patients receiving oral anticoagulation should be tested often enough to optimize control, but excessive testing increases burden and cost. We examined the relationship between the follow-up interval after an in-range (2.0-3.0) INR and anticoagulation control. METHODS: We studied 104,451 patients receiving anticoagulation from 100 anticoagulation clinics in the Veterans Health Administration (VA). Most patients (98,877) recorded at least one in-range INR followed by another INR within 56 days. For each such patient, we selected the last in-range INR and characterized the interval between this index value and the next INR. The independent variable was site mean follow-up interval after an in-range INR. The dependent variable was site mean risk-adjusted percent time in therapeutic range (TTR). RESULTS: Site mean interval varied from 25-38 days. As the mean site follow-up interval became longer, risk-adjusted TTR was worse (-0.51% per day, p = 0.004). This relationship persisted when the index value was the first consecutive in-range INR (-0.63%, p \u3c 0.001), or the second (-0.58%, p \u3c 0.001), but not the third or greater (-0.12%, p = 0.46). CONCLUSIONS: Sites varied widely regarding follow-up intervals after in-range INR (25-38 days). Shorter intervals were generally associated with better anticoagulation control, but after a third consecutive in-range value, this relationship was greatly attenuated and no longer statistically significant. Our results suggest that a maximum interval of 28 days after the first or second in-range value and consideration of a longer interval after the third or greater consecutive in-range value may be appropriate

INR targets and site-level anticoagulation control: results from the Veterans AffaiRs Study to Improve Anticoagulation (VARIA)

BACKGROUND: Not all clinicians target the same International Normalized Ratio (INR) for patients with a guideline-recommended target range of 2-3. A patient\u27s mean INR value suggests the INR that was actually targeted. We hypothesized that sites would vary by mean INR, and that sites of care with mean values nearest to 2.5 would achieve better anticoagulation control, as measured by per cent time in therapeutic range (TTR). OBJECTIVES: To examine variations among sites in mean INR and the relationship with anticoagulation control in an integrated system of care. PATIENTS/METHODS: We studied 103,897 patients receiving oral anticoagulation with an expected INR target between 2 and 3 at 100 Veterans Health Administration (VA) sites from 1 October 2006 to 30 September 2008. Key site-level variables were: proportion near 2.5 (that is, percentage of patients with mean INR between 2.3 and 2.7) and mean risk-adjusted TTR. RESULTS: Site mean INR ranged from 2.22 to 2.89; proportion near 2.5, from 30 to 64%. Sites\u27 proportions of patients near 2.5, below 2.3 and above 2.7 were consistent from year to year. A 10 percentage point increase in the proportion near 2.5 predicted a 3.8 percentage point increase in risk-adjusted TTR (P \u3c 0.001). CONCLUSIONS: Proportion of patients with mean INR near 2.5 is a site-level \u27signature\u27 of care and an implicit measure of targeted INR. This proportion varies by site and is strongly associated with site-level TTR. Our study suggests that sites wishing to improve TTR, and thereby improve patient outcomes, should avoid the explicit or implicit pursuit of non-standard INR targets