Can We Geographically Validate a Natural Language Processing Algorithm for Automated Detection of Incidental Durotomy Across Three Independent Cohorts From Two Continents?

Background Incidental durotomy is an intraoperative complication in spine surgery that can lead to postoperative complications, increased length of stay, and higher healthcare costs. Natural language processing (NLP) is an artificial intelligence method that assists in understanding free-text notes that may be useful in the automated surveillance of adverse events in orthopaedic surgery. A previously developed NLP algorithm is highly accurate in the detection of incidental durotomy on internal validation and external validation in an independent cohort from the same country. External validation in a cohort with linguistic differences is required to assess the transportability of the developed algorithm, referred to geographical validation. Ideally, the performance of a prediction model, the NLP algorithm, is constant across geographic regions to ensure reproducibility and model validity. Question/purpose Can we geographically validate an NLP algorithm for the automated detection of incidental durotomy across three independent cohorts from two continents? Methods Patients 18 years or older undergoing a primary procedure of (thoraco)lumbar spine surgery were included. In Massachusetts, between January 2000 and June 2018, 1000 patients were included from two academic and three community medical centers. In Maryland, between July 2016 and November 2018, 1279 patients were included from one academic center, and in Australia, between January 2010 and December 2019, 944 patients were included from one academic center. The authors retrospectively studied the free-text operative notes of included patients for the primary outcome that was defined as intraoperative durotomy. Incidental durotomy occurred in 9% (93 of 1000), 8% (108 of 1279), and 6% (58 of 944) of the patients, respectively, in the Massachusetts, Maryland, and Australia cohorts. No missing reports were observed. Three datasets (Massachusetts, Australian, and combined Massachusetts and Australian) were divided into training and holdout test sets in an 80:20 ratio. An extreme gradient boosting (an efficient and flexible tree-based algorithm) NLP algorithm was individually trained on each training set, and the performance of the three NLP algorithms (respectively American, Australian, and combined) was assessed by discrimination via area under the receiver operating characteristic curves (AUC-ROC; this measures the model's ability to distinguish patients who obtained the outcomes from those who did not), calibration metrics (which plot the predicted and the observed probabilities) and Brier score (a composite of discrimination and calibration). In addition, the sensitivity (true positives, recall), specificity (true negatives), positive predictive value (also known as precision), negative predictive value, Fl-score (composite of precision and recall), positive likelihood ratio, and negative likelihood ratio were calculated. Results The combined NLP algorithm (the combined Massachusetts and Australian data) achieved excellent performance on independent testing data from Australia (AUC-ROC 0.97 [95% confidence interval 0.87 to 0.99]), Massachusetts (AUC-ROC 0.99 [95% CI 0.80 to 0.99]) and Maryland (AUC-ROC 0.95 [95% CI 0.93 to 0.97]). The NLP developed based on the Massachusetts cohort had excellent performance in the Maryland cohort (AUC-ROC 0.97 [95% CI 0.95 to 0.99]) but worse performance in the Australian cohort (AUC-ROC 0.74 [95% CI 0.70 to 0.77]). Conclusion We demonstrated the clinical utility and reproducibility of an NLP algorithm with combined datasets retaining excellent performance in individual countries relative to algorithms developed in the same country alone for detection of incidental durotomy. Further multi-institutional, international collaborations can facilitate the creation of universal NLP algorithms that improve the quality and safety of orthopaedic surgery globally. The combined NLP algorithm has been incorporated into a freely accessible web application that can be found at https://sorg-apps.shinyapps.io/nlp_incidental_durotomy/. Clinicians and researchers can use the tool to help incorporate the model in evaluating spine registries or quality and safety departments to automate detection of incidental durotomy and optimize prevention efforts

Research productivity during orthopedic surgery residency correlates with pre‑planned and protected research time: a survey of German‑speaking countries

Purpose The purpose of this study was to identify modifiable factors associated with research activity among residents working in orthopedic surgery and traumatology. Methods Residents at 796 university-affiliated hospitals in Austria, Germany, and Switzerland were invited to participate. The online survey consisted of questions that ascertained 13 modifiable and 17 non-modifiable factors associated with the residents’ current research activities. Responses of 129 residents were analyzed. Univariate linear regression was used to determine the association of individual factors with the current research activity (hours per week). The impact of significant non-modifiable factors (with unadjusted p values < 0.05) was controlled for using multivariate linear regression. Results The univariate analysis demonstrated six non-modifiable factors that were significantly associated with the current research activity: a University hospital setting (p < 0.001), an A-level hospital setting (p = 0.024), Swiss residents (p = 0.0012), the completion of a dedicated research year (p = 0.007), female gender (p = 0.016), and the department’s size (p = 0.048). Multivariate regression demonstrated that the number of protected research days per year (p < 0.029) and the percentage of protected days, that were known 1 week before (p < 0.001) or the day before (p < 0.001), were significantly associated with a higher research activity. Conclusions As hypothesized, more frequent and predictable protected research days were associated with higher research activity among residents in orthopedic surgery and traumatology. Level of evidence III

Social determinants of health in prognostic machine learning models for orthopaedic outcomes: A systematic review

Rational: Social determinants of health (SDOH) are being considered more frequently when providing orthopaedic care due to their impact on treatment outcomes. Simultaneously, prognostic machine learning (ML) models that facilitate clinical decision making have become popular tools in the field of orthopaedic surgery. When ML-driven tools are developed, it is important that the perpetuation of potential disparities is minimized. One approach is to consider SDOH during model development. To date, it remains unclear whether and how existing prognostic ML models for orthopaedic outcomes consider SDOH variables. Objective: To investigate whether prognostic ML models for orthopaedic surgery outcomes account for SDOH, and to what extent SDOH variables are included in the final models. Methods: A systematic search was conducted in PubMed, Embase and Cochrane for studies published up to 17 November 2020. Two reviewers independently extracted SDOH features using the PROGRESS+ framework (place of residence, race/ethnicity, Occupation, gender/sex, religion, education, social capital, socioeconomic status, ‘Plus+’ age, disability, and sexual orientation). Results: The search yielded 7138 studies, of which 59 met the inclusion criteria. Across all studies, 96% (57/59) considered at least one PROGRESS+ factor during development. The most common factors were age (95%; 56/59) and gender/sex (96%; 57/59). Differential effect analyses, such as subgroup analysis, covariate adjustment, and baseline comparison, were rarely reported (10%; 6/59). The majority of models included age (92%; 54/59) and gender/sex (69%; 41/59) as final input variables. However, factors such as insurance status (7%; 4/59), marital status (7%; 4/59) and income (3%; 2/59) were seldom included. Conclusion: The current level of reporting and consideration of SDOH during the development of prognostic ML models for orthopaedic outcomes is limited. Healthcare providers should be critical of the models they consider using and knowledgeable regarding the quality of model development, such as adherence to recognized methodological standards. Future efforts should aim to avoid bias and disparities when developing ML-driven applications for orthopaedics

Inpatient Opioid Use Varies by Construct Length Among Laminoplasty Versus Laminectomy and Fusion Patients

BACKGROUND: Laminoplasty (LP) and laminectomy and fusion (LF) are utilized to achieve decompression in patients with symptomatic degenerative cervical myelopathy (DCM). Comparative analyses aimed at determining outcomes and clarifying indications between these procedures represent an area of active research. Accordingly, we sought to compare inpatient opioid use between LP and LF patients and to determine if opioid use correlated with length of stay. METHODS: Sociodemographic information, surgical and hospitalization data, and medication administration records were abstracted for patients \u3e18 years of age who underwent LP or LF for DCM in the Mass General Brigham (MGB) health system between 2017 and 2019. Specifically, morphine milligram equivalents (MME) of oral and parenteral pain medication given after arrival in the recovery area until discharge from the hospital were collected. Categorical variables were analyzed using chi-squared analysis or Fisher exact test when appropriate. Continuous variables were compared using Independent samples RESULTS: One hundred eight patients underwent LF, while 138 patients underwent LP. Total inpatient opioid use was significantly higher in the LF group (312 vs. 260 MME, p=.03); this difference was primarily driven by higher postoperative day 0 pain medication requirements. Furthermore, more LF patients required high dose (\u3e80 MME/day) regimens. While length of stay was significantly different between groups, with LF patients staying approximately 1 additional day, postoperative day 0 MME was not a significant predictor of this difference. When operative levels including C2, T1, and T2 were excluded, the differences in total opioid use and average length of stay lost significance. CONCLUSIONS: Inpatient opioid use and length of stay were significantly greater in LF patients compared to LP patients; however, when constructs including C2, T1, T2 were excluded from analysis, these differences lost significance. Such findings highlight the impact of operative extent between these procedures. Future studies incorporating patient reported outcomes and evaluating long-term pain needs will provide a more complete understanding of postoperative outcomes between these 2 procedures

Vancomycin-impregnated calcium sulfate beads compared with vancomycin powder in adult spinal deformity patients undergoing thoracolumbar fusion

Introduction: Adult spinal deformity (ASD) surgery patients are at higher risk for surgical site infections (SSIs) due to large incisions, high blood loss, long surgical duration, and extensive instrumentation. The use of vancomycin powder has demonstrated inconsistent results in ASD surgery. Antibiotic-impregnated calcium sulfate beads have been used in arthroplasty and foot and ankle surgery with promising results. The purpose of this study was to provide preliminary data on the use of vancomycin-impregnated calcium sulfate beads in the prevention of SSI following ASD surgery and provide comparisons to the use of vancomycin powder. Methods: A retrospective chart review was performed for 95 consecutive surgical ASD patients at a tertiary care center from January 2017 until March 2020. Patients received either vancomycin powder (powder group) or vancomycin-impregnated calcium sulfate beads (bead group) intrawound prior to closure. Patient demographics, operative course, and incidence of postoperative infections were recorded. A two-tailed chi-squared test was performed to compare infection rates. Results: Ninety-five patients were included for review. Forty-two patients were in the powder group and 53 patients were in the bead group. The bead group was older (59.8 vs 67.8 years, p < 0.01) with similar BMI and rates of diabetes, smoking, and length of surgery. There were four postoperative SSI in the powder group requiring operative irrigation and debridement and one SSI in the bead group (9.5% vs 1.9%, p = 0.09). All infections occurred in the first 90 days of the postoperative period. Conclusion: Preliminary examination of the use of vancomycin-impregnated calcium sulfate beads demonstrated a 1.9% surgical site infection rate in adult spinal deformity surgical patients, which was not significantly different compared with the 9.5% infection rate in patients who received vancomycin powder. Prospective study is needed to determine if the differences found are significant in a larger number of patients

Reliability of self-reported health literacy screening in spine patients

BACKGROUND CONTEXT: Limited health literacy has previously been associated with increased health care utilization, worse general health status and self-reported health, and increased mortality. Identifying and accommodating patients with limited health literacy may offer an avenue towards mitigating adverse health outcomes and reduce unnecessary health care expenditure. Due to the challenges associated with implementation of lengthy health literacy assessments, the Brief Health Literacy Screening Instrument was developed. However, to our knowledge, there are no reports on the accuracy of this screening questionnaire, with or without the inclusion of sociodemographic characteristics, when predicting limited health literacy in orthopaedic spine patients. PURPOSE: To evaluate the reliability and predictive accuracy of self-reported health literacy screening questions with and without the inclusion of sociodemographic variables in orthopaedic spine patients. STUDY DESIGN: Cross-sectional. PATIENT SAMPLE: Patients seen at a tertiary urban academic hospital-based multi-surgeon spine center OUTCOME MEASURES: Brief Health Literacy Screening Instrument (BRIEF), and the Newest Vital Sign (NVS) health literacy assessment tool. METHODS: Between December 2021 and February 2022, consecutive English-speaking patients over the age of 18 presenting as new patients to an urban, hospital-based outpatient spine clinic were approached for participation. A sociodemographic survey, the BRIEF, and the NVS Health Literacy Assessment Tool were administered verbally. Simple and multivariable logistic regression was utilized to assess the accuracy of each BRIEF question individually, and collectively, at predicting limited health literacy as defined by the NVS. Further regression analysis included sociodemographic variables (age, body mass index, race, ethnicity, highest educational degree, employment status, marital status, annual household income, insurance status, and self-reported health. RESULTS: A total of 262 patients [mean age (years), 57 ± 17] were included in this study. One hundred thirty-four (51%) were male, 223 (85%) were White, and 151 (58%) were married. Patient BRIEF scores were as follows: 23 (9%) limited, 43 (16%) marginal, and 196 (75%) adequate. NVS scores identified 87 (33%) patients with possible limited health literacy. BRIEF items collectively demonstrated fair accuracy in the prediction of limited health literacy (area under the receiver operating characteristic curve, 0.76; 95% CI, 0.70−0.82). Individually, the fourth BRIEF item (“How confident are you in filling out medical forms by yourself?”) was the best predictor of limited health literacy (area under the receiver operating characteristic curve, 0.67; 95% CI, 0.60−0.73). The predictive accuracy of the BRIEF items, both individually and collectively, increased with the inclusion of sociodemographic variables within the logistic regression. Specific characteristics independently associated with limited health literacy were self-identified Black race, retired or disabled employment status, single or divorced marital status, high school education or below, and self-reported “poor” health. CONCLUSIONS: Limited health literacy has implications for patient outcomes and health care costs. Our results show that the BRIEF questionnaire is a low-cost screening tool that demonstrates fair predictability in determining limited health literacy within a population of spine patients. Self-reported health literacy assessments may be more feasible in daily practice and easier to implement into clinical workflow

Prevalence of- and factors associated with limited health literacy in spine patients

Background: Limited health literacy exacerbates health inequity and has serious implications for patient care. It hinders successful communication and comprehension of relevant health information, which can lead to suboptimal care. Despite the evidence regarding the significance of health literacy, the topic has received little consideration in orthopedic spine patients. Purpose: To investigate the prevalence of- and factors associated with limited health literacy among outpatients presenting to a tertiary urban academic hospital-based orthopedic spine center. Study design: Cross-sectionals. Patient sample: Patients 18 years of age or older seen at a tertiary urban academic hospital-based multi-surgeon outpatient spine center. Outcome measures: The Newest Vital Sign (NVS) health literacy assessment. Methods: Between December 2021 and March 2022, 447 consecutive English-speaking patients over the age of 18 years and new to the outpatient spine clinic were approached for participation in a cross-sectional survey study, of which 405 agreed to participate. Patients completed the Newest Vital Sign (NVS) health literacy assessment tool, the Rapid Estimation of Adult Literacy in Medicine Short Form (REALM-SF), and a sociodemographic survey (including race/ethnicity, level of education, employment status, income, and marital status). The NVS scores were divided into limited (0–3) and adequate (4–6) health literacy. REALM-SF scores were classified into reading levels below ninth grade (0–6) or at least ninth grade (7). Additional demographic data was extracted from patient records. Online mapping tools were used to collect the Social Vulnerability Index (SVI) and the Area Deprivation Index (ADI) for each patient. Subsequently, multivariable regression modeling was performed to identify independent factors associated with limited health literacy. Results: The prevalence of limited health literacy in patients presenting to an urban academic outpatient spine center was 33% (135/405). Unadjusted analysis found that patients who were socioeconomically disadvantaged (eg, unemployed, lower household income, publicly insured and higher SVI) and had more unfavorable social determinant of health features (eg, housing concerns, higher ADI, less years of education, below ninth grade reading level, unmarried) had high rates of limited health literacy. Adjusted regression analysis demonstrated that limited health literacy was independently associated with higher ADI state decile, living less than 10 years at current address, having housing concerns, not being employed, non-native English speaking, having less years of education and below ninth grade reading level. Conclusions: This study found that a substantial portion of the patients presenting to an outpatient spine center have limited health literacy, more so if they are socially disadvantaged. Future efforts should investigate the impact of limited health literacy on access to care, treatment outcomes and health care utilization in orthopedic patients. Neighborhood social vulnerability measures may be a feasible way to identify patients at risk of limited health literacy in clinical practice and offer opportunities for tailored patient care. This may contribute to prioritizing the mitigation of disparities and aid in the development of meaningful interventions to improve health equity in orthopedics

Impact of Health Literacy on Self-Reported Health Outcomes in Spine Patients

Study Design. Cross-sectional survey study. Objective. The aim was to determine if health literacy level is associated with patient-reported outcomes and self-reported health status among patients presenting to an academic outpatient spine center. Summary of Background Data. Patient reports are critical to assessing symptom severity and treatment success in orthopedic spine patients. Patient-reported outcome measures (PROMs) are important instruments commonly used for this purpose. However, the influence of patient health literacy on PROMs has not yet been given much consideration in spine literature. Materials and Methods. Consecutive English-speaking patients over the age of 18 years and new to our clinic verbally completed the Newest Vital Sign health literacy assessment tool and a sociodemographic survey, including self-reported health status. In addition, seven Patient-Reported Outcomes Measurement Information System scores were extracted from patient records. Regression modeling was performed with PROMs considered as dependent variables, health literacy level as the primary predictor, and all other factors (age, sex, race, ethnicity, native English speaker, highest educational degree, grade-level reading, marital status, employment status, annual household income, and type of insurance) as covariates. Results. Among the 318 included patients, 33% had limited health literacy. Adjusted regression analysis demonstrated that patients with limited health literacy had worse PROM scores across all seven domains (Physical Function: P=0.028; Depression: P=0.035; Global Health - Physical: P=0.001; Global Health - Mental: P=0.007; Pain Interference: P=0.036; Pain Intensity: P=0.002; Anxiety: P=0.047). In addition, patients with limited health literacy reported worse self-reported health status (P<0.001). Conclusions. Spine patients with limited health literacy have worse baseline PROM scores confounders and report worse general health. Further investigations are necessary to elucidate if limited health literacy is a marker or the root cause of these disparities. Findings from this study urge the consideration of patient health literacy when interpreting PROMs as well as the implications for patient assessment and discussion of treatment options