Can We Geographically Validate a Natural Language Processing Algorithm for Automated Detection of Incidental Durotomy Across Three Independent Cohorts From Two Continents?

Background Incidental durotomy is an intraoperative complication in spine surgery that can lead to postoperative complications, increased length of stay, and higher healthcare costs. Natural language processing (NLP) is an artificial intelligence method that assists in understanding free-text notes that may be useful in the automated surveillance of adverse events in orthopaedic surgery. A previously developed NLP algorithm is highly accurate in the detection of incidental durotomy on internal validation and external validation in an independent cohort from the same country. External validation in a cohort with linguistic differences is required to assess the transportability of the developed algorithm, referred to geographical validation. Ideally, the performance of a prediction model, the NLP algorithm, is constant across geographic regions to ensure reproducibility and model validity. Question/purpose Can we geographically validate an NLP algorithm for the automated detection of incidental durotomy across three independent cohorts from two continents? Methods Patients 18 years or older undergoing a primary procedure of (thoraco)lumbar spine surgery were included. In Massachusetts, between January 2000 and June 2018, 1000 patients were included from two academic and three community medical centers. In Maryland, between July 2016 and November 2018, 1279 patients were included from one academic center, and in Australia, between January 2010 and December 2019, 944 patients were included from one academic center. The authors retrospectively studied the free-text operative notes of included patients for the primary outcome that was defined as intraoperative durotomy. Incidental durotomy occurred in 9% (93 of 1000), 8% (108 of 1279), and 6% (58 of 944) of the patients, respectively, in the Massachusetts, Maryland, and Australia cohorts. No missing reports were observed. Three datasets (Massachusetts, Australian, and combined Massachusetts and Australian) were divided into training and holdout test sets in an 80:20 ratio. An extreme gradient boosting (an efficient and flexible tree-based algorithm) NLP algorithm was individually trained on each training set, and the performance of the three NLP algorithms (respectively American, Australian, and combined) was assessed by discrimination via area under the receiver operating characteristic curves (AUC-ROC; this measures the model's ability to distinguish patients who obtained the outcomes from those who did not), calibration metrics (which plot the predicted and the observed probabilities) and Brier score (a composite of discrimination and calibration). In addition, the sensitivity (true positives, recall), specificity (true negatives), positive predictive value (also known as precision), negative predictive value, Fl-score (composite of precision and recall), positive likelihood ratio, and negative likelihood ratio were calculated. Results The combined NLP algorithm (the combined Massachusetts and Australian data) achieved excellent performance on independent testing data from Australia (AUC-ROC 0.97 [95% confidence interval 0.87 to 0.99]), Massachusetts (AUC-ROC 0.99 [95% CI 0.80 to 0.99]) and Maryland (AUC-ROC 0.95 [95% CI 0.93 to 0.97]). The NLP developed based on the Massachusetts cohort had excellent performance in the Maryland cohort (AUC-ROC 0.97 [95% CI 0.95 to 0.99]) but worse performance in the Australian cohort (AUC-ROC 0.74 [95% CI 0.70 to 0.77]). Conclusion We demonstrated the clinical utility and reproducibility of an NLP algorithm with combined datasets retaining excellent performance in individual countries relative to algorithms developed in the same country alone for detection of incidental durotomy. Further multi-institutional, international collaborations can facilitate the creation of universal NLP algorithms that improve the quality and safety of orthopaedic surgery globally. The combined NLP algorithm has been incorporated into a freely accessible web application that can be found at https://sorg-apps.shinyapps.io/nlp_incidental_durotomy/. Clinicians and researchers can use the tool to help incorporate the model in evaluating spine registries or quality and safety departments to automate detection of incidental durotomy and optimize prevention efforts

Research productivity during orthopedic surgery residency correlates with pre‑planned and protected research time: a survey of German‑speaking countries

Purpose The purpose of this study was to identify modifiable factors associated with research activity among residents working in orthopedic surgery and traumatology. Methods Residents at 796 university-affiliated hospitals in Austria, Germany, and Switzerland were invited to participate. The online survey consisted of questions that ascertained 13 modifiable and 17 non-modifiable factors associated with the residents’ current research activities. Responses of 129 residents were analyzed. Univariate linear regression was used to determine the association of individual factors with the current research activity (hours per week). The impact of significant non-modifiable factors (with unadjusted p values < 0.05) was controlled for using multivariate linear regression. Results The univariate analysis demonstrated six non-modifiable factors that were significantly associated with the current research activity: a University hospital setting (p < 0.001), an A-level hospital setting (p = 0.024), Swiss residents (p = 0.0012), the completion of a dedicated research year (p = 0.007), female gender (p = 0.016), and the department’s size (p = 0.048). Multivariate regression demonstrated that the number of protected research days per year (p < 0.029) and the percentage of protected days, that were known 1 week before (p < 0.001) or the day before (p < 0.001), were significantly associated with a higher research activity. Conclusions As hypothesized, more frequent and predictable protected research days were associated with higher research activity among residents in orthopedic surgery and traumatology. Level of evidence III

Social determinants of health in prognostic machine learning models for orthopaedic outcomes: A systematic review

Rational: Social determinants of health (SDOH) are being considered more frequently when providing orthopaedic care due to their impact on treatment outcomes. Simultaneously, prognostic machine learning (ML) models that facilitate clinical decision making have become popular tools in the field of orthopaedic surgery. When ML-driven tools are developed, it is important that the perpetuation of potential disparities is minimized. One approach is to consider SDOH during model development. To date, it remains unclear whether and how existing prognostic ML models for orthopaedic outcomes consider SDOH variables. Objective: To investigate whether prognostic ML models for orthopaedic surgery outcomes account for SDOH, and to what extent SDOH variables are included in the final models. Methods: A systematic search was conducted in PubMed, Embase and Cochrane for studies published up to 17 November 2020. Two reviewers independently extracted SDOH features using the PROGRESS+ framework (place of residence, race/ethnicity, Occupation, gender/sex, religion, education, social capital, socioeconomic status, ‘Plus+’ age, disability, and sexual orientation). Results: The search yielded 7138 studies, of which 59 met the inclusion criteria. Across all studies, 96% (57/59) considered at least one PROGRESS+ factor during development. The most common factors were age (95%; 56/59) and gender/sex (96%; 57/59). Differential effect analyses, such as subgroup analysis, covariate adjustment, and baseline comparison, were rarely reported (10%; 6/59). The majority of models included age (92%; 54/59) and gender/sex (69%; 41/59) as final input variables. However, factors such as insurance status (7%; 4/59), marital status (7%; 4/59) and income (3%; 2/59) were seldom included. Conclusion: The current level of reporting and consideration of SDOH during the development of prognostic ML models for orthopaedic outcomes is limited. Healthcare providers should be critical of the models they consider using and knowledgeable regarding the quality of model development, such as adherence to recognized methodological standards. Future efforts should aim to avoid bias and disparities when developing ML-driven applications for orthopaedics

Inpatient Opioid Use Varies by Construct Length Among Laminoplasty Versus Laminectomy and Fusion Patients

BACKGROUND: Laminoplasty (LP) and laminectomy and fusion (LF) are utilized to achieve decompression in patients with symptomatic degenerative cervical myelopathy (DCM). Comparative analyses aimed at determining outcomes and clarifying indications between these procedures represent an area of active research. Accordingly, we sought to compare inpatient opioid use between LP and LF patients and to determine if opioid use correlated with length of stay. METHODS: Sociodemographic information, surgical and hospitalization data, and medication administration records were abstracted for patients \u3e18 years of age who underwent LP or LF for DCM in the Mass General Brigham (MGB) health system between 2017 and 2019. Specifically, morphine milligram equivalents (MME) of oral and parenteral pain medication given after arrival in the recovery area until discharge from the hospital were collected. Categorical variables were analyzed using chi-squared analysis or Fisher exact test when appropriate. Continuous variables were compared using Independent samples RESULTS: One hundred eight patients underwent LF, while 138 patients underwent LP. Total inpatient opioid use was significantly higher in the LF group (312 vs. 260 MME, p=.03); this difference was primarily driven by higher postoperative day 0 pain medication requirements. Furthermore, more LF patients required high dose (\u3e80 MME/day) regimens. While length of stay was significantly different between groups, with LF patients staying approximately 1 additional day, postoperative day 0 MME was not a significant predictor of this difference. When operative levels including C2, T1, and T2 were excluded, the differences in total opioid use and average length of stay lost significance. CONCLUSIONS: Inpatient opioid use and length of stay were significantly greater in LF patients compared to LP patients; however, when constructs including C2, T1, T2 were excluded from analysis, these differences lost significance. Such findings highlight the impact of operative extent between these procedures. Future studies incorporating patient reported outcomes and evaluating long-term pain needs will provide a more complete understanding of postoperative outcomes between these 2 procedures

Reliability of self-reported health literacy screening in spine patients

BACKGROUND CONTEXT: Limited health literacy has previously been associated with increased health care utilization, worse general health status and self-reported health, and increased mortality. Identifying and accommodating patients with limited health literacy may offer an avenue towards mitigating adverse health outcomes and reduce unnecessary health care expenditure. Due to the challenges associated with implementation of lengthy health literacy assessments, the Brief Health Literacy Screening Instrument was developed. However, to our knowledge, there are no reports on the accuracy of this screening questionnaire, with or without the inclusion of sociodemographic characteristics, when predicting limited health literacy in orthopaedic spine patients. PURPOSE: To evaluate the reliability and predictive accuracy of self-reported health literacy screening questions with and without the inclusion of sociodemographic variables in orthopaedic spine patients. STUDY DESIGN: Cross-sectional. PATIENT SAMPLE: Patients seen at a tertiary urban academic hospital-based multi-surgeon spine center OUTCOME MEASURES: Brief Health Literacy Screening Instrument (BRIEF), and the Newest Vital Sign (NVS) health literacy assessment tool. METHODS: Between December 2021 and February 2022, consecutive English-speaking patients over the age of 18 presenting as new patients to an urban, hospital-based outpatient spine clinic were approached for participation. A sociodemographic survey, the BRIEF, and the NVS Health Literacy Assessment Tool were administered verbally. Simple and multivariable logistic regression was utilized to assess the accuracy of each BRIEF question individually, and collectively, at predicting limited health literacy as defined by the NVS. Further regression analysis included sociodemographic variables (age, body mass index, race, ethnicity, highest educational degree, employment status, marital status, annual household income, insurance status, and self-reported health. RESULTS: A total of 262 patients [mean age (years), 57 ± 17] were included in this study. One hundred thirty-four (51%) were male, 223 (85%) were White, and 151 (58%) were married. Patient BRIEF scores were as follows: 23 (9%) limited, 43 (16%) marginal, and 196 (75%) adequate. NVS scores identified 87 (33%) patients with possible limited health literacy. BRIEF items collectively demonstrated fair accuracy in the prediction of limited health literacy (area under the receiver operating characteristic curve, 0.76; 95% CI, 0.70−0.82). Individually, the fourth BRIEF item (“How confident are you in filling out medical forms by yourself?”) was the best predictor of limited health literacy (area under the receiver operating characteristic curve, 0.67; 95% CI, 0.60−0.73). The predictive accuracy of the BRIEF items, both individually and collectively, increased with the inclusion of sociodemographic variables within the logistic regression. Specific characteristics independently associated with limited health literacy were self-identified Black race, retired or disabled employment status, single or divorced marital status, high school education or below, and self-reported “poor” health. CONCLUSIONS: Limited health literacy has implications for patient outcomes and health care costs. Our results show that the BRIEF questionnaire is a low-cost screening tool that demonstrates fair predictability in determining limited health literacy within a population of spine patients. Self-reported health literacy assessments may be more feasible in daily practice and easier to implement into clinical workflow

Prevalence of- and factors associated with limited health literacy in spine patients

Background: Limited health literacy exacerbates health inequity and has serious implications for patient care. It hinders successful communication and comprehension of relevant health information, which can lead to suboptimal care. Despite the evidence regarding the significance of health literacy, the topic has received little consideration in orthopedic spine patients. Purpose: To investigate the prevalence of- and factors associated with limited health literacy among outpatients presenting to a tertiary urban academic hospital-based orthopedic spine center. Study design: Cross-sectionals. Patient sample: Patients 18 years of age or older seen at a tertiary urban academic hospital-based multi-surgeon outpatient spine center. Outcome measures: The Newest Vital Sign (NVS) health literacy assessment. Methods: Between December 2021 and March 2022, 447 consecutive English-speaking patients over the age of 18 years and new to the outpatient spine clinic were approached for participation in a cross-sectional survey study, of which 405 agreed to participate. Patients completed the Newest Vital Sign (NVS) health literacy assessment tool, the Rapid Estimation of Adult Literacy in Medicine Short Form (REALM-SF), and a sociodemographic survey (including race/ethnicity, level of education, employment status, income, and marital status). The NVS scores were divided into limited (0–3) and adequate (4–6) health literacy. REALM-SF scores were classified into reading levels below ninth grade (0–6) or at least ninth grade (7). Additional demographic data was extracted from patient records. Online mapping tools were used to collect the Social Vulnerability Index (SVI) and the Area Deprivation Index (ADI) for each patient. Subsequently, multivariable regression modeling was performed to identify independent factors associated with limited health literacy. Results: The prevalence of limited health literacy in patients presenting to an urban academic outpatient spine center was 33% (135/405). Unadjusted analysis found that patients who were socioeconomically disadvantaged (eg, unemployed, lower household income, publicly insured and higher SVI) and had more unfavorable social determinant of health features (eg, housing concerns, higher ADI, less years of education, below ninth grade reading level, unmarried) had high rates of limited health literacy. Adjusted regression analysis demonstrated that limited health literacy was independently associated with higher ADI state decile, living less than 10 years at current address, having housing concerns, not being employed, non-native English speaking, having less years of education and below ninth grade reading level. Conclusions: This study found that a substantial portion of the patients presenting to an outpatient spine center have limited health literacy, more so if they are socially disadvantaged. Future efforts should investigate the impact of limited health literacy on access to care, treatment outcomes and health care utilization in orthopedic patients. Neighborhood social vulnerability measures may be a feasible way to identify patients at risk of limited health literacy in clinical practice and offer opportunities for tailored patient care. This may contribute to prioritizing the mitigation of disparities and aid in the development of meaningful interventions to improve health equity in orthopedics

Limited Health Literacy Results in Lower Health-Related Quality of Life in Spine Patients

Background Context: Spinal conditions impact health-related quality of life (HRQoL). Patient education and counseling improve HRQoL, yet the effects may be limited for patients with inadequate health literacy (HL). Despite the established relationship between HRQoL and HL in other fields, research in the orthopedic spine population is lacking. Purpose: To investigate if limited HL results in lower HRQoL and to evaluate factors are associated with HRQoL in patients seen at an outpatient orthopedic spine center. Study Design/Setting: Prospective single-center cross-sectional study. Patient Sample: Patients 18 years of age or older seen at a tertiary urban academic hospital- based multi-surgeon outpatient spine center. Outcome Measures: EQ-5D-5L health-related quality of life (HRQoL) questionnaire, and the Newest Vital Sign (NVS) HL assessment tool. Methods: Between October 2022 and February 2023, consecutive English-speaking patients over the age of 18 and new to the outpatient spine clinic were approached for participation in this cross-sectional survey study. Patients completed a sociodemographic survey, EQ-5D-5L HRQoL questionnaire, and Newest Vital Sign (NVS) HL assessment tool. The EQ-5D-5L yields two continuous outcomes: an index score ranging from below 0 to 1 and a visual analog scale (EQ-VAS) score ranging from 0 to 100. The NVS scores were divided into limited (0–3) and adequate (4–6) HL. Multivariate linear regression with purposeful selection of variables was performed to identify independent factors associated with HRQoL. Results: Out of 397 eligible patients, 348 (88%) agreed to participate and were included in statistical analysis. Limited HL was independently associated with lower EQ-5D-5L index scores (B=1.07 [95% CI 1.00–1.15], p=.049. Other factors associated with lower EQ-5D-5L index scores were being obese (BMI≥30), having housing concerns, and being an active smoker. Factors associated with lower EQ-VAS scores were being underweight (BMI<18.5), obese, having housing concerns, and higher updated Charlson comorbidity index (uCCI) scores. Being married was associated with higher EQ-VAS scores. Conclusions: Limited HL is associated with lower EQ-5D-5L index scores in spine patients, indicating lower HRQoL. To effectively apply HL-related interventions in this population, a better understanding of the complex interactions between patient characteristics, social determinants of health, and HRQoL outcomes is required. Further research should focus on interventions to improve HRQoL in patients with limited HL and how to accurately identify these patients. Level of evidence: Level II prognostic

Pedicled omental flaps for complex wound reconstruction following surgery for primary spine tumors of the mobile spine and sacrum

Surgery for primary tumors of the mobile spine and sacrum often requires complex reconstruction techniques to cover soft-tissue defects and to treat wound and CSF-related complications. The anatomical, vascular, and immunoregulatory characteristics of the omentum make it an excellent local substrate for the management of radiation soft-tissue injury, infection, and extensive wound defects. This study describes the authors' experience in complex wound reconstruction using pedicled omental flaps to cover defects in surgery for mobile spine and sacral primary tumors. A retrospective cohort analysis was conducted on 34 patients who underwent pedicled omental flap reconstruction after en bloc resection of primary sacral and mobile spine tumors between 2010 and 2020. The study focused on assessing the indications for omental flap usage, including soft-tissue coverage, protection against postoperative radiation therapy, infection management, vascular supply for bone grafts, and dural defect and CSF leak repair. Patient demographic characteristics, tumor characteristics, surgical outcomes, and follow-up data were analyzed to determine the procedure's efficacy and complication rates. From 2010 to 2020, 34 patients underwent pedicled omental flap reconstruction after en bloc resection of sacral (24 of 34 [71%]) and mobile spine (10 of 34 [29%]) primary tumors, mostly chordomas. The patient cohort included 21 men and 13 women with a median (range) age of 60 (32-89) years. The most common indication for omental flap was soft-tissue coverage (20 of 34 [59%]). Other indications included protecting abdominopelvic organs for postoperative radiation therapy (6 of 34 [18%]), treating infections (5 of 34 [15%]), providing vascular supply for free fibular bone graft (1 of 34 [3%]), and repairing large dural defects and CSF leak (2 of 34 [6%]). The median (range) follow-up was 24 (0-132) months, during which 71% (24 of 34) of patients did not require additional surgery for wound-related complications. At last follow-up, 59% (20 of 34) had stable disease and 32% (11 of 34) had recurrence, had progression of disease, or had been discharged to hospice after treatment. The pedicled omentum is an effective local tissue graft that can be used for complex wound reconstruction and management of high-risk closures in primary spine tumors. This technique may have a lower rate of complications than other approaches and may influence surgical planning and flap selection in challenging cases