Can We Geographically Validate a Natural Language Processing Algorithm for Automated Detection of Incidental Durotomy Across Three Independent Cohorts From Two Continents?

Background Incidental durotomy is an intraoperative complication in spine surgery that can lead to postoperative complications, increased length of stay, and higher healthcare costs. Natural language processing (NLP) is an artificial intelligence method that assists in understanding free-text notes that may be useful in the automated surveillance of adverse events in orthopaedic surgery. A previously developed NLP algorithm is highly accurate in the detection of incidental durotomy on internal validation and external validation in an independent cohort from the same country. External validation in a cohort with linguistic differences is required to assess the transportability of the developed algorithm, referred to geographical validation. Ideally, the performance of a prediction model, the NLP algorithm, is constant across geographic regions to ensure reproducibility and model validity. Question/purpose Can we geographically validate an NLP algorithm for the automated detection of incidental durotomy across three independent cohorts from two continents? Methods Patients 18 years or older undergoing a primary procedure of (thoraco)lumbar spine surgery were included. In Massachusetts, between January 2000 and June 2018, 1000 patients were included from two academic and three community medical centers. In Maryland, between July 2016 and November 2018, 1279 patients were included from one academic center, and in Australia, between January 2010 and December 2019, 944 patients were included from one academic center. The authors retrospectively studied the free-text operative notes of included patients for the primary outcome that was defined as intraoperative durotomy. Incidental durotomy occurred in 9% (93 of 1000), 8% (108 of 1279), and 6% (58 of 944) of the patients, respectively, in the Massachusetts, Maryland, and Australia cohorts. No missing reports were observed. Three datasets (Massachusetts, Australian, and combined Massachusetts and Australian) were divided into training and holdout test sets in an 80:20 ratio. An extreme gradient boosting (an efficient and flexible tree-based algorithm) NLP algorithm was individually trained on each training set, and the performance of the three NLP algorithms (respectively American, Australian, and combined) was assessed by discrimination via area under the receiver operating characteristic curves (AUC-ROC; this measures the model's ability to distinguish patients who obtained the outcomes from those who did not), calibration metrics (which plot the predicted and the observed probabilities) and Brier score (a composite of discrimination and calibration). In addition, the sensitivity (true positives, recall), specificity (true negatives), positive predictive value (also known as precision), negative predictive value, Fl-score (composite of precision and recall), positive likelihood ratio, and negative likelihood ratio were calculated. Results The combined NLP algorithm (the combined Massachusetts and Australian data) achieved excellent performance on independent testing data from Australia (AUC-ROC 0.97 [95% confidence interval 0.87 to 0.99]), Massachusetts (AUC-ROC 0.99 [95% CI 0.80 to 0.99]) and Maryland (AUC-ROC 0.95 [95% CI 0.93 to 0.97]). The NLP developed based on the Massachusetts cohort had excellent performance in the Maryland cohort (AUC-ROC 0.97 [95% CI 0.95 to 0.99]) but worse performance in the Australian cohort (AUC-ROC 0.74 [95% CI 0.70 to 0.77]). Conclusion We demonstrated the clinical utility and reproducibility of an NLP algorithm with combined datasets retaining excellent performance in individual countries relative to algorithms developed in the same country alone for detection of incidental durotomy. Further multi-institutional, international collaborations can facilitate the creation of universal NLP algorithms that improve the quality and safety of orthopaedic surgery globally. The combined NLP algorithm has been incorporated into a freely accessible web application that can be found at https://sorg-apps.shinyapps.io/nlp_incidental_durotomy/. Clinicians and researchers can use the tool to help incorporate the model in evaluating spine registries or quality and safety departments to automate detection of incidental durotomy and optimize prevention efforts

Research productivity during orthopedic surgery residency correlates with pre‑planned and protected research time: a survey of German‑speaking countries

Purpose The purpose of this study was to identify modifiable factors associated with research activity among residents working in orthopedic surgery and traumatology. Methods Residents at 796 university-affiliated hospitals in Austria, Germany, and Switzerland were invited to participate. The online survey consisted of questions that ascertained 13 modifiable and 17 non-modifiable factors associated with the residents’ current research activities. Responses of 129 residents were analyzed. Univariate linear regression was used to determine the association of individual factors with the current research activity (hours per week). The impact of significant non-modifiable factors (with unadjusted p values < 0.05) was controlled for using multivariate linear regression. Results The univariate analysis demonstrated six non-modifiable factors that were significantly associated with the current research activity: a University hospital setting (p < 0.001), an A-level hospital setting (p = 0.024), Swiss residents (p = 0.0012), the completion of a dedicated research year (p = 0.007), female gender (p = 0.016), and the department’s size (p = 0.048). Multivariate regression demonstrated that the number of protected research days per year (p < 0.029) and the percentage of protected days, that were known 1 week before (p < 0.001) or the day before (p < 0.001), were significantly associated with a higher research activity. Conclusions As hypothesized, more frequent and predictable protected research days were associated with higher research activity among residents in orthopedic surgery and traumatology. Level of evidence III

Vancomycin-impregnated calcium sulfate beads compared with vancomycin powder in adult spinal deformity patients undergoing thoracolumbar fusion

Introduction: Adult spinal deformity (ASD) surgery patients are at higher risk for surgical site infections (SSIs) due to large incisions, high blood loss, long surgical duration, and extensive instrumentation. The use of vancomycin powder has demonstrated inconsistent results in ASD surgery. Antibiotic-impregnated calcium sulfate beads have been used in arthroplasty and foot and ankle surgery with promising results. The purpose of this study was to provide preliminary data on the use of vancomycin-impregnated calcium sulfate beads in the prevention of SSI following ASD surgery and provide comparisons to the use of vancomycin powder. Methods: A retrospective chart review was performed for 95 consecutive surgical ASD patients at a tertiary care center from January 2017 until March 2020. Patients received either vancomycin powder (powder group) or vancomycin-impregnated calcium sulfate beads (bead group) intrawound prior to closure. Patient demographics, operative course, and incidence of postoperative infections were recorded. A two-tailed chi-squared test was performed to compare infection rates. Results: Ninety-five patients were included for review. Forty-two patients were in the powder group and 53 patients were in the bead group. The bead group was older (59.8 vs 67.8 years, p < 0.01) with similar BMI and rates of diabetes, smoking, and length of surgery. There were four postoperative SSI in the powder group requiring operative irrigation and debridement and one SSI in the bead group (9.5% vs 1.9%, p = 0.09). All infections occurred in the first 90 days of the postoperative period. Conclusion: Preliminary examination of the use of vancomycin-impregnated calcium sulfate beads demonstrated a 1.9% surgical site infection rate in adult spinal deformity surgical patients, which was not significantly different compared with the 9.5% infection rate in patients who received vancomycin powder. Prospective study is needed to determine if the differences found are significant in a larger number of patients

Supplemental Material - Neurological Outcomes and the Need for Retreatments Among Multiple Myeloma Patients With High-Grade Spinal Cord Compression: Radiotherapy vs Surgery

Supplemental Material for Neurological Outcomes and the Need for Retreatments Among Multiple Myeloma Patients With High-Grade Spinal Cord Compression: Radiotherapy vs Surgery by Hester Zijlstra, MD, Alexander M. Crawford, MD, Brendan M. Striano, MD, Robert-Jan Pierik, BSc, Daniel G. Tobert, MD, Nienke Wolterbeek, PhD, Diyar Delawi, MD, PhD, Wim E. Terpstra, MD, PhD, Diederik H. R. Kempen, MD, PhD, Jorrit-Jan Verlaan, MD, PhD, and Joseph H. Schwab, MD, MS in Global Spine Journal</p