Machine learning based natural language processing of radiology reports in orthopaedic trauma

OBJECTIVES: To compare different Machine Learning (ML) Natural Language Processing (NLP) methods to classify radiology reports in orthopaedic trauma for the presence of injuries. Assessing NLP performance is a prerequisite for downstream tasks and therefore of importance from a clinical perspective (avoiding missed injuries, quality check, insight in diagnostic yield) as well as from a research perspective (identification of patient cohorts, annotation of radiographs). METHODS: Datasets of Dutch radiology reports of injured extremities (n = 2469, 33% fractures) and chest radiographs (n = 799, 20% pneumothorax) were collected in two different hospitals and labeled by radiologists and trauma surgeons for the presence or absence of injuries. NLP classification was applied and optimized by testing different preprocessing steps and different classifiers (Rule-based, ML, and Bidirectional Encoder Representations from Transformers (BERT)). Performance was assessed by F1-score, AUC, sensitivity, specificity and accuracy. RESULTS: The deep learning based BERT model outperforms all other classification methods which were assessed. The model achieved an F1-score of (95 ± 2)% and accuracy of (96 ± 1)% on a dataset of simple reports (n= 2469), and an F1 of (83 ± 7)% with accuracy (93 ± 2)% on a dataset of complex reports (n= 799). CONCLUSION: BERT NLP outperforms traditional ML and rule-base classifiers when applied to Dutch radiology reports in orthopaedic trauma

What Are the Interobserver and Intraobserver Variability of Gap and Stepoff Measurements in Acetabular Fractures?

BACKGROUND: Gap and stepoff values in the treatment of acetabular fractures are correlated with clinical outcomes. However, the interobserver and intraobserver variability of gap and stepoff measurements for all imaging modalities in the preoperative, intraoperative, and postoperative phase of treatment is unknown. Recently, a standardized CT-based measurement method was introduced, which provided the opportunity to assess the level of variability. QUESTIONS/PURPOSES: (1) In patients with acetabular fractures, what is the interobserver variability in the measurement of the fracture gaps and articular stepoffs determined by each observer to be the maximum one in the weightbearing dome, as measured on pre- and postoperative pelvic radiographs, intraoperative fluoroscopy, and pre- and postoperative CT scans? (2) What is the intraobserver variability in these measurements? METHODS: Sixty patients with a complete subset of pre-, intra- and postoperative high-quality images (CT slices of < 2 mm), representing a variety of fracture types with small and large gaps and/or stepoffs, were included. A total of 196 patients with nonoperative treatment (n = 117), inadequate available imaging (n = 60), skeletal immaturity (n = 16), bilateral fractures (n = 2) or a primary THA (n = 1) were excluded. The maximum gap and stepoff values in the weightbearing dome were digitally measured on pelvic radiographs and CT images by five independent observers. Observers were free to decide which gap and/or stepoff they considered the maximum and then measure these before and after surgery. The observers were two trauma surgeons with more than 5 years of experience in pelvic surgery, two trauma surgeons with less than 5 years of experience in pelvic surgery, and one surgical resident. Additionally, the final intraoperative fluoroscopy images were assessed for the presence of a gap or stepoff in the weightbearing dome. All observers used the same standardized measurement technique and each observer measured the first five patients together with the responsible researcher. For 10 randomly selected patients, all measurements were repeated by all observers, at least 2 weeks after the initial measurements. The intraclass correlation coefficient (ICC) for pelvic radiographs and CT images and the kappa value for intraoperative fluoroscopy measurements were calculated to determine the inter- and intraobserver variability. Interobserver variability was defined as the difference in the measurements between observers. Intraobserver variability was defined as the difference in repeated measurements by the same observer. RESULTS: Preoperatively, the interobserver ICC was 0.4 (gap and stepoff) on radiographs and 0.4 (gap) and 0.3 (stepoff) on CT images. The observers agreed on the indication for surgery in 40% (gap) and 30% (stepoff) on pelvic radiographs. For CT scans the observers agreed in 95% (gap) and 70% (stepoff) of images. Postoperatively, the interobserver ICC was 0.4 (gap) and 0.2 (stepoff) on radiographs. The observers agreed on whether the reduction was acceptable or not in 60% (gap) and 40% (stepoff). On CT images the ICC was 0.3 (gap) and 0.4 (stepoff). The observers agreed on whether the reduction was acceptable in 35% (gap) and 38% (stepoff). The preoperative intraobserver ICC was 0.6 (gap and stepoff) on pelvic radiographs and 0.4 (gap) and 0.6 (stepoff) for CT scans. Postoperatively, the intraobserver ICC was 0.7 (gap) and 0.1 (stepoff) on pelvic radiographs. On CT the intraobserver ICC was 0.5 (gap) and 0.3 (stepoff). There was no agreement between the observers on the presence of a gap or stepoff on intraoperative fluoroscopy images (kappa -0.1 to 0.2). CONCLUSIONS: We found an insufficient interobserver and intraobserver agreement on measuring gaps and stepoffs for supporting clinical decisions in acetabular fracture surgery. If observers cannot agree on the size of the gap and stepoff, it will be challenging to decide when to perform surgery and study the results of acetabular fracture surgery. LEVEL OF EVIDENCE: Level III, diagnostic study

What is the long-term clinical outcome after fragility fractures of the pelvis? - A CT-based cross-sectional study

Background: Recently, Rommens and Hoffman introduced a CT-based classification system for fragility fractures of the pelvis (FFP). Although fracture characteristics have been described, the relationship with clinical outcome is lacking. The purpose of this study was to get insight into the type of treatment and subsequent clinical outcome after all types of FFP. Methods: A cross-sectional cohort study was performed including all elderly patients (≥ 65 years) with a CT-diagnosed FFP, between 2007-2019 in two level 1 trauma centers. Data regarding treatment, mortality and clinical outcome was gathered from the electronic patient files. Patients were asked to complete patient-reported outcome measures (PROMs) regarding physical functioning (SMFA) and quality of life (EQ-5D). Additionally, a standardized multidisciplinary treatment algorithm was constructed. Results: A total of 187 patients were diagnosed with an FFP of whom 117 patients were available for follow-up analysis and 58 patients responded. FFP type I was most common (60%), followed by type II (27%), type III (8%) and type IV (5%). Almost all injuries were treated non-operatively (98%). Mobility at six weeks ranged from 50% (type III) to 80% type II). Mortality at 1 year was respectively 16% (type I and II), 47% (type III) and 13% (type IV). Physical functioning (SMFA function index) ranged from 62 (type III and IV) to 69 (type II) and was significantly decreased (P=<0.001) compared to the age-matched general population. Quality of life was also significantly decreased, ranging from 0.26 (type III) to 0.69 (type IV). Conclusions: FFP type I and II are most common. Treatment is mainly non-operative, resulting in good mobility after six weeks, especially for patients with FFP type I and II. Mortality rates at one year were substantial in all patients. Physical functioning and quality of life was about 20-30% decreased compared to the general population

The accuracy of gap and step-off measurements in acetabular fracture treatment

The assessment of gaps and steps in acetabular fractures is challenging. Data from various imaging techniques to enable accurate quantification of acetabular fracture displacement are limited. The aim of this study was to assess the accuracy of pelvic radiographs, intraoperative fluoroscopy, and computed tomography (CT) in detecting gaps and step-offs in acetabular fractures. Sixty patients, surgically treated for acetabular fractures, were included. Five observers (5400 measurements) measured the gaps and step-offs on radiographs and CT scans. Intraoperative fluoroscopy images were reassessed for the presence of gaps and/or step-offs. Preoperatively, 25% of the gaps and 40% of the step-offs were undetected on radiographs compared to CT. Postoperatively, 52% of the gaps and 80% of the step-offs were missed on radiographs compared to CT. Radiograph analysis led to a significantly smaller gap and step-off compared to the CT measurements, an underestimation by a factor of two. Approximately 70% of the residual gaps and step-offs was not detected using intraoperative fluoroscopy. Gaps and step-offs that exceed the critical cut-off indicating worse prognosis often remained undetected on radiographs compared to CT scans. Less-experienced observers tend to overestimate gaps and step-offs compared to the more-experienced observers. In acetabular fracture treatment, gaps and step-offs were often undetected and underestimated on radiographs and intraoperative fluoroscopy in comparison with CT scans. This means that CT is superior to radiographs in detecting acetabular fracture displacement, which is clinically relevant for patient counselling regarding treatment decisions and prognosis