Training improves interobserver reliability for the diagnosis of scaphoid fracture displacement

Background The diagnosis of displacement in scaphoid fractures is notorious for poor interobserver reliability.Questions/purposes We tested whether training can improve interobserver reliability and sensitivity, specificity, and accuracy for the diagnosis of scaphoid fracture displacement on radiographs and CT scans.Methods Sixty-four orthopaedic surgeons rated a set of radiographs and CT scans of 10 displaced and 10 nondisplaced scaphoid fractures for the presence of displacement, using a web-based rating application. Before rating, observers were randomized to a training group (34 observers) and a nontraining group (30 observers). The training group received an online training module before the rating session, and the nontraining group did not. Interobserver reliability for training and nontraining was assessed by Siegel&rsquo;s multirater kappa and the Z-test was used to test for significance.Results There was a small, but significant difference in the interobserver reliability for displacement ratings in favor of the training group compared with the nontraining group. Ratings of radiographs and CT scans combined resulted in moderate agreement for both groups. The average sensitivity, specificity, and accuracy of diagnosing displacement of scaphoid fractures were, respectively, 83%, 85%, and 84% for the nontraining group and 87%, 86%, and 87% for the training group. Assuming a 5% prevalence of fracture displacement, the positive predictive value was 0.23 in the nontraining group and 0.25 in the training group. The negative predictive value was 0.99 in both groups.Conclusions Our results suggest training can improve interobserver reliability and sensitivity, specificity and accuracy for the diagnosis of scaphoid fracture displacement, but the improvements are slight. These findings are encouraging for future research regarding interobserver variation and how to reduce it further.<br /

Diagnosis of elbow fracture patterns on radiographs: interobserver reliability and diagnostic accuracy

Item does not contain fulltextBACKGROUND: Studies of traumatic elbow instability suggest that recognition of a pattern in the combination and character of the fractures and joint displacements helps predict soft tissue injury and guide the treatment of traumatic elbow instability, but there is no evidence that patterns can be identified reliably. QUESTIONS/PURPOSES: We therefore determined (1) the interobserver reliability of identifying specific patterns of traumatic elbow instability on radiographs for subgroups of orthopaedic surgeons; and (2) the diagnostic accuracy of radiographic diagnosis. METHODS: Seventy-three orthopaedic surgeons evaluated 53 sets of radiographs and diagnosed one of five common patterns of traumatic elbow instability by using a web-based survey. The interobserver reliability was analyzed using Cohen's multirater kappa. Intraoperative diagnosis was the reference for fracture pattern in calculations of the sensitivity, specificity, accuracy, and positive predictive and negative predictive values of radiographic diagnosis. RESULTS: The overall interobserver reliability for patterns of traumatic elbow instability on radiographs was kappa=0.41. Treatment of greater than five such injuries a year was associated with greater interobserver agreement, but years in practice were not. Diagnostic accuracy ranged from 76% to 93% and was lowest for the terrible triad pattern of injury. CONCLUSIONS: Specific patterns of traumatic elbow instability can be diagnosed with moderate interobserver reliability and reasonable accuracy on radiographs. LEVEL OF EVIDENCE: Level III, diagnostic study. See Guidelines for Authors for a complete description of levels of evidence