Persistent inter-observer variability of breast density assessment using BI-RADS® 5th edition guidelines

Due to most states' legislation, mammographic density categorization has potentially far-reaching implications, but remains subjective based on BIRADS® guidelines. We aimed to determine 1) effect of BI-RADS® 5th edition (5th-ed) vs 4th-edition (4th-ed) guidelines on reader agreement regarding density assessment; 2) 5th-ed vs 4th-ed density distribution, and visual vs quantitative assessment agreement; 3) agreement between experienced vs less experienced readers. In a retrospective review, six breast imaging radiologists (BIR) (23–30 years' experience) visually assessed density of 200 screening mammograms performed September 2012–January 2013 using 5th-ed guidelines. Results were compared to 2016 data of the same readers evaluating the same mammograms using 4th-ed guidelines after a training module. 5th-ed density categorization by seven junior BIR (1–5 years' experience) was compared to eight experienced BIR. Nelson et al.'s kappas (κm, κw), Fleiss' κF, and Cohen's κ were calculated. Quantitative density using Volpara was compared with reader assessments. Inter-reader weighted agreement using 5th-ed is moderately strong, 0.73 (κw, s.e. = 0.01), similar to 4th-ed, 0.71 (κw, s.e. = 0.03). Intra-reader Cohen's κ is 0.23–0.34, similar to 4th-ed. Binary not-dense vs dense categorization, using 5th-ed results in higher dense categorization vs 4th-ed (p < 0.001). 5th-ed density distribution results in higher numbers in categories B/C vs 4th-ed (p < 0.001). Distribution for 5th-ed does not differ based on reader experience (p = 0.09). Reader vs quantitative weighted agreement is similar (5th-ed, Cohen's κ = 0.76–0.85; 4th-ed, Cohen's κ = 0.68–0.83). There is persistent subjectivity of visually assessed mammographic density using 5th-ed guidelines; experience does not correlate with better inter-reader agreement. •Mammographic density inter-reader agreement remains variable and persistently subjective.•Using BI-RADS® 5th edition compared to the 4th edition, breast imagers assess more mammograms as dense.•No difference in breast density distribution is observed based on experience level