Challenge of prostate MRI segmentation on T2-weighted images: inter-observer variability and impact of prostate morphology

International audienc

Reference standard for the evaluation of automatic segmentation algorithms: Quantification of inter observer variability of manual delineation of prostate contour on MRI

International audienceThe purpose of this study was to investigate the relationship between inter-reader variability in manual prostate contour segmentation on magnetic resonance imaging (MRI) examinations and determine the optimal number of readers required to establish a reliable reference standard. Materials and methods: Seven radiologists with various experiences independently performed manual segmentation of the prostate contour (whole-gland [WG] and transition zone [TZ]) on 40 prostate MRI examinations obtained in 40 patients. Inter-reader variability in prostate contour delineations was estimated using standard metrics (Dice similarity coefficient [DSC], Hausdorff distance and volume-based metrics). The impact of the number of readers (from two to seven) on segmentation variability was assessed using pairwise metrics (consistency) and metrics with respect to a reference segmentation (conformity), obtained either with majority voting or simultaneous truth and performance level estimation (STAPLE) algorithm. Results: The average segmentation DSC for two readers in pairwise comparison was 0.919 for WG and 0.876 for TZ. Variability decreased with the number of readers: the interquartile ranges of the DSC were 0.076 (WG) / 0.021 (TZ) for configurations with two readers, 0.005 (WG) / 0.012 (TZ) for configurations with three readers, and 0.002 (WG) / 0.0037 (TZ) for configurations with six readers. The interquartile range decreased slightly faster between two and three readers than between three and six readers. When using consensus methods, variability often reached its minimum with three readers (with STAPLE, DSC = 0.96 [range: 0.945 −0.971] for WG and DSC = 0.94 [range: 0.912−0.957] for TZ, and interquartile range was minimal for configurations with three readers. Conclusion: The number of readers affects the inter-reader variability, in terms of inter-reader consistency and conformity to a reference. Variability is minimal for three readers, or three readers represent a tipping point in the variability evolution, with both pairwise-based metrics or metrics with respect to a reference. Accordingly, three readers may represent an optimal number to determine references for artificial intelligence applications

Prostate volume prediction on MRI: tools, accuracy and variability

The original publication is available at www.springerlink.com: https://link.springer.com/article/10.1007/s00330-022-08554-4International audienceOBJECTIVE: A reliable estimation of prostate volume (PV) is essential to prostate cancer management. Theobjective of our multi-rater study was to compare intra and inter-rater variability of PV frommanual planimetry and ellipsoid formulas.METHODS: Forty treatment-naive patients who underwent prostate MRI were selected from a localdatabase. PV and corresponding PSA density (PSAd) were estimated on 3D T2-weighted MRI(3T) by 7 independent radiologists using the traditional ellipsoid formula (TEF),the newerbiproximate ellipsoid formula (BPEF), and the manual planimetry method (MPM) used asground truth. Intra and inter-rater variability was calculated using the mixed model basedintraclass correlation coefficient (ICC).RESULTS: Mean volumes were 67.00 (±36.61), 66.07(±35.03), and 64.77(±38.27)cm 3 with the TEF,BPEF, and MPM methods respectively. Both TEF and BPEF overestimated PV relative toMPM, with the former presenting significant differences (+1.91cm3, IQ=[-0.33cm3, 5.07cm3],p-val=0.03). Both intra (ICC>0.90) and inter-rater (ICC>0.90) reproducibility were excellent. MPM had thehighest inter-rater reproducibility (ICC=0.999). Inter-rater PV variation led to discrepancies inclassification according to the clinical criterion of PSAd>0.15ng/mL for 2 patients (5%), 7patients (17.5%), and 9 patients (22.5%) when using MPM, TEF, and BPEF respectively.CONCLUSION: PV measurements using ellipsoid formulas and MPM are highly reproducible. MPM is a robustmethod for PV assessment and PSAd calculation, with the lowest variability. TEF showed ahigh degree of concordance with MPM but a slight overestimation of PV. Precise anatomiclandmarks as defined with the BPEF led to a more accurate PV estimation, but also to a highervariability

Correction to: Prostate Volume Prediction on MRI: Tools, Accuracy and Variability

International audienceA reliable estimation of prostate volume (PV) is essential to prostate cancer management. The objective of our multi-rater study was to compare intra- and inter-rater variability of PV from manual planimetry and ellipsoid formulas