Almutairi, Fawaz

Callis, Samuel

Chatzitolios, Anastasios

Cornelia, Szecsei

Iczkowski, Kenneth

Kravtsov, Oleksandr

Lenon, Maria Sarah

Lucia, M Scott

Manucha, Varsha

Simko, Jeffry

Tangen, Cathy

Thompson, Ian

Yeh, Yunshin

Zhang, Lian

English

eScholarship - University of California

UCSFUC San Francisco Previously Published WorksTitle818 Digital Whole-Slide Inter-Observer Concordance of Contemporary Prostatic Biopsy GradingPermalinkhttps://escholarship.org/uc/item/7sj2d2cwJournalLaboratory Investigation, 105(3)ISSN0023-6837AuthorsIczkowski, KennethCallis, SamuelAlmutairi, Fawazet al.Publication Date2025-03-01DOI10.1016/j.labinv.2024.103051 Peer reviewedeScholarship.org Powered by the California Digital LibraryUniversity of CaliforniaDigital Whole-Slide Inter-Observer Concordance of Contemporary Prostatic Biopsy Grading1Kenneth A. Iczkowski, 2Sam Callis, 3Jeffry Simko, 4Fawaz Almutairi,5Anastasios Chatzitolios, 6Oleksandr Kravtsov, 7Sarah Lenon, 8Varsha Manchua, 5Cornelia Szecsei, 9Yunshin Albert Yeh,  10Lian Zhang, 11Ian Thompson, 2Cathy Tangen, 10M. Scott LuciaIntroductionThe future of pathology is digital. Prior consensus studies on modern Gleason grade group (GG) diagnosis were limited to photomicrographs, ignoring tumor heterogeneity and multifocality.  The critical GG 1 vs. 2 threshold may dictate active surveillance versus immediate treatment (PMID: 35780705). A related concern is subjective discernment of poorly formed glands (PMID: 26099009) among Gleason 4 patterns.  This scanned slide-based, multi-institutional concordance study is, to the best of our knowledge, the first of its kind. We organized eleven (11) evaluators of different skill levels to evaluate needle biopsies with cancer.Raters were practicing fellowship-trained urologic pathologists with: 3-5 years’ independent experience (3); 5-15 y (5); >15 y (3). The metadata for 120 whole slide images from separate prostatic needle biopsy cases spanning all GG, were independently entered by raters on the Proscia Concentriq platform. Raters needed to provide a GG, specify the one main Gleason 4 pattern, the % Gleason 4, and measure tumor length (mm) with a green automated measuring spline. Consensus was defined as agreement by 7 of 11 raters (64%) on GG for a given case.  Model-based Kappa statistics were calculated (PMID: 36038846) with R software.Consensus was achieved in 85/120 (71%) cases. Participants worked mostly from both home and work. Monitors ranged from 10.25 – 50 inches. The majority reported that telepathology was slower than glass slides, but efficiency improved over time. The overall experience was rated 3.2 (1=better than glass and 5= worse than glass).  The weighted Kappa was 0.34 for paired inter-rater agreement (95% CI 0.28, 0.39). MethodsResultsResultsRater score grouped by consensus GG is shown (Figure). Agreement was highest for cases graded as consensus GG1 with a pairwise percent agreement of 73.2%. GG4 stood out for showing the lowest concordance rate. Consensus GG1 cases (CG1) were rated Group 2 for 14% of all the scores given. Among those rated as Consensus GG2 (CG2), the percent of pattern 4 was ≤10% in 76% and was 1%-5% in 53%. Consensus GG2 cases had 11% of their ratings called Group 1.     Among 912 ratings of four Gleason 4 morphologies, 282 were non-consensus ratings.  The 282 non-consensus ratings differed by pattern: most frequent morphology was poorly formed glands, 131/353 total ratings (34.0%), then fused glands 109/367 (28.3%), large cribriform 23/140 (6.0%), and small cribriform 52/282 (4.9%). Regrouping: Since GG1 versus GG2 often determines eligibility for active surveillance, attention was turned to ways to improve agreement at this threshold. We postulated that low risk be defined as combining GG1 with cases with ≤10% Gleason 4; intermediate risk was GG2 with >10% Gleason 4 plus GG3, and high risk was GG4 and GG5. Under this 3-tiered scheme, consensus in the first look at the images rose to 108/120 or 90% (from 71%). Paired inter-rater agreement improved to 0.476 (from 0.34).• When interobserver variability for the GG 1 vs. 2 threshold was problematic, 86% of cases involved amounts ≤10% of Gleason 4. The pattern in such cases was frequently uncertain.  Regrouping of risk groups into 3 tiers could improve consensus from 71% to 90%.• Poorly-formed Gleason 4 morphology was notably problematic for consensus. • GG 4 showing the lowest consensus was also understandable since it may be difficult to determine whether a minute amount of Gleason 3 glands or Gleason 5 cells (or necrosis) is present. • These thresholds warrant second opinions and deserve extra attention for both human and AI discernment.ConclusionsNon-concordance was defined by grade group, not pattern so p-value could not be calculated for pattern recognition. Missing and superfluous pattern data arose in many cases. LimitationsUniversity of Colorado, Aurora, CODiscrepancies: Of 34 non-consensus cases, 25 had ≥9 raters who agreed with either 1 of 2 adjacent Grade groups: 14 cases of GG1 versus GG2, and 11 cases of GG2 versus GG3. We speculated that a low % of Gleason 4 would explain the GG1 vs. 2 threshold and % poorly-formed glands would predominate at both thresholds (Figures).00.10.20.30.40.50.60.70.80.9Consensus G1 Consensus G2 Consensus G3 Consensus G4 Consensus G5Consensus Images (85) Stratified by Grade Group: Distribution of % Rater ScoresGroup 1 Group 2 Group 3 Group 4 Group 5CG1: 25 images,268 scoresCG2: 28 images,308 scoresCG3: 17 images,186 scoresCG4: 5 images,55 scoresCG5: 10 images,110 scoresDepartments of Pathology, 1University of California - Davis, Sacramento, CA, 2Fred Hutchinson Cancer Research Center, Seattle, WA, 3University of California - San Francisco, 4King Abdulaziz University, Jeddah, Saudi Arabia, 5North Bristol NHS Trust, Bristol, England, 6Upstate Health System, Syracuse, NY, 7University of Southern California, Los Angeles, CA, 8University of Mississippi, Jackson, MS, 9Overton Brooks VA Medical Center, Shreveport, LA, 10University of Colorado Anschutz Medical Campus, Aurora, CO, 11Christus Health, Irving, TX6 votes for GG1, 5 for GG2. Few far-left glands are problematic for “fusion”6 votes for GG1, 5 for GG2. Are there poorly-formed glands?4 votes for GG2, 6 for GG3. Is the % of high-grade glands greater than or less than 50%?GG1 vs. GG2 threshold: In 14 cases, 72 total votes were for GG2. The %4 was given as 1-5% or 6-10% in 62/72 (86%) instances. Patterns: Poorly-formed and fused glands predominated at both GG1 vs. 2 and GG2 vs. 3 thresholds. The GG1 vs. 2 threshold had much higher uncertainty (N/A) about pattern than did the GG2 vs. 3: GG1 vs. 2 threshold pattern responses:GG2 vs. 3  threshold pattern responses:Fused GlandsLarge Cribriform GlandsPoorly Formed GlandsSmall Cribriform GlandsUncertain, N/AGrand total48 3 52 10 8 12148/121=40% 3/121=2% 52/121=43% 10/121=8% 8/121=7%Fused GlandsLarge Cribriform GlandsPoorly Formed GlandsSmall Cribriform GlandsUncertain, N/AGrand total31 3 37 3 80 15431/154=20% 3/154=2% 37/154=24% 3/154=2% 80/154=52%

818 Digital Whole-Slide Inter-Observer Concordance of Contemporary Prostatic Biopsy Grading

https://escholarship.org/content/qt7sj2d2cw/qt7sj2d2cw.pdf

818 Digital Whole-Slide Inter-Observer Concordance of Contemporary Prostatic Biopsy Grading

Abstract

Similar works

Full text

Available Versions

eScholarship - University of California