How Reliable Is Ki-67 Immunohistochemistry in Grade 2 Breast Carcinomas? A QA Study of the Swiss Working Group of Breast- and Gynecopathologists

Adjuvant chemotherapy decisions in breast cancer are increasingly based on the pathologist's assessment of tumor proliferation. The Swiss Working Group of Gyneco- and Breast Pathologists has surveyed inter- and intraobserver consistency of Ki-67-based proliferative fraction in breast carcinomas. Methods Five pathologists evaluated MIB-1-labeling index (LI) in ten breast carcinomas (G1, G2, G3) by counting and eyeballing. In the same way, 15 pathologists all over Switzerland then assessed MIB-1-LI on three G2 carcinomas, in self-selected or pre-defined areas of the tumors, comparing centrally immunostained slides with slides immunostained in the different laboratoires. To study intra-observer variability, the same tumors were re-examined 4 months later. Results The Kappa values for the first series of ten carcinomas of various degrees of differentiation showed good to very good agreement for MIB-1-LI (Kappa 0.56–0.72). However, we found very high inter-observer variabilities (Kappa 0.04–0.14) in the read-outs of the G2 carcinomas. It was not possible to explain the inconsistencies exclusively by any of the following factors: (i) pathologists' divergent definitions of what counts as a positive nucleus (ii) the mode of assessment (counting vs. eyeballing), (iii) immunostaining technique, and (iv) the selection of the tumor area in which to count. Despite intensive confrontation of all participating pathologists with the problem, inter-observer agreement did not improve when the same slides were re-examined 4 months later (Kappa 0.01–0.04) and intra-observer agreement was likewise poor (Kappa 0.00–0.35). Conclusion Assessment of mid-range Ki-67-LI suffers from high inter- and intra-observer variability. Oncologists should be aware of this caveat when using Ki-67-LI as a basis for treatment decisions in moderately differentiated breast carcinomas

Intra-observer variabilities of MIB-1-LI results.

<p>Shown are in each panel the MIB-1-LI results for each pathologist during step 2 of the study (left end of each line, one line representing the values obtained by one pathologist) and during step 3, 4 months later (right end of each line). The grey zone delineated the 8% and 15% cut-offs used for calculaing Kappa values. The three rows represent the results obtained for the three different carcinomas (cases 1–3). The left two columns of panels show the results obtained by reading MIB-1-LI in areas that the pathologists selected themselves (self-selected) and the panels in the two right columns the values assessed in pre-defined areas (pre-defined). In each pair of columns, values were assessed by eyeballing (left) and by counting (right). As expected, Kappa values for intraobserver variability were slightly better in pre-defined areas than in self-selected areas, and, in analogy to the interobserver variability studies, better in eyeballed than in counted MIB-1-LI read-outs.</p

Absence of linear correlations between the MIB-1 counts obtained in the 6 high power fields (figure 1) and the MIB-1-LI results obtained on the three moderately differentiated carcinomas (figure 4).

<p>The linear correlations were calculated separately for data obtained by eyeballing (upper panels, A,C) and by counting (lower panels, B,D) and for data obtained on centrally immunostained slides (left panels, A,B) and on locally immunostained slides (right panels, C,D). This analysis was done in order to test the hypothesis that under-raters (i.e. pathologists 4,5, and 6 in <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0037379#pone-0037379-g004" target="_blank"><b>figure 4</b></a>) considered only very intensely immunostained nuclei as positive and vice versa. However, this hypothesis was proven wrong: we found no significant linear correlations between the MIB-1-LI results on the three carcinomas and the number of MIB-1 positive nuclei counted by the same 15 pathologists in the 6 panel plate.</p

Variability of MIB-1-LI results (ten carcinomas, five pathologists): MIB-1-LI data obtained by five pathologists on immunostained slides of five moderately differentiated breast carcinomas (G2, red circles) and five G1 and G3 carcinomas (yellow circles), once assessed by counting the number of MIB-1 positive nuclei among 2000 tumor nuclei (left graph) and once by eyeballing the LI (right graph).

<p>Mean values over the ten carcinomas are shown in horizontal bars and standard deviations in vertical bars. A. local vs central immunostains: shown are fractions of MIB-1-LI results on locally immunostained slides over the results obtained on centrally immunostained slides. The observers 3 and 5 significantly underrated MIB-1-LI on the slides immunostained in their own laboratory when compared to the central laboratory (**P<0⋅01 Wilcoxon). B,C. individual pathologist vs group mean: shown are fractions of MIB-1-LI results by individual pathologists over the mean value calculated for the group of fibve pathologists for each individual carcinoma. Note that the observer 1 tended to significantly underrate MIB-1-LI when compared to the mean values obtained by the entire group of five pathologists (*P<0⋅05 and **P<0⋅01 Wilcoxon). Also note that the deviations from the group mean values tended to be smaller for the eyeballed than for the counted data.</p

Kappa values for rating scores.

<p>Kappa values for rating scores.</p

Laboratory protocols for Ki-67 immunostains in the 12 participating pathology institutes.

<p>Laboratory protocols for Ki-67 immunostains in the 12 participating pathology institutes.</p

How does a MIB-1 positive nucleus look like? MIB-1-positive cells counted by the 15 pathologists on the 6 high power fields shown in figure 1A.

<p>The 16th bar in red shows the counts provided by Pr. Giuseppe Viale who helped us standardize which nuclei should be considered MIB-1-positive in the right panel of <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0037379#pone-0037379-g001" target="_blank"><b>figure 1 (1B)</b></a>. Note that several pathologists counted only those nuclei that were intensely immunostained (pathologists 8,11,14), while most included in their count also most faintly immunostained nuclei. Note that the results were very homogeneous for the 6 different high power fields.</p

MIB-1-LI results of ten breast carcinomas read by five pathologists (represented by lines in different colors): The left panels (A,B) depict the results obtained in centrally immunostained slides and the right panels (C,D) the results on locally immunostained slides.

<p>The upper panels (A,C) are eyeballed and the lower panels (B,D) counted data. Despite marked variability between the five observers, MIB-1-LI results were all below 8% for the two G1 carcinomas and above 30% for the three G3 carcinomas. Note that MIB-1-LI results varied considerably for the five moderately differentiated carcinomas. Shown in grey is the zone deliminated by the 8% and 15% cut-offs used for calculating the kappa scores for interobserver correlations.</p

Variability of MIB-1-LI results (three carcinomas, 15 pathologists): MIB-1-LI data obtained by 15 pathologists on immunostained slides of three moderately differentiated breast carcinomas, assessed either by eyeballing the LI or by counting the number of MIB-1 positive nuclei among 500 tumor nuclei.

<p>A–D Interobserver variabilities of MIB-1-LI results: Shown are the fraction of MIB-1-LI results by individual pathologists over the mean value calculated for the entire group of 15 pathologists for this same carcinoma. Red and yellow circles indicate results obtained in self- selected and predefined areas, respectively. Mean values are shown in horizontal bars. Shown are the results obtained on centrally immunostained slides (A,C) and on locally immunostained slides (B,D). Several pathologists systematically under-rated (pathologists 4,5 and 6) or over-rated MIB-1-LI (pathologists 8,14) when compared to the mean values obtained by the entire group of 15 pathologists (*P<0⋅05 and **P<0⋅01 Wilcoxon). Note that for the entire group of 15 pathologists, the standard deviations around the mean values were smaller when MIB-1-LI were eyeballed (A,B, SD = 21⋅7 and 24⋅2) as compared to the data that were obtained by counting MIB-1 postive nuclei (C,D, SD = 34⋅5 and 30⋅6). E,G impact of immunostaining technique: shown are for each carcinoma the fraction of MIB-1-LI data on locally immunostained slides over those read on centrally immunostained slides. Red and yellow circles indicate results obtained in self- selected and predefined areas, respectively. Mean values are shown in horizontal bars. The observers 1 and 2 significantly under-rated MIB-1-LI on the slides immunostained in their own laboratory when compared to the central laboratory, suggesting that their immunostaining techniques yielded fainter results than the one of the central laboratory (*P<0⋅05 and **P<0⋅01 Wilcoxon). Note that for the entire group of 15 pathologists, the standard deviations around the mean values were slightly smaller when MIB-1-LI were eyeballed (E, SD = 21⋅4) as compared to the data that were obtained by counting MIB-1 positive nuclei among 500 tumor cells (G, SD = 25⋅3). F,H impact of the tumor area: shown are for each carcinoma the fraction of MIB-1-LI read-outs in areas that each pathologist had selected his/herself over read-outs repeated in areas that had been pre-selected by the principal investigator. Red and yellow circles indicate results obtained in centrally and locally immunostained slides, respectively. Mean values are shown in horizontal bars. For four of the 15 pathologists, there was a near perfect match between self-selected and pre-defined areas within the tumor. Overall, there was a very small variability for the entire group of 15 pathologists.</p