Abstract. Despite the prognostic importance of mitotic count as one of the components of the Bloom -Richardson grad

Akira Saito

Atsushi Kurata

Christopher Malon

Elena Brachtel

Eric Cosatto

Hans Peter Graf

John S Meyer

Masahiko Kuroda

Shulin Wu

Yukako Yagi

CiteSeerX

Analytical Cellular Pathology 35 (2012) 97–100
DOI 10.3233/ACP-2011-0029
IOS Press
97
Mitotic figure recognition: Agreement among
pathologists and computerized detector
Christopher Malona,∗, Elena Brachtelb, Eric Cosattoa, Hans Peter Grafa, Atsushi Kuratac,
Masahiko Kurodac, John S. Meyerd, Akira Saitoe, Shulin Wub and Yukako Yagib
aDepartment of Machine Learning, NEC Laboratories America, NJ, USA
bDepartment of Pathology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
cDepartment of Molecular Pathology, Tokyo Medical University, Tokyo, Japan
dDepartment of Pathology, St. Luke’s Hospital (St. Louis), Chesterfield, MO, USA
eInnovative Service Solutions Division, NEC Corporation, Tokyo, Japan
Abstract. Despite the prognostic importance of mitotic count as one of the components of the Bloom – Richardson grade [3],
several studies [2, 9, 10] have found that pathologists’ agreement on the mitotic grade is fairly modest. Collecting a set of more
than 4,200 candidate mitotic figures, we evaluate pathologists’ agreement on individual figures, and train a computerized system
for mitosis detection, comparing its performance to the classifications of three pathologists. The system’s and the pathologists’
classifications are based on evaluation of digital micrographs of hematoxylin and eosin stained breast tissue. On figures where
the majority of pathologists agree on a classification, we compare the performance of the trained system to that of the individual
pathologists. We find that the level of agreement of the pathologists ranges from slight to moderate, with strong biases, and
that the system performs competitively in rating the ground truth set. This study is a step towards automatic mitosis count to
accelerate a pathologist’s work and improve reproducibility.
1. Introduction
Beginning with Greenough’s 1925 grading system
[7], pathologists have attempted to quantify factors that
provide a measure of an invasive breast tumor’s locality
and prognosis. From the seven factors in Greenough’s
system, Bloom and Richardson settled on three fac-
tors [3], and the 1991 Nottingham revisions gave more
stable definitions to these variables [6]. The grades
are widely used to inform the selection of high-risk
treatments, through the information they provide about
survival or the likelihood of distant metastasis.
∗Corresponding author: Christopher Malon, PhD, Department of
Machine Learning, NEC Laboratories America, 4 Independence
Way, Princeton, NJ 08540, USA. Tel.: +1 609 951 2594; Fax: +1
609 951 2482; E-mail: malon@nec-labs.com.
In light of the critical role of grading, many authors
have investigated agreement between pathologists on
individual components of the grade [2, 9, 10]. The
level of agreement may be reported in Cohen’s Kappa
statistic [5], which equals one in case of perfect
agreement and zero in the case of probabililstically
independent decisions. The range 0–0.2 is often con-
sidered as slight agreement, 0.2–0.4 as fair, 0.4–0.6 as
moderate, 0.6–0.8 as good, and 0.8–1 as almost perfect.
Meyer’s study of agreement on Bloom-Richardson
grading [9] involved groups of five to seven patholo-
gists, with each group examining 10–23 patients’ slides
of hematoxylin and eosin stained biopsy tissue through
an analog microscope. Agreement on the overall grade
was moderate, with κ = 0.50–0.59 for the various
groups. The tubularity grade achieved stronger agree-
ment than for the other components (κ = 0.57–0.83).
2210-7177/12/$27.50 © 2012 – IOS Press and the authors. All rights reserved
98 C. Malon et al. / Mitotic figure recognition
Agreement on the pleomorphism component was
weakest (κ = 0.27–0.50). The range of agreement for
the mitotic grade was κ = 0.45–0.67.
The NEC e-Pathologist Project aims to provide diag-
nosis support for anatomical pathology, by providing
computerized image analysis for virtual microscopy. It
has released modules for analysis of gastric tissue that
are in use in a major commercial laboratory in Japan. A
breast module for hematoxylin and eosin stained tissue
is under development, to target the least two stable parts
of the Bloom-Richardson grade. For pleomorphism,
the system aims to produce output representing care-
fully measured statistics of nuclear shape. For mitosis,
the system aims to provide a classifier with consistent
judgments.
Usage of a computerized mitosis detector could
be the continuation of a line of efforts to devise
more stable, prognostically useful guidelines for mito-
sis grading. The original Bloom-Richardson system
directed that hyperchromatic nuclei be counted along
with true mitotic figures. Cutoffs between grades
were vaguely prescribed. In the Nottingham revision,
pathologists were directed instead to avoid hyper-
chromatic nuclei, apoptotic figures, and pyknosis.
Mitotic figures in prophase were no longer to be
counted because agreement was low. Figures were to
be counted in ten high power fields (25× or 40× mag-
nification), and grading cutoffs were made to depend
on the high power field size. Each high power field was
to be taken from the tumor’s periphery.
The rigor of the grade given in clinical practice
appears to vary widely. Comparing pathologists who
performed a quick 30-second impression of ten high
power fields to those who spent 2–3 minutes apply-
ing the rules for the WHO Mitotic Activity Index
(MAI), Skaland [11] found that those who followed the
MAI procedure gave grades that were prognostic with
two more orders of magnitude in p-value than those
who did the quick impression. Some see the need for
even more concrete guidelines. Baak [1, 12] attempts
to describe figures to be counted in image analysis
terms (loss of nuclear membrane, presence of “clear,
hairy extensions of nuclear material,” etc.,). One of the
authors (AK) believes a complete decision tree of such
image analysis rules could be drawn. However, another
author (EB) believes that it may be impossible to apply
even such specific guidelines by viewing just one focal
plane, because some details of mitotic figures are rec-
ognizable only when focusing up and down with the
microscope.
Whereas most previous investigations of agreement
of mitosis have focused on the mitotic grade, we exam-
ine agreement on individual figures. In the largest
previous such study we found [9], seven pathologists
examined 43 potential mitotic figures. Average pair-
wise agreement was κ = 0.38.
2. Methods
In the present study, we asked three of the authors
(AK, JM, and SW) who are pathologists actively sign-
ing out breast cases, to examine 4,204 potential mitotic
figures. These figures were taken from 2,444 high
power fields in 94 breast slides, stained in hematoxylin
and eosin. Tissues were provided by Massachusetts
General Hospital and Tokyo Medical University, and
scanned on Hamamatsu Nanozoomer scanners. At full
resolution (40×) the scanners afforded a resolution of
4.39 pixels per micron. Because a random selection
of nuclei would include too many obvious non-mitotic
figures, one author selected the candidates manually,
intending to obtain figures that were mitotic or worth
a closer look.
Each pathologist examined all 4,204 figures,
answering the question “Is this figure mitosis or not?”
with “Yes,” “No,” or “Maybe”. For each pair of
pathologists, we considered the figures where both
pathologists committed to a “Yes” or a “No,” and com-
puted Cohen’s Kappa on this subset. In this way, figures
where the digital image was inadequate for decision
should be excluded.
The second part of our investigation concerned the
performance of a computerized detector. As discussed
above, the description of mitotic figures in terms of
decisions about shape and structure may be complex
and unclear. Translating those verbal directives into
manually coded image analysis rules could be danger-
ous. Avoiding such a heuristic approach, we developed
a detector based on machine learning.
In our system, a simple rule that extracts blobs repre-
senting nuclei of poss-sible mitotic figures (and many
other nuclei, to be rejected later) establishes a set of
candidates from the digital micrograph of the high
power field (HPF). Machine learning is applied in three
phases. One phase applies a support vector regression
[13], which remaps the color palette of the HPF to
normalized values. The next phase is a convolutional
neural network [8], applied at each extracted blob. The
convolutional neural network contributes a feature to
C. Malon et al. / Mitotic figure recognition 99
a feature vector, which also contains many other mea-
surements regarding the shape, color, mass, and texture
of the blob and its neighborhood. In the final phase, a
support vector machine [13] uses the feature vector to
classify the area around the blob as a mitotic figure or
not.
Machine learning algorithms learn classification
rules by taking a set of training data where fea-
tures can be measured and true classifications for each
example are known. Using the training data, they find
parameters to be used for classification by solving a
minimization problem. The learned parameters can
then be applied to new examples, where the features
can be measured but the classification is not given, to
predict the correct classification.
Applying machine learning thus requires that we
divide our data into a set for training and a set for test-
ing. We reserved 799 mitotic figures for testing and
used the remainder for developing our algorithm. Fig-
ures for testing and figures for training never came
from the same slides. This careful separation ensures
that our test figures reflect tissues totally unseen during
development.
Training the machine also requires that we assign
one ground truth classification to each figure in the
training set, but we have three classifications–one from
each pathologist. We aggregated the pathologists’ deci-
sions using majority voting. Namely, figures with two
or three “Yes” labels were taken as ground truth pos-
itive, and figures with two or three “No” labels were
taken as ground truth negative. Figures for which there
were neither two “Yes” verdicts nor two “No” verdicts,
such as a set of labels “Yes,” “Maybe,” and “Maybe,”
were excluded from the training set. The same proce-
dure was used on the testing data set.
We find that this protocol is useful for investigation
purposes because it allows a fair comparison of the
machine to pathologists. If the only figures in the data
set were those for which all three pathologists agreed,
there would be no fair baseline for the machine’s
performance (trivially, each participating pathologist
would perfectly predict all the ground truth labels).
3. Results
Table 1 shows pairwise agreement of our three
pathologists, in cases where both committed to “Yes”
or “No” decisions. Statistics of agreement are com-
Table 1
Pathologist agreement of mitotic figures
Pathologists Yes/Yes No/Yes Yes/No No/No
A/B 1352 4 789 102
A/C 2705 20 172 83
B/C 1506 756 15 461
Table 2
Statistics of pathologist agreement of mitotic figures
Pathologists Cohen’s κ Prevalence Bias
A/B 0.13 0.56 −0.35
A/C 0.44 0.88 −0.05
B/C 0.39 0.38 0.27
puted in Table 2. The prevalence index and the bias
index express attributes of agreement that do not affect
Cohen’s Kappa [4]. The prevalence index describes the
relative frequency of agreed positives versus agreed
negatives. The bias index compares the frequency of
positive/negative disagreements and negative/positive
disagreements.
We find that strong biases exist between pairs of
observers. B is much more likely to reject a figure
called mitotic by C or A than to count a figure rejected
by C or A. A measurable but much less significant bias
exists between C and A: C is more likely to reject a
figure counted by A than A is likely to reject a figure
counted by C.
In Table 3, we examine how each pathologist’s vote
predicted the ground truth labels in the test set, which
were determined by the result of majority voting. This
calculation is intended as a baseline for the binary clas-
sifier trained by machine learning, which decides “Yes”
or “No” for every candidate mitotic figure. The pathol-
ogists had the additional option of saying “Maybe” to a
figure, so their binary classification performance may
be considered as a range, bounded by the performance
levels if all “Maybe” decisions were changed to either
“Yes” or “No”.
Observer C’s performance at first looks surpris-
ingly high, but all observers were advantaged over the
machine by contributing a vote to the “majority label.”
Recall from Table 1 that A and B disagreed on 35%
of the cases where they both committed to “Yes” or
“No”. In each of these cases, C’s vote actually defined
the majority label. As the tie-breaker in this protocol,
it is naturally expected that he has high agreement with
the majority label.
100 C. Malon et al. / Mitotic figure recognition
Table 3
Prediction of test figures where two pathologists agreed
Observer Majority label Observer Lower bound Upper bound
Yes/Maybe/No Agreement agreement
A Positive (726) 658/65/3 90.6% 99.6%
Negative (73) 15/36/22 30.1% 79.4%
B Positive (726) 394/166/166 54.3% 77.1%
Negative (73) 0/0/73 100.0% 100.0%
C Positive (726) 720/4/2 99.2% 99.7%
Negative (73) 2/2/69 94.5% 97.3%
Machine Positive (726) 462/0/264 63.6% 63.6%
Negative (73) 1/0/72 98.6% 98.6%
The last line of Table 3 shows the machine’s perfor-
mance. Compared to A, the machine performs much
more strongly on negatives, but not as well on positives.
Compared to B, the machine misclassifies only one
more negative figure, while falling within the bounds
of B’s performance on positive figures.
We find this result encouraging, considering sev-
eral disadvantages of the machine. One disadvantage
is that the machine was confined to looking at a small
box around each mitotic figure, whereas the patholo-
gist could examine the entire HPF. Another is the lack
of a special strategy for telophase figures. Each blob in
a telophase figure is regarded separately (although the
second blob is visible). Features considering the rela-
tionship between the two nearby blobs should improve
the performance further.
Although the range of Cohen’s Kappa for patholo-
gists on individual mitotic figure recognition is perhaps
not surprising, given its range on grade-level agree-
ment, we were surprised to find strong biases. The
biases suggest that different pathologists interpret
grading guidelines differently. We hope that the devel-
opment of a computerized mitotic detector may be one
step towards the establishment of more stable tissue
grading.
References
[1] J. Baak, Mitosis counting in tumors, Human Pathology 21(7)
(1990), 683–685.
[2] J.P.A. Baak, E. Gudlaugsson, I. Skaland, L.H.R. Guo, J. Klos,
T.H. Lende, H. Soiland, E.A.M. Janssen and A. zur Hausen,
Proliferation is the strongest prognosticator in node-negative
breast cancer: Significance, error sources, alternatives, and
comparison with molecular prognostic markers, Breast Cancer
Res Treat 115 (2009), 241–254.
[3] H.J. Bloom and W.W. Richardson, Histological grading and
prognosis in breast cancer; a study of 1409 cases of which
359 have been followed for 15 years, Br J Cancer 11 (1957),
359–377.
[4] T. Byrt, J. Bishop and J. B. Carlin, Bias, prevalence and kappa,
Journal of Clinical Epidemiology 46(5) (1993), 423–429.
[5] J. Cohen, A coefficient of agreement for nominal scales,
Educational and Psychological Measurement 20(1) (1960),
37–46.
[6] C.W. Elston and I.O. Ellis, Pathological prognostic factors in
breast cancer, i. the value of histological grade in breast can-
cer: Experience from a large study with long-term follow-up,
Histopathology 19(5) (1991), 403–410.
[7] R.B. Greenough, Varying degrees of malignancy in cancer of
the breast, J Cancer Res 9 (1925), 425–463.
[8] Y. Le Cun, L. Bottou, Y. Bengio and P. Haffner, Gradient-based
learning applied to document recognition, Proceedings of the
IEEE 86(11) (1998), 2278–2324.
[9] J. S. Meyer, C. Alvarez, C. Milikowski, N. Olson, I. Russo, J.
Russo, A. Glass, B.A. Zehnbauer, K. Lister and R. Parwaresch,
Breast carcinoma malignancy grading by Bloom-Richardson
system vs proliferation index: Reproducibility of grade and
advantages of proliferation index, Modern Pathology 18
(2005), 1067–1078.
[10] P. Robbins, S. Pinder, N. de Klerk, H. Dawkins, J. Harvey, G.
Sterrett, I. Ellis and C. Elston, Histological grading of breast
carcinomas: A study of interobserver agreement, Hum Pathol
26(8) (1995), 873–879.
[11] I. Skaland, P.J. van Diest, E.A.M. Janssen, E. Gudlaugs-
son and J.P.A. Baak, Prognostic differences of world health
organization-assessed mitotic activity index and mitotic
impression by quick scanning in invasive ductal breast can-
cer patients younger than 55 years, Hum Pathol 39(4) (2008),
584–590.
[12] P. van Diest, J. Baak, P. Matze-Cok, E. Wisse-Brekelmans,
C. van Galen, P. Kurver, S. Bellot, J. Fijnheer, L. van Gorp,
W. Kwee, J. Los, J. Pe-terse, H. Ruitenberg, R. Schapers,
M. Schipper, J. Somsen, A. Willig and A. Ariens. Repro-
ducibility of mitosis counting in 2,469 breast cancer specimens:
Results from the multicenter morphometric mammary carci-
noma project, Human Pathology 23(6) (1992), 603–607.
[13] V. Vapnik, Statistical learning theory, Wiley (1998).
Submit your manuscripts at
http://www.hindawi.com
Stem Cells
International
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
MEDIATORS
INFLAMMATION
of
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
Behavioural 
Neurology
Endocrinology
International Journal of
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
Disease Markers
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
BioMed 
Research International
Oncology
Journal of
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
Oxidative Medicine and 
Cellular Longevity
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
PPAR Research
The Scientific 
World Journal
Hindawi Publishing Corporation 
http://www.hindawi.com Volume 2014
Immunology Research
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
Journal of
Obesity
Journal of
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
 Computational and  
Mathematical Methods 
in Medicine
Ophthalmology
Journal of
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
Diabetes Research
Journal of
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
Research and Treatment
AIDS
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
Gastroenterology 
Research and Practice
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
Parkinson’s 
Disease
Evidence-Based 
Complementary and 
Alternative Medicine
Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com


Mitotic figure recognition: Agreement among pathologists and computerized detector

http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=4E33370EAC8766C6C98923CBD00A91DA?doi=10.1.1.1046.3170&rep=rep1&type=pdf

Mitotic figure recognition: Agreement among pathologists and computerized detector

Abstract

Similar works

Full text

Available Versions

CiteSeerX