A comprehensive multi-domain dataset for mitotic figure detection

The prognostic value of mitotic figures in tumor tissue is well-established for many tumor types and automating this task is of high research interest. However, especially deep learning-based methods face performance deterioration in the presence of domain shifts, which may arise from different tumor types, slide preparation and digitization devices. We introduce the MIDOG++ dataset, an extension of the MIDOG 2021 and 2022 challenge datasets. We provide region of interest images from 503 histological specimens of seven different tumor types with variable morphology with in total labels for 11,937 mitotic figures: breast carcinoma, lung carcinoma, lymphosarcoma, neuroendocrine tumor, cutaneous mast cell tumor, cutaneous melanoma, and (sub)cutaneous soft tissue sarcoma. The specimens were processed in several laboratories utilizing diverse scanners. We evaluated the extent of the domain shift by using state-of-the-art approaches, observing notable differences in single-domain training. In a leave-one-domain-out setting, generalizability improved considerably. This mitotic figure dataset is the first that incorporates a wide domain shift based on different tumor types, laboratories, whole slide image scanners, and species

Digital Pathology for the Primary Diagnosis of Breast Histopathological Specimens: An Innovative Validation and Concordance Study

Aim: To train and individually validate a group of breast pathologists in specialty specific digital primary diagnosis using a novel protocol endorsed by the Royal College of Pathologists’ new guideline for digital pathology. The protocol allows early exposure to live digital reporting, in a risk mitigated environment, and focusses on patient safety and professional development. Methods and Results: 3 specialty breast pathologist completed training in use of a digital microscopy system, and were exposed to a training set of 20 challenging cases, designed to help them identify personal digital diagnostic pitfalls. Following this, the 3 pathologists viewed a total of 694 live, entire breast cases. All primary diagnoses were made on digital slides, with immediate glass review and reconciliation before final case sign out. There was complete clinical concordance between the glass and digital impression of the case in 98.8% of cases. Only 1.2% of cases had a clinically significant difference in diagnosis/prognosis on glass and digital slide reads. All pathologists elected to continue using the digital microscope as standard for breast histopathology specimens, with deferral to glass for a limited number of clinical/histological scenarios as a safety net. Conclusion: Individual training and validation for digital primary diagnosis allows pathologists to develop competence and confidence in their digital diagnostic skills, and aids safe and responsible transition from the light microscope to the digital microscope

Deep Learning-Based Grading of Ductal Carcinoma In Situ in Breast Histopathology Images

Ductal carcinoma in situ (DCIS) is a non-invasive breast cancer that can progress into invasive ductal carcinoma (IDC). Studies suggest DCIS is often overtreated since a considerable part of DCIS lesions may never progress into IDC. Lower grade lesions have a lower progression speed and risk, possibly allowing treatment de-escalation. However, studies show significant inter-observer variation in DCIS grading. Automated image analysis may provide an objective solution to address high subjectivity of DCIS grading by pathologists. In this study, we developed a deep learning-based DCIS grading system. It was developed using the consensus DCIS grade of three expert observers on a dataset of 1186 DCIS lesions from 59 patients. The inter-observer agreement, measured by quadratic weighted Cohen's kappa, was used to evaluate the system and compare its performance to that of expert observers. We present an analysis of the lesion-level and patient-level inter-observer agreement on an independent test set of 1001 lesions from 50 patients. The deep learning system (dl) achieved on average slightly higher inter-observer agreement to the observers (o1, o2 and o3) ($\kappa_{o1,dl}=0.81, \kappa_{o2,dl}=0.53, \kappa_{o3,dl}=0.40$) than the observers amongst each other ($\kappa_{o1,o2}=0.58, \kappa_{o1,o3}=0.50, \kappa_{o2,o3}=0.42$) at the lesion-level. At the patient-level, the deep learning system achieved similar agreement to the observers ($\kappa_{o1,dl}=0.77, \kappa_{o2,dl}=0.75, \kappa_{o3,dl}=0.70$) as the observers amongst each other ($\kappa_{o1,o2}=0.77, \kappa_{o1,o3}=0.75, \kappa_{o2,o3}=0.72$). In conclusion, we developed a deep learning-based DCIS grading system that achieved a performance similar to expert observers. We believe this is the first automated system that could assist pathologists by providing robust and reproducible second opinions on DCIS grade

Assessment of algorithms for mitosis detection in breast cancer histopathology images

The proliferative activity of breast tumors, which is routinely estimated by counting of mitotic figures in hematoxylin and eosin stained histology sections, is considered to be one of the most important prognostic markers. However, mitosis counting is laborious, subjective and may suffer from low inter-observer agreement. With the wider acceptance of whole slide images in pathology labs, automatic image analysis has been proposed as a potential solution for these issues. In this paper, the results from the Assessment of Mitosis Detection Algorithms 2013 (AMIDA13) challenge are described. The challenge was based on a data set consisting of 12 training and 11 testing subjects, with more than one thousand annotated mitotic figures by multiple observers. Short descriptions and results from the evaluation of eleven methods are presented. The top performing method has an error rate that is comparable to the inter-observer agreement among pathologists

External validation and clinical utility assessment of PREDICT breast cancer prognostic model in young, systemic treatment-naïve women with node-negative breast cancer

Background: The validity of the PREDICT breast cancer prognostic model is unclear for young patients without adjuvant systemic treatment. This study aimed to validate PREDICT and assess its clinical utility in young women with node-negative breast cancer who did not receive systemic treatment. Methods: We selected all women from the Netherlands Cancer Registry who were diagnosed with node-negative breast cancer under age 40 between 1989 and 2000, a period when adjuvant systemic treatment was not standard practice for women with node-negative disease. We evaluated the calibration and discrimination of PREDICT using the observed/expected (O/E) mortality ratio, and the area under the receiver operating characteristic curve (AUC), respectively. Additionally, we compared the potential clinical utility of PREDICT for selectively administering chemotherapy to the chemotherapy-to-all strategy using decision curve analysis at predefined thresholds. Results: A total of 2264 women with a median age at diagnosis of 36 years were included. Of them, 71.2% had estrogen receptor (ER)-positive tumors and 44.0% had grade 3 tumors. Median tumor size was 16 mm. PREDICT v2.2 underestimated 10-year all-cause mortality by 33% in all women (O/E ratio:1.33, 95%CI:1.22–1.43). Model discrimination was moderate overall (AUC10-year:0.65, 95%CI:0.62–0.68), and poor for women with ER-negative tumors (AUC10-year:0.56, 95%CI:0.51–0.62). Compared to the chemotherapy-to-all strategy, PREDICT only showed a slightly higher net benefit in women with ER-positive tumors, but not in women with ER-negative tumors. Conclusions: PREDICT yields unreliable predictions for young women with node-negative breast cancer. Further model updates are needed before PREDICT can be routinely used in this patient subset.</p

Mitosis domain generalization in histopathology images -- The MIDOG challenge

The density of mitotic figures within tumor tissue is known to be highly correlated with tumor proliferation and thus is an important marker in tumor grading. Recognition of mitotic figures by pathologists is known to be subject to a strong inter-rater bias, which limits the prognostic value. State-of-the-art deep learning methods can support the expert in this assessment but are known to strongly deteriorate when applied in a different clinical environment than was used for training. One decisive component in the underlying domain shift has been identified as the variability caused by using different whole slide scanners. The goal of the MICCAI MIDOG 2021 challenge has been to propose and evaluate methods that counter this domain shift and derive scanner-agnostic mitosis detection algorithms. The challenge used a training set of 200 cases, split across four scanning systems. As a test set, an additional 100 cases split across four scanning systems, including two previously unseen scanners, were given. The best approaches performed on an expert level, with the winning algorithm yielding an F_1 score of 0.748 (CI95: 0.704-0.781). In this paper, we evaluate and compare the approaches that were submitted to the challenge and identify methodological factors contributing to better performance.Comment: 19 pages, 9 figures, summary paper of the 2021 MICCAI MIDOG challeng

Predicting breast tumor proliferation from whole-slide images : the TUPAC16 challenge

Tumor proliferation is an important biomarker indicative of the prognosis of breast cancer patients. Assessment of tumor proliferation in a clinical setting is a highly subjective and labor-intensive task. Previous efforts to automate tumor proliferation assessment by image analysis only focused on mitosis detection in predefined tumor regions. However, in a real-world scenario, automatic mitosis detection should be performed in whole-slide images (WSIs) and an automatic method should be able to produce a tumor proliferation score given a WSI as input. To address this, we organized the TUmor Proliferation Assessment Challenge 2016 (TUPAC16) on prediction of tumor proliferation scores from WSIs. The challenge dataset consisted of 500 training and 321 testing breast cancer histopathology WSIs. In order to ensure fair and independent evaluation, only the ground truth for the training dataset was provided to the challenge participants. The first task of the challenge was to predict mitotic scores, i.e., to reproduce the manual method of assessing tumor proliferation by a pathologist. The second task was to predict the gene expression based PAM50 proliferation scores from the WSI. The best performing automatic method for the first task achieved a quadratic-weighted Cohen's kappa score of κ = 0.567, 95% CI [0.464, 0.671] between the predicted scores and the ground truth. For the second task, the predictions of the top method had a Spearman's correlation coefficient of r = 0.617, 95% CI [0.581 0.651] with the ground truth. This was the first comparison study that investigated tumor proliferation assessment from WSIs. The achieved results are promising given the difficulty of the tasks and weakly-labeled nature of the ground truth. However, further research is needed to improve the practical utility of image analysis methods for this task

Prognostic Value of Stromal Tumor-Infiltrating Lymphocytes in Young, Node-Negative, Triple-Negative Breast Cancer Patients Who Did Not Receive (neo)Adjuvant Systemic Therapy

PURPOSE: Triple-negative breast cancer (TNBC) is considered aggressive, and therefore, virtually all young patients with TNBC receive (neo)adjuvant chemotherapy. Increased stromal tumor-infiltrating lymphocytes (sTILs) have been associated with a favorable prognosis in TNBC. However, whether this association holds for patients who are node-negative (N0), young (< 40 years), and chemotherapy-naïve, and thus can be used for chemotherapy de-escalation strategies, is unknown. METHODS: We selected all patients with N0 TNBC diagnosed between 1989 and 2000 from a Dutch population-based registry. Patients were age < 40 years at diagnosis and had not received (neo)adjuvant systemic therapy, as was standard practice at the time. Formalin-fixed paraffin-embedded blocks were retrieved (PALGA: Dutch Pathology Registry), and a pathology review including sTILs was performed. Patients were categorized according to sTILs (< 30%, 30%-75%, and ≥ 75%). Multivariable Cox regression was performed for overall survival, with or without sTILs as a covariate. Cumulative incidence of distant metastasis or death was analyzed in a competing risk model, with second primary tumors as competing risk. RESULTS: sTILs were scored for 441 patients. High sTILs (≥ 75%; 21%) translated into an excellent prognosis with a 15-year cumulative incidence of a distant metastasis or death of only 2.1% (95% CI, 0 to 5.0), whereas low sTILs (< 30%; 52%) had an unfavorable prognosis with a 15-year cumulative incidence of a distant metastasis or death of 38.4% (32.1 to 44.6). In addition, every 10% increment of sTILs decreased the risk of death by 19% (adjusted hazard ratio: 0.81; 95% CI, 0.76 to 0.87), which are an independent predictor adding prognostic information to standard clinicopathologic variables (χ2 = 46.7, P < .001). CONCLUSION: Chemotherapy-naïve, young patients with N0 TNBC with high sTILs (≥ 75%) have an excellent long-term prognosis. Therefore, sTILs should be considered for prospective clinical trials investigating (neo)adjuvant chemotherapy de-escalation strategies

Prognostic value of histopathologic traits independent of stromal tumor-infiltrating lymphocyte levels in chemotherapy-naïve patients with triple-negative breast cancer

Background: In the absence of prognostic biomarkers, most patients with early-stage triple-negative breast cancer (eTNBC) are treated with combination chemotherapy. The identification of biomarkers to select patients for whom treatment de-escalation or escalation could be considered remains an unmet need. We evaluated the prognostic value of histopathologic traits in a unique cohort of young, (neo)adjuvant chemotherapy-naïve patients with early-stage (stage I or II), node-negative TNBC and long-term follow-up, in relation to stromal tumor-infiltrating lymphocytes (sTILs) for which the prognostic value was recently reported. Materials and methods: We studied all 485 patients with node-negative eTNBC from the population-based PARADIGM cohort which selected women aged &lt;40 years diagnosed between 1989 and 2000. None of the patients had received (neo)adjuvant chemotherapy according to standard practice at the time. Associations between histopathologic traits and breast cancer-specific survival (BCSS) were analyzed with Cox proportional hazard models. Results: With a median follow-up of 20.0 years, an independent prognostic value for BCSS was observed for lymphovascular invasion (LVI) [adjusted (adj.) hazard ratio (HR) 2.35, 95% confidence interval (CI) 1.49-3.69], fibrotic focus (adj. HR 1.61, 95% CI 1.09-2.37) and sTILs (per 10% increment adj. HR 0.75, 95% CI 0.69-0.82). In the sTILs &lt;30% subgroup, the presence of LVI resulted in a higher cumulative incidence of breast cancer death (at 20 years, 58%; 95% CI 41% to 72%) compared with when LVI was absent (at 20 years, 32%; 95% CI 26% to 39%). In the ≥75% sTILs subgroup, the presence of LVI might be associated with poor survival (HR 11.45, 95% CI 0.71-182.36, two deaths). We confirm the lack of prognostic value of androgen receptor expression and human epidermal growth factor receptor 2 -low status. Conclusions: sTILs, LVI and fibrotic focus provide independent prognostic information in young women with node-negative eTNBC. Our results are of importance for the selection of patients for de-escalation and escalation trials.</p