Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology

Stain variation is a phenomenon observed when distinct pathology laboratories stain tissue slides that exhibit similar but not identical color appearance. Due to this color shift between laboratories, convolutional neural networks (CNNs) trained with images from one lab often underperform on unseen images from the other lab. Several techniques have been proposed to reduce the generalization error, mainly grouped into two categories: stain color augmentation and stain color normalization. The former simulates a wide variety of realistic stain variations during training, producing stain-invariant CNNs. The latter aims to match training and test color distributions in order to reduce stain variation. For the first time, we compared some of these techniques and quantified their effect on CNN classification performance using a heterogeneous dataset of hematoxylin and eosin histopathology images from 4 organs and 9 pathology laboratories. Additionally, we propose a novel unsupervised method to perform stain color normalization using a neural network. Based on our experimental results, we provide practical guidelines on how to use stain color augmentation and stain color normalization in future computational pathology applications.Comment: Accepted in the Medical Image Analysis journa

Exploring immune status in peripheral blood and tumor tissue in association with survival in patients with multi-organ metastatic colorectal cancer

Colorectal cancer (CRC) raises considerable clinical challenges, including a high mortality rate once the tumor spreads to distant sites. At this advanced stage, more accurate prediction of prognosis and treatment outcome is urgently needed. The role of cancer immunity in metastatic CRC (mCRC) is poorly understood. Here, we explore cellular immune cell status in patients with multi-organ mCRC. We analyzed T cell infiltration in primary tumor sections, surveyed the lymphocytic landscape of liver metastases, and assessed circulating mononuclear immune cells. Besides asking whether immune cells are associated with survival at this stage of the disease, we investigated correlations between the different tissue types; as this could indicate a dominant immune phenotype. Taken together, our analyses corroborate previous observations that higher levels of CD8+ T lymphocytes link to better survival outcomes. Our findings therefore extend evidence from earlier stages of CRC to indicate an important role for cancer immunity in disease control even after metastatic spreading to multiple organs. This finding may help to improve predicting outcome of patients with mCRC and suggests a future role for immunotherapeutic strategies.</p

Improving tumor budding reporting in colorectal cancer : a Delphi consensus study

Tumor budding is a long-established independent adverse prognostic marker in colorectal cancer, yet methods for its assessment have varied widely. In an effort to standardize its reporting, a group of experts met in Bern, Switzerland, in 2016 to reach consensus on a single, international, evidence-based method for tumor budding assessment and reporting (International Tumor Budding Consensus Conference [ITBCC]). Tumor budding assessment using the ITBCC criteria has been validated in large cohorts of cancer patients and incorporated into several international colorectal cancer pathology and clinical guidelines. With the wider reporting of tumor budding, new issues have emerged that require further clarification. To better inform researchers and health-care professionals on these issues, an international group of experts in gastrointestinal pathology participated in a modified Delphi process to generate consensus and highlight areas requiring further research. This effort serves to re-affirm the importance of tumor budding in colorectal cancer and support its continued use in routine clinical practice.Peer reviewe

Residual cyclegan for robust domain transformation of histopathological tissue slides

Variation between stains in histopathology is commonplace across different medical centers. This can have a significant effect on the reliability of machine learning algorithms. In this paper, we propose to reduce performance variability by using-consistent generative adversarial (CycleGAN) networks to remove staining variation. We improve upon the regular CycleGAN by incorporating residual learning. We comprehensively evaluate the performance of our stain transformation method and compare its usefulness in addition to extensive data augmentation to enhance the robustness of tissue segmentation algorithms. Our steps are as follows: first, we train a model to perform segmentation on tissue slides from a single source center, while heavily applying augmentations to increase robustness to unseen data. Second, we evaluate and compare the segmentation performance on data from other centers, both with and without applying our CycleGAN stain transformation. We compare segmentation performances in a colon tissue segmentation and kidney tissue segmentation task, covering data from 6 different centers. We show that our transformation method improves the overall Dice coefficient by 9% over the non-normalized target data and by 4% over traditional stain transformation in our colon tissue segmentation task. For kidney segmentation, our residual CycleGAN increases performance by 10% over no transformation and around 2% compared to the non-residual CycleGAN. (c) 2021 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY license ( http://creativecommons.org/licenses/by/4.0/ )Funding Agencies|Knut and Alice Wallenberg foundationKnut &amp; Alice Wallenberg Foundation</p

Tumor budding t-cell graphs ::assessing the need for resection in pT1 colorectal cancer patients

Colon resection is often the treatment of choice for colorectal cancer (CRC) patients. However, especially for minimally invasive cancer, such as pT1, simply removing the polyps may be enough to stop cancer progression. Different histopathological risk factors such as tumor grade and invasion depth currently found the basis for the need for colon resection in pT1 CRC patients. Here, we investigate two additional risk factors, tumor budding and lymphocyte infiltration at the invasive front, which are known to be clinically relevant. We capture the spatial layout of tumor buds and T-cells and use graph-based deep learning to investigate them as potential risk predictors. Our pT1 Hotspot Tumor Budding T-cell Graph (pT1-HBTG) dataset consists of 626 tumor budding hotspots from 575 patients. We propose and compare three different graph structures, as well as combinations of the node labels. The best-performing Graph Neural Network architecture is able to increase specificity by 20% compared to the currently recommended risk stratification based on histopathological risk factors, without losing any sensitivity. We believe that using a graph-based analysis can help to assist pathologists in making risk assessments for pT1 CRC patients, and thus decrease the number of patients undergoing potentially unnecessary surgery. Both the code and dataset are made publicly available

Taking Tumour Budding to the Next Frontier - a Post-ITBCC 2016 Review.

Tumour budding in colorectal cancer, defined as single tumour cells or small clusters containing four tumour cells or less, is a robust and independent biomarker of aggressive tumour biology. Based on published data in the literature, evidence is certainly in favour of reporting tumour budding in routine practice. One important aspect of implementing tumour budding has been to establish a standardized and evidence-based scoring method as has been recommended by the International Tumour Budding Consensus Conference (ITBCC) in 2016. Further developments have aimed at establishing methods of automated tumour budding assessment. A digital approach to scoring tumour buds has great potential to assist in performing an objective budding count, but in analogy to the manual consensus method, must be validated and standardized. The aim of the present review is to present general considerations behind the ITBCC scoring method and a broad overview of the current status quo and challenges faced by automated tumour budding detection methods

Are tumour grade and tumour budding equivalent in colorectal cancer? A retrospective analysis of 771 patients.

BACKGROUND Tumour grade is traditionally considered in the management of patients with colorectal cancer. However, a large body of literature suggests that a related feature, namely tumour budding, may have a more important clinical impact. The aim of our study is to determine the correlation between tumour grade and tumour budding and their impact on patient outcome. METHODS A retrospective collective of 771 patients with colorectal cancer were included in the study. Clinicopathological information included tumour grade (World Health Organisation 2010; G1, G2 and G3) and tumour budding evaluated as BD1, BD2 and BD3 and representing 0-4 buds, 5-9 buds and 10 or more buds per 0.785 mm2, respectively. RESULTS Tumour grade and tumour budding were correlated (p < 0.0001, percent concordance: 33.8%). Of the BD1 cases, 18.1% were of G3. Only two BD3 cases were G1. Both high tumour grade and tumour budding were associated with higher pT, lymph node metastasis, distant metastasis and lymphatic and venous vessel invasion (p < 0.01, all), but only tumour grade was additionally associated with right-sided tumour location and mucinous histology. Higher tumour budding led to worse overall (p = 0.0286) and disease-free survival (p = 0.001), but tumour grade did not. Budding was independent of tumour grade in multivariate analysis. DISCUSSION Tumour grade and tumour budding are distinct features, as recognised by their different clinicopathological associations, reflecting different underlying biological processes. Nonetheless, tumour budding seems to outperform tumour grade in terms of predicting disease-free survival

Automatic quantification of tumor-stroma ratio as a prognostic marker for pancreatic cancer.

PurposeThis study aims to introduce an innovative multi-step pipeline for automatic tumor-stroma ratio (TSR) quantification as a potential prognostic marker for pancreatic cancer, addressing the limitations of existing staging systems and the lack of commonly used prognostic biomarkers.MethodsThe proposed approach involves a deep-learning-based method for the automatic segmentation of tumor epithelial cells, tumor bulk, and stroma from whole-slide images (WSIs). Models were trained using five-fold cross-validation and evaluated on an independent external test set. TSR was computed based on the segmented components. Additionally, TSR's predictive value for six-month survival on the independent external dataset was assessed.ResultsMedian Dice (inter-quartile range (IQR)) of 0.751(0.15) and 0.726(0.25) for tumor epithelium segmentation on internal and external test sets, respectively. Median Dice of 0.76(0.11) and 0.863(0.17) for tumor bulk segmentation on internal and external test sets, respectively. TSR was evaluated as an independent prognostic marker, demonstrating a cross-validation AUC of 0.61±0.12 for predicting six-month survival on the external dataset.ConclusionOur pipeline for automatic TSR quantification offers promising potential as a prognostic marker for pancreatic cancer. The results underscore the feasibility of computational biomarker discovery in enhancing patient outcome prediction, thus contributing to personalized patient management

Deep learning for multi-class semantic segmentation enables colorectal cancer detection and classification in digital pathology images

In colorectal cancer (CRC), artificial intelligence (AI) can alleviate the laborious task of characterization and reporting on resected biopsies, including polyps, the numbers of which are increasing as a result of CRC population screening programs ongoing in many countries all around the globe. Here, we present an approach to address two major challenges in the automated assessment of CRC histopathology whole-slide images. We present an AI-based method to segment multiple (n=14 ) tissue compartments in the H &amp;E-stained whole-slide image, which provides a different, more perceptible picture of tissue morphology and composition. We test and compare a panel of state-of-the-art loss functions available for segmentation models, and provide indications about their use in histopathology image segmentation, based on the analysis of (a) a multi-centric cohort of CRC cases from five medical centers in the Netherlands and Germany, and (b) two publicly available datasets on segmentation in CRC. We used the best performing AI model as the basis for a computer-aided diagnosis system that classifies colon biopsies into four main categories that are relevant pathologically. We report the performance of this system on an independent cohort of more than 1000 patients. The results show that with a good segmentation network as a base, a tool can be developed which can support pathologists in the risk stratification of colorectal cancer patients, among other possible uses. We have made the segmentation model available for research use on .Funding Agencies|Dutch Cancer Society [10602/2016-2]; Alpe dHuZes / Dutch Cancer Society Fund [KUN 2014-7032]; European Unions Horizon 2020 research and innovation programme [825292]</p