Machine learning methods for histopathological image analysis

Abundant accumulation of digital histopathological images has led to the increased demand for their analysis, such as computer-aided diagnosis using machine learning techniques. However, digital pathological images and related tasks have some issues to be considered. In this mini-review, we introduce the application of digital pathological image analysis using machine learning algorithms, address some problems specific to such analysis, and propose possible solutions.Comment: 23 pages, 4 figure

Symmetric Dense Inception Network for Simultaneous Cell Detection and Classification in Multiplex Immunohistochemistry Images

Deep-learning based automatic analysis of the multiplex immunohistochemistry (mIHC) enables distinct cell populations to be localized on a large scale, providing insights into disease biology and therapeutic targets. However, standard deep-learning pipelines performed cell detection and classification as two-stage tasks, which is computationally inefficient and faces challenges to incorporate neighbouring tissue context for determining the cell identity. To overcome these limitations and to obtain a more accurate mapping of cell phenotypes, we presented a symmetric dense inception neural network for detecting and classifying cells in mIHC slides simultaneously. The model was applied with a novel stop-gradient strategy and a loss function accounted for class imbalance. When evaluated on an ovarian cancer dataset containing 6 cell types, the model achieved an F1 score of 0.835 in cell detection, and a weighted F1-score of 0.867 in cell classification, which outperformed separate models trained on individual tasks by 1.9% and 3.8% respectively. Taken together, the proposed method boosts the learning efficiency and prediction accuracy of cell detection and classification by simultaneously learning from both tasks

Self-supervised deep learning for highly efficient spatial immunophenotyping

Background: Efficient biomarker discovery and clinical translation depend on the fast and accurate analytical output from crucial technologies such as multiplex imaging. However, reliable cell classification often requires extensive annotations. Label-efficient strategies are urgently needed to reveal diverse cell distribution and spatial interactions in large-scale multiplex datasets. / Methods: This study proposed Self-supervised Learning for Antigen Detection (SANDI) for accurate cell phenotyping while mitigating the annotation burden. The model first learns intrinsic pairwise similarities in unlabelled cell images, followed by a classification step to map learnt features to cell labels using a small set of annotated references. We acquired four multiplex immunohistochemistry datasets and one imaging mass cytometry dataset, comprising 2825 to 15,258 single-cell images to train and test the model. / Findings: With 1% annotations (18–114 cells), SANDI achieved weighted F1-scores ranging from 0.82 to 0.98 across the five datasets, which was comparable to the fully supervised classifier trained on 1828–11,459 annotated cells (−0.002 to −0.053 of averaged weighted F1-score, Wilcoxon rank-sum test, P = 0.31). Leveraging the immune checkpoint markers stained in ovarian cancer slides, SANDI-based cell identification reveals spatial expulsion between PD1-expressing T helper cells and T regulatory cells, suggesting an interplay between PD1 expression and T regulatory cell-mediated immunosuppression. / Interpretation: By striking a fine balance between minimal expert guidance and the power of deep learning to learn similarity within abundant data, SANDI presents new opportunities for efficient, large-scale learning for histology multiplex imaging data. / Funding: This study was funded by the Royal Marsden/ ICR National Institute of Health Research Biomedical Research Centre

Better prognostic markers for nonmuscle invasive papillary urothelial carcinomas

Bladder cancer is a common type of cancer, especially among men in developed countries. Most cancers in the urinary bladder are papillary urothelial carcinomas. They are characterized by a high recurrence frequency (up to 70 %) after local resection. It is crucial for prognosis to discover these recurrent tumours at an early stage, especially before they become muscle-invasive. Reliable prognostic biomarkers for tumour recurrence and stage progression are lacking. This is why patients diagnosed with a non-muscle invasive bladder cancer follow extensive follow-up regimens with possible serious side effects and with high costs for the healthcare systems. WHO grade and tumour stage are two central biomarkers currently having great impact on both treatment decisions and follow-up regimens. However, there are concerns regarding the reproducibility of WHO grading, and stage classification is challenging in small and fragmented tumour material. In Paper I, we examined the reproducibility and the prognostic value of all the individual microscopic features making up the WHO grading system. Among thirteen extracted features there was considerable variation in both reproducibility and prognostic value. The only feature being both reasonably reproducible and statistically significant prognostic was cell polarity. We concluded that further validation studies are needed on these features, and that future grading systems should be based on well-defined features with true prognostic value. With the implementation of immunotherapy, there is increasing interest in tumour immune response and the tumour microenvironment. In a search for better prognostic biomarkers for tumour recurrence and stage progression, in Paper II, we investigated the prognostic value of tumour infiltrating immune cells (CD4, CD8, CD25 and CD138) and previously investigated cell proliferation markers (Ki-67, PPH3 and MAI). Low Ki 67 and tumour multifocality were associated with increased recurrence risk. Recurrence risk was not affected by the composition of immune cells. For stage progression, the only prognostic immune cell marker was CD25. High values for MAI was also strongly associated with stage progression. However, in a multivariate analysis, the most prognostic feature was a combination of MAI and CD25. BCG-instillations in the bladder are indicated in intermediate and high-risk non-muscle invasive bladder cancer patients. This old-fashion immunotherapy has proved to reduce both recurrence- and progression-risk, although it is frequently followed by unpleasant side-effects. As many as 30-50% of high-risk patients receiving BCG instillations, fail by develop high-grade recurrences. They do not only suffer from unnecessary side-effects, but will also have a delay in further treatment. Together with colleagues at three different Dutch hospitals, in Paper III, we looked at the prognostic and predictive value of T1-substaging. A T1-tumour invades the lamina propria, and we wanted to separate those with micro- from those with extensive invasion. We found that BCG-failure was more common among patients with extensive invasion. Furthermore, T1-substaging was associated with both high-grade recurrence-free and progression-free survival. Finally, in Paper IV, we wanted to investigate the prognostic value of two classical immunohistochemical markers, p53 and CK20, and compare them with previously investigated proliferation markers. p53 is a surrogate marker for mutations in the gene TP53, considered to be a main characteristic for muscle-invasive tumours. CK20 is a surrogate marker for luminal tumours in the molecular classification of bladder cancer, and is frequently used to distinguish reactive urothelial changes from urothelial carcinoma in situ. We found both positivity for p53 and CK20 to be significantly associated with stage progression, although not performing better than WHO grade and stage. The proliferation marker MAI, had the highest prognostic value in our study. Any combination of variables did not perform better in a multivariate analysis than MAI alone

EndoNet: model for automatic calculation of H-score on histological slides

H-score is a semi-quantitative method used to assess the presence and distribution of proteins in tissue samples by combining the intensity of staining and percentage of stained nuclei. It is widely used but time-consuming and can be limited in accuracy and precision. Computer-aided methods may help overcome these limitations and improve the efficiency of pathologists' workflows. In this work, we developed a model EndoNet for automatic calculation of H-score on histological slides. Our proposed method uses neural networks and consists of two main parts. The first is a detection model which predicts keypoints of centers of nuclei. The second is a H-score module which calculates the value of the H-score using mean pixel values of predicted keypoints. Our model was trained and validated on 1780 annotated tiles with a shape of 100x100 $\mu m$ and performed 0.77 mAP on a test dataset. Moreover, the model can be adjusted to a specific specialist or whole laboratory to reproduce the manner of calculating the H-score. Thus, EndoNet is effective and robust in the analysis of histology slides, which can improve and significantly accelerate the work of pathologists

Microenvironmental control of malignant growth

The tumor microenvironment (TME) comprises a complex milieu of different cell types, including cancer associated fibroblasts (CAFs) and immune cells, blood vessels, and the extracellular matrix. Through its interaction with cancer cells, it plays an essential role in cancer invasion and metastasis. The inherent complexity of the TME presents a challenge to study it within experimental model systems. It underscores the importance of complementing such research with observation from human tumor tissues, wherein this intricate complexity is preserved. In Paper IV, we introduce a new software designed to explore the Human Protein Atlas, an online database that includes image data on the protein expression across normal and cancerous tissues from immunohistochemically (IHC) stained tissues. In Paper I, we use this software to identify 12 novel proteins expressed in cancerassociated fibroblasts, four revealing connections to Rho-kinase signaling. We contrast their expression across various tumors and against normal tissue fibroblasts, uncovering expression variability among cancer types and confirm their similarities with the myofibroblastic phenotype. In Paper II, we explore the expression of the proteoglycan Decorin, abundantly present in normal connective tissue and having tumor inhibitory properties, showing its downregulation in the connective tissue surrounding tumors. In Paper III, based on our observations in Paper I of the connection of Rhosignaling in CAFs, we study the effects of knocking out the related RhoA in fibroblasts both in vitro and in vivo models. We demonstrate that the knockout fibroblasts compromise their tumor inhibitory capacity, enhancing cancer cell growth, migration, and metastasis. In Paper VI, we develop a new method for analyzing the extensive data within the Human Protein Atlas by developing a deep-learning-based image classifier. Utilizing a limited training image set, we classify all images available for the prostate, identifying 44 new markers of prostate basal cells. In Paper IV, we explore the influence of the TME on cancer cells by systematically analyzing 20 pancreatic cancer patient samples utilizing an IHC panel. We define shifts in cancer cell phenotype relative to tissue localization, including a transition to a more indolent cancer phenotype, an effect on cancer cell proliferation, and a tendency to normalize the cancer cell phenotype. In conclusion, we developed two new methods that enable us to study protein expression in normal and cancerous tissues by enhancing the capabilities of the HPA. We identified new markers of CAFs and revealed a connection to Rhosignaling. Knocking out the related RhoA in experimental systems resulted in the fibroblasts losing their cancer inhibitory capacity. Finally, we show the remarkable plasticity of cancer cells, demonstrating that their phenotype undergoes significant alterations based on their spatial localization within normal tissue