Divide-and-Rule: Self-Supervised Learning for Survival Analysis in Colorectal Cancer

With the long-term rapid increase in incidences of colorectal cancer (CRC), there is an urgent clinical need to improve risk stratification. The conventional pathology report is usually limited to only a few histopathological features. However, most of the tumor microenvironments used to describe patterns of aggressive tumor behavior are ignored. In this work, we aim to learn histopathological patterns within cancerous tissue regions that can be used to improve prognostic stratification for colorectal cancer. To do so, we propose a self-supervised learning method that jointly learns a representation of tissue regions as well as a metric of the clustering to obtain their underlying patterns. These histopathological patterns are then used to represent the interaction between complex tissues and predict clinical outcomes directly. We furthermore show that the proposed approach can benefit from linear predictors to avoid overfitting in patient outcomes predictions. To this end, we introduce a new well-characterized clinicopathological dataset, including a retrospective collective of 374 patients, with their survival time and treatment information. Histomorphological clusters obtained by our method are evaluated by training survival models. The experimental results demonstrate statistically significant patient stratification, and our approach outperformed the state-of-the-art deep clustering methods

k is the Magic Number -- Inferring the Number of Clusters Through Nonparametric Concentration Inequalities

Most convex and nonconvex clustering algorithms come with one crucial parameter: the $k$ in $k$-means. To this day, there is not one generally accepted way to accurately determine this parameter. Popular methods are simple yet theoretically unfounded, such as searching for an elbow in the curve of a given cost measure. In contrast, statistically founded methods often make strict assumptions over the data distribution or come with their own optimization scheme for the clustering objective. This limits either the set of applicable datasets or clustering algorithms. In this paper, we strive to determine the number of clusters by answering a simple question: given two clusters, is it likely that they jointly stem from a single distribution? To this end, we propose a bound on the probability that two clusters originate from the distribution of the unified cluster, specified only by the sample mean and variance. Our method is applicable as a simple wrapper to the result of any clustering method minimizing the objective of $k$-means, which includes Gaussian mixtures and Spectral Clustering. We focus in our experimental evaluation on an application for nonconvex clustering and demonstrate the suitability of our theoretical results. Our \textsc{SpecialK} clustering algorithm automatically determines the appropriate value for $k$, without requiring any data transformation or projection, and without assumptions on the data distribution. Additionally, it is capable to decide that the data consists of only a single cluster, which many existing algorithms cannot

Uncertainty driven pooling network for microvessel segmentation in routine histology images

Lymphovascular invasion (LVI) and tumor angiogenesis are correlated with metastasis, cancer recurrence and poor patient survival. In most of the cases, the LVI quantification and angiogenic analysis is based on microvessel segmentation and density estimation in immunohistochemically (IHC) stained tissues. However, in routine H&E stained images, the microvessels display a high level of heterogeneity in terms of size, shape, morphology and texture which makes microvessel segmentation a non-trivial task. Manual delineation of microvessels for biomarker analysis is labor-intensive, time consuming, irreproducible and can suffer from subjectivity among pathologists. Moreover, it is often beneficial to account for the uncertainty of a prediction when making a diagnosis. To address these challenges, we proposed a framework for microvessel segmentation in H&E stained histology images. The framework extends DeepLabV3+ by using an improved dice coefficient based custom loss function and also incorporating an uncertainty prediction mechanism. The proposed method uses an aligned Xception model, followed by atrous spatial pyramid pooling for feature extraction at multiple scales. This architecture counters the challenge of segmenting blood vessels of varying morphological appearance. To incorporate uncertainty, random transformations are introduced at test time for a superior segmentation result and simultaneous uncertainty map generation, highlighting ambiguous regions. The method is evaluated using 1167 images of size 512×512 pixels, extracted from 13 WSIs of oral squamous cell carcinoma (OSCC) tissue at 20x magnification. The proposed net-work achieves state-of-the-art performance compared to current semantic segmentation deep neural networks (FCN-8, U-Net, SegNet and DeepLabV3+)

Direct image to subtype prediction for brain tumors using deep learning

BACKGROUND: Deep Learning (DL) can predict molecular alterations of solid tumors directly from routine histopathology slides. Since the 2021 update of the World Health Organization (WHO) diagnostic criteria, the classification of brain tumors integrates both histopathological and molecular information. We hypothesize that DL can predict molecular alterations as well as WHO subtyping of brain tumors from hematoxylin and eosin-stained histopathology slides. METHODS: We used weakly supervised DL and applied it to three large cohorts of brain tumor samples, comprising N = 2845 patients. RESULTS: We found that the key molecular alterations for subtyping, IDH and ATRX, as well as 1p19q codeletion, were predictable from histology with an area under the receiver operating characteristic curve (AUROC) of 0.95, 0.90, and 0.80 in the training cohort, respectively. These findings were upheld in external validation cohorts with AUROCs of 0.90, 0.79, and 0.87 for prediction of IDH, ATRX, and 1p19q codeletion, respectively. CONCLUSIONS: In the future, such DL-based implementations could ease diagnostic workflows, particularly for situations in which advanced molecular testing is not readily available

Deep Learning for the detection of microsatellite instability from histology images in colorectal cancer: a systematic literature review

Microsatellite instability (MSI) or deficient mismatch repair (dMMR) is a clinically important genetic feature affecting 10-15% of colorectal cancer (CRC) patients. Patients with metastatic MSI/dMMR CRC are eligible for therapy with immune checkpoint inhibitors, making MSI/dMMR the most important immuno-oncological biomarker in CRC. Gold standard tests for detection of MSI/dMMR in CRC are based on wet laboratory tests such as immunohistochemistry (IHC) or DNA extraction with subsequent polymerase chain reaction (PCR). However, since 2019, advances in Deep Learning (DL), an Artificial Intelligence (AI) technology, have enabled the prediction of MSI/dMMR directly from digitized routine haematoxylin and eosin (H&E) histopathology slides with high accuracy. In addition to the initial proof-of-concept publication in 2019, twelve subsequent studies have refined, improved, and further validated this approach. At this moment, MSI/dMMR prediction using Deep Learning has become a widely used benchmark task for academic studies in the field of computational pathology. Beyond academic use, this assay has attracted commercial interest from companies with the possibility of approval as a diagnostic device in the near future. In this review, we summarize and quantitatively compare the existing evidence on Deep-Learning-based detection of MSI/dMMR in CRC and discuss the need for further improvement and potential for integration into routine pathological workflows. Ultimately, this DL-based method could facilitate the identification of patients eligible for treatment with immune checkpoint inhibitors by pre-screening or replacement of current methods

Encrypted federated learning for secure decentralized collaboration in cancer image analysis.

Artificial intelligence (AI) has a multitude of applications in cancer research and oncology. However, the training of AI systems is impeded by the limited availability of large datasets due to data protection requirements and other regulatory obstacles. Federated and swarm learning represent possible solutions to this problem by collaboratively training AI models while avoiding data transfer. However, in these decentralized methods, weight updates are still transferred to the aggregation server for merging the models. This leaves the possibility for a breach of data privacy, for example by model inversion or membership inference attacks by untrusted servers. Somewhat-homomorphically-encrypted federated learning (SHEFL) is a solution to this problem because only encrypted weights are transferred, and model updates are performed in the encrypted space. Here, we demonstrate the first successful implementation of SHEFL in a range of clinically relevant tasks in cancer image analysis on multicentric datasets in radiology and histopathology. We show that SHEFL enables the training of AI models which outperform locally trained models and perform on par with models which are centrally trained. In the future, SHEFL can enable multiple institutions to co-train AI models without forsaking data governance and without ever transmitting any decryptable data to untrusted servers

Generalizable biomarker prediction from cancer pathology slides with self-supervised deep learning: A retrospective multi-centric study

Deep learning (DL) can predict microsatellite instability (MSI) from routine histopathology slides of colorectal cancer (CRC). However, it is unclear whether DL can also predict other biomarkers with high performance and whether DL predictions generalize to external patient populations. Here, we acquire CRC tissue samples from two large multi-centric studies. We systematically compare six different state-of-the-art DL architectures to predict biomarkers from pathology slides, including MSI and mutations in BRAF, KRAS, NRAS, and PIK3CA. Using a large external validation cohort to provide a realistic evaluation setting, we show that models using self-supervised, attention-based multiple-instance learning consistently outperform previous approaches while offering explainable visualizations of the indicative regions and morphologies. While the prediction of MSI and BRAF mutations reaches a clinical-grade performance, mutation prediction of PIK3CA, KRAS, and NRAS was clinically insufficient

Loss of Regulator of G Protein Signaling 5 Exacerbates Obesity, Hepatic Steatosis, Inflammation and Insulin Resistance

BACKGROUND: The effect of regulator of G protein signaling 5 (RGS5) on cardiac hypertrophy, atherosclerosis and angiogenesis has been well demonstrated, but the role in the development of obesity and insulin resistance remains completely unknown. We determined the effect of RGS5 deficiency on obesity, hepatic steatosis, inflammation and insulin resistance in mice fed either a normal-chow diet (NC) or a high-fat diet (HF). METHODOLOGY/PRINCIPAL FINDINGS: Male, 8-week-old RGS5 knockout (KO) and littermate control mice were fed an NC or an HF for 24 weeks and were phenotyped accordingly. RGS5 KO mice exhibited increased obesity, fat mass and ectopic lipid deposition in the liver compared with littermate control mice, regardless of diet. When fed an HF, RGS5 KO mice had a markedly exacerbated metabolic dysfunction and inflammatory state in the blood serum. Meanwhile, macrophage recruitment and inflammation were increased and these increases were associated with the significant activation of JNK, IκBα and NF-κBp65 in the adipose tissue, liver and skeletal muscle of RGS5 KO mice fed an HF relative to control mice. These exacerbated metabolic dysfunction and inflammation are accompanied with decreased systemic insulin sensitivity in the adipose tissue, liver and skeletal muscle of RGS5 KO mice, reflected by weakened Akt/GSK3β phosphorylation. CONCLUSIONS/SIGNIFICANCE: Our data suggest that loss of RGS5 exacerbates HF-induced obesity, hepatic steatosis, inflammation and insulin resistance

Pan-cancer image-based detection of clinically actionable genetic alterations

Molecular alterations in cancer can cause phenotypic changes in tumor cells and their microenvironment. Routine histopathology tissue slides, which are ubiquitously available, can reflect such morphological changes. Here, we show that deep learning can consistently infer a wide range of genetic mutations, molecular tumor subtypes, gene expression signatures and standard pathology biomarkers directly from routine histology. We developed, optimized, validated and publicly released a one-stop-shop workflow and applied it to tissue slides of more than 5,000 patients across multiple solid tumors. Our findings show that a single deep learning algorithm can be trained to predict a wide range of molecular alterations from routine, paraffin-embedded histology slides stained with hematoxylin and eosin. These predictions generalize to other populations and are spatially resolved. Our method can be implemented on mobile hardware, potentially enabling point-of-care diagnostics for personalized cancer treatment. More generally, this approach could elucidate and quantify genotype–phenotype links in cancer

Swarm learning for decentralized artificial intelligence in cancer histopathology

Artificial intelligence (AI) can predict the presence of molecular alterations directly from routine histopathology slides. However, training robust AI systems requires large datasets for which data collection faces practical, ethical and legal obstacles. These obstacles could be overcome with swarm learning (SL), in which partners jointly train AI models while avoiding data transfer and monopolistic data governance. Here, we demonstrate the successful use of SL in large, multicentric datasets of gigapixel histopathology images from over 5,000 patients. We show that AI models trained using SL can predict BRAF mutational status and microsatellite instability directly from hematoxylin and eosin (H&E)-stained pathology slides of colorectal cancer. We trained AI models on three patient cohorts from Northern Ireland, Germany and the United States, and validated the prediction performance in two independent datasets from the United Kingdom. Our data show that SL-trained AI models outperform most locally trained models, and perform on par with models that are trained on the merged datasets. In addition, we show that SL-based AI models are data efficient. In the future, SL can be used to train distributed AI models for any histopathology image analysis task, eliminating the need for data transfer