Weakly Supervised Deep Learning Predicts Immunotherapy Response in Solid Tumors Based on PD-L1 Expression

Deep learning; Immunotherapy; Solid tumorsAprenentatge profund; Immunoteràpia; Tumors sòlidsAprendizaje profundo; Inmunoterapia; Tumores sólidosProgrammed death-ligand 1 (PD-L1) IHC is the most commonly used biomarker for immunotherapy response. However, quantification of PD-L1 status in pathology slides is challenging. Neither manual quantification nor a computer-based mimicking of manual readouts is perfectly reproducible, and the predictive performance of both approaches regarding immunotherapy response is limited. In this study, we developed a deep learning (DL) method to predict PD-L1 status directly from raw IHC image data, without explicit intermediary steps such as cell detection or pigment quantification. We trained the weakly supervised model on PD-L1–stained slides from the non–small cell lung cancer (NSCLC)-Memorial Sloan Kettering (MSK) cohort (N = 233) and validated it on the pan-cancer-Vall d'Hebron Institute of Oncology (VHIO) cohort (N = 108). We also investigated the performance of the model to predict response to immune checkpoint inhibitors (ICI) in terms of progression-free survival. In the pan-cancer-VHIO cohort, the performance was compared with tumor proportion score (TPS) and combined positive score (CPS). The DL model showed good performance in predicting PD-L1 expression (TPS ≥ 1%) in both NSCLC-MSK and pan-cancer-VHIO cohort (AUC 0.88 ± 0.06 and 0.80 ± 0.03, respectively). The predicted PD-L1 status showed an improved association with response to ICIs [HR: 1.5 (95% confidence interval: 1–2.3), P = 0.049] compared with TPS [HR: 1.4 (0.96–2.2), P = 0.082] and CPS [HR: 1.2 (0.79–1.9), P = 0.386]. Notably, our explainability analysis showed that the model does not just look at the amount of brown pigment in the IHC slides, but also considers morphologic factors such as lymphocyte conglomerates. Overall, end-to-end weakly supervised DL shows potential for improving patient stratification for cancer immunotherapy by analyzing PD-L1 IHC, holistically integrating morphology and PD-L1 staining intensity. Significance: The weakly supervised DL model to predict PD-L1 status from raw IHC data, integrating tumor staining intensity and morphology, enables enhanced patient stratification in cancer immunotherapy compared with traditional pathologist assessment.J.N. Kather is supported by the German Federal Ministry of Health (DEEP LIVER, ZMVI1-2520DAT111) and the Max-Eder-Programme of the German Cancer Aid (grant no. 70113864), the German Federal Ministry of Education and Research (PEARL, 01KD2104C; CAMINO, 01EO2101; SWAG, 01KD2215A; TRANSFORM LIVER, 031L0312A; TANGERINE, 01KT2302 through ERA-NET Transcan), the German Academic Exchange Service (SECAI, 57616814), the German Federal Joint Committee (Transplant.KI, 01VSF21048) the European Union's Horizon Europe and innovation programme (ODELIA, 101057091; GENIAL, 101096312) and the National Institute for Health and Care Research (NIHR, NIHR213331) Leeds Biomedical Research Centre. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. R. Perez-Lopez is supported by LaCaixa Foundation, a CRIS Foundation Talent Award (TALENT19-05), the FERO Foundation, the Instituto de Salud Carlos III-Investigacion en Salud (PI18/01395 and PI21/01019) and the Prostate Cancer Foundation (18YOUN19). M. Ligero is supported by the PERIS PIF-Salut Grant. As per the ICMJE guidelines of April 2023, we hereby disclose that the following artificial intelligence tools were used to write this article: ChatGPT-4 for checking and correcting spelling and grammar

The prognostic impact of the tumour stroma fraction: A machine learning-based analysis in 16 human solid tumour types

Background: The development of a reactive tumour stroma is a hallmark of tumour progression and pronounced tumour stroma is generally considered to be associated with clinical aggressiveness. The variability between tumour types regarding stroma fraction, and its prognosis associations, have not been systematically analysed.Methods: Using an objective machine-learning method we quantified the tumour stroma in 16 solid cancer types from 2732 patients, representing retrospective tissue collections of surgically resected primary tumours. Image analysis performed tissue segmentation into stromal and epithelial compartment based on pan-cytokeratin staining and autofluorescence patterns.Findings: The stroma fraction was highly variable within and across the tumour types, with kidney cancer showing the lowest and pancreato-biliary type periampullary cancer showing the highest stroma proportion (median 19% and 73% respectively). Adjusted Cox regression models revealed both positive (pancreato-biliary type periampullary cancer and oestrogen negative breast cancer, HR(95%CI)=0.56(0.34-0.92) and HR (95%CI)=0.41(0.17-0.98) respectively) and negative (intestinal type periampullary cancer, HR(95%CI)=3.59 (1.49-8.62)) associations of the tumour stroma fraction with survival.Interpretation: Our study provides an objective quantification of the tumour stroma fraction across major types of solid cancer. Findings strongly argue against the commonly promoted view of a general associations between high stroma abundance and poor prognosis. The results also suggest that full exploitation of the prognostic potential of tumour stroma requires analyses that go beyond determination of stroma abundance.</p

An immune score reflecting pro- and anti-tumoural balance of tumour microenvironment has major prognostic impact and predicts immunotherapy response in solid cancers

Background: Cancer immunity is based on the interaction of a multitude of cells in the spatial context of the tumour tissue. Clinically relevant immune signatures are therefore anticipated to fundamentally improve the accuracy in predicting disease progression. Methods: Through a multiplex in situ analysis we evaluated 15 immune cell classes in 1481 tumour samples. Single-cell and bulk RNAseq data sets were used for functional analysis and validation of prognostic and predictive associations. Findings: By combining the prognostic information of anti-tumoural CD8+ lymphocytes and tumour supportive CD68+CD163+ macrophages in colorectal cancer we generated a signature of immune activation (SIA). The prognostic impact of SIA was independent of conventional parameters and comparable with the state-of-art immune score. The SIA was also associated with patient survival in oesophageal adenocarcinoma, bladder cancer, lung adenocarcinoma and melanoma, but not in endometrial, ovarian and squamous cell lung carcinoma. We identified CD68+CD163+ macrophages as the major producers of complement C1q, which could serve as a surrogate marker of this macrophage subset. Consequently, the RNA-based version of SIA (ratio of CD8A to C1QA) was predictive for survival in independent RNAseq data sets from these six cancer types. Finally, the CD8A/C1QA mRNA ratio was also predictive for the response to checkpoint inhibitor therapy. Interpretation: Our findings extend current concepts to procure prognostic information from the tumour immune microenvironment and provide an immune activation signature with high clinical potential in common human cancer types. Funding: Swedish Cancer Society, Lions Cancer Foundation, Selanders Foundation, P.O. Zetterling Foundation, U-CAN supported by SRA CancerUU, Uppsala University and Region Uppsala

Figure S1 from Weakly Supervised Deep Learning Predicts Immunotherapy Response in Solid Tumors Based on PD-L1 Expression

Performance overview of the model for predicting PD-L1 status and response to immunotherapy when trained on pan-cancer-VHIO cohort and validated in NSCLC-MSK. Area under the receiver operating characteristic (ROC) curves for the model to predict PD-L1 status (TPS≥1%) in the training (pan-cancer-VHIO cohort) (A) and in the test cohort (NSCLC-MSK cohort) (B) for the 5-folds cross-validation. All trained models were deployed in the test cohort. Kaplan-Meier curves for the predicted PD-L1 status (high/low) differentiates patients with longer Progression Free Survival to immunotherapy from patients with shorter survival in both pan-cancer-VHIO (C) and NSCLC-MSK cohort (D).</p

FIGURE 4 from Weakly Supervised Deep Learning Predicts Immunotherapy Response in Solid Tumors Based on PD-L1 Expression

Explainability for model performance in the pan-cancer-VHIO validation cohort. Overall performance of the model in the validation cohort (A). Distribution of true negative ratio (TNR), true positive ratio (TPR), false negative ratio (FNR), and false positive ratio (FPR) per tumor type in the pan-cancer-VHIO validation cohort (B). Distribution of predicted scores in the pan-cancer-VHIO cohort compared with continuous TPS (C) and CPS (D). Examples of false positives due to histologic differences from NSCLC: PD-L1–stained IHC image from melanoma tumor sample predicted as high PD-L1 by the model (predicted TPS ≥ 1%) with low PD-L1 expression in the tumor cells (TPS E). Examples of false negative due to low cellularity samples: PD-L1–stained IHC image from gastrointestinal tumor sample predicted as low PD-L1 by the model (predicted TPS F).</p

TABLE 1 from Weakly Supervised Deep Learning Predicts Immunotherapy Response in Solid Tumors Based on PD-L1 Expression

Clinical population characteristics for NSCLC-MSK and pan-cancer-VHIO cohort</p

Table S1 from Weakly Supervised Deep Learning Predicts Immunotherapy Response in Solid Tumors Based on PD-L1 Expression

Model classification performance for the training (NSCLC-MSK) and test (pan-cancer-VHIO) cohorts: Accuracy, Sensitivity, Specificity, Positive Predictive Value (PPV) and Negative Predictive Value (NPV). For the Pan-cancer cohort, the performance for the different tumor types is also reported.</p

FIGURE 1 from Weakly Supervised Deep Learning Predicts Immunotherapy Response in Solid Tumors Based on PD-L1 Expression

Overview of the study design. A, Population description of the NSCLC-MSK (training) and pan-cancer-VHIO (test) cohorts. B, Workflow of the attention-based MIL pipeline to classify PD-L1 status on IHC slides. The predicted PD-L1 status was investigated as predictive biomarker of response to immunotherapy.</p

FIGURE 2 from Weakly Supervised Deep Learning Predicts Immunotherapy Response in Solid Tumors Based on PD-L1 Expression

Performance overview of the model for predicting PD-L1 status and response to immunotherapy in NSCLC-MSK and pan-cancer-VHIO cohort. AUC curves for the model to predict PD-L1 status (TPS ≥ 1%) in the training (NSCLC-MSK cohort; A) and in the test cohort (pan-cancer-VHIO cohort; B) for the 5-folds cross-validation. All trained models were deployed in the test cohort. Kaplan–Meier curves for the predicted PD-L1 status (high/low) from the model differentiates patients with longer PFS to immunotherapy from patients with shorter survival in both NSCLC-MSK (C) and pan-cancer-VHIO cohort (D).</p

Figure S2 from Weakly Supervised Deep Learning Predicts Immunotherapy Response in Solid Tumors Based on PD-L1 Expression

Visualization of PanCytokeratin IHC staining, PD-L1 IHC staining and model attention heatmaps of a high (top) and low (bottom) PD-L1 score patients pan-cancer-VHIO cohort. PanCytokeratin IHC stained images differentiate tumor tissue (A, D). PD-L1 IHC stained images highlight tumor cells with high PD-L1 expression (B, E). Attention heatmaps indicate that the model is considering tumor areas for evaluating TPS (C,F)</p