Spatial Organization and Molecular Correlation of Tumor-Infiltrating Lymphocytes Using Deep Learning on Pathology Images

Beyond sample curation and basic pathologic characterization, the digitized H&E-stained images of TCGA samples remain underutilized. To highlight this resource, we present mappings of tumorinfiltrating lymphocytes (TILs) based on H&E images from 13 TCGA tumor types. These TIL maps are derived through computational staining using a convolutional neural network trained to classify patches of images. Affinity propagation revealed local spatial structure in TIL patterns and correlation with overall survival. TIL map structural patterns were grouped using standard histopathological parameters. These patterns are enriched in particular T cell subpopulations derived from molecular measures. TIL densities and spatial structure were differentially enriched among tumor types, immune subtypes, and tumor molecular subtypes, implying that spatial infiltrate state could reflect particular tumor cell aberration states. Obtaining spatial lymphocytic patterns linked to the rich genomic characterization of TCGA samples demonstrates one use for the TCGA image archives with insights into the tumor-immune microenvironment

Pitfalls in machine learning-based assessment of tumor-infiltrating lymphocytes in breast cancer: a report of the international immuno-oncology biomarker working group

The clinical significance of the tumor-immune interaction in breast cancer is now established, and tumor-infiltrating lymphocytes (TILs) have emerged as predictive and prognostic biomarkers for patients with triple-negative (estrogen receptor, progesterone receptor, and HER2-negative) breast cancer and HER2-positive breast cancer. How computational assessments of TILs might complement manual TIL assessment in trial and daily practices is currently debated. Recent efforts to use machine learning (ML) to automatically evaluate TILs have shown promising results. We review state-of-the-art approaches and identify pitfalls and challenges of automated TIL evaluation by studying the root cause of ML discordances in comparison to manual TIL quantification. We categorize our findings into four main topics: (1) technical slide issues, (2) ML and image analysis aspects, (3) data challenges, and (4) validation issues. The main reason for discordant assessments is the inclusion of false-positive areas or cells identified by performance on certain tissue patterns or design choices in the computational implementation. To aid the adoption of ML for TIL assessment, we provide an in-depth discussion of ML and image analysis, including validation issues that need to be considered before reliable computational reporting of TILs can be incorporated into the trial and routine clinical management of patients with triple-negative breast cancer. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland

Pitfalls in machine learning‐based assessment of tumor‐infiltrating lymphocytes in breast cancer: a report of the international immuno‐oncology biomarker working group

The clinical significance of the tumor-immune interaction in breast cancer (BC) has been well established, and tumor-infiltrating lymphocytes (TILs) have emerged as a predictive and prognostic biomarker for patients with triple-negative (estrogen receptor, progesterone receptor, and HER2 negative) breast cancer (TNBC) and HER2-positive breast cancer. How computational assessment of TILs can complement manual TIL-assessment in trial- and daily practices is currently debated and still unclear. Recent efforts to use machine learning (ML) for the automated evaluation of TILs show promising results. We review state-of-the-art approaches and identify pitfalls and challenges by studying the root cause of ML discordances in comparison to manual TILs quantification. We categorize our findings into four main topics; (i) technical slide issues, (ii) ML and image analysis aspects, (iii) data challenges, and (iv) validation issues. The main reason for discordant assessments is the inclusion of false-positive areas or cells identified by performance on certain tissue patterns, or design choices in the computational implementation. To aid the adoption of ML in TILs assessment, we provide an in-depth discussion of ML and image analysis including validation issues that need to be considered before reliable computational reporting of TILs can be incorporated into the trial- and routine clinical management of patients with TNBC

Quantitative Analysis of Immune Infiltrates in Primary Melanoma.

Novel methods to analyze the tumor microenvironment (TME) are urgently needed to stratify melanoma patients for adjuvant immunotherapy. Tumor-infiltrating lymphocyte (TIL) analysis, by conventional pathologic methods, is predictive but is insufficiently precise for clinical application. Quantitative multiplex immunofluorescence (qmIF) allows for evaluation of the TME using multiparameter phenotyping, tissue segmentation, and quantitative spatial analysis (qSA). Given that CD3+CD8+ cytotoxic lymphocytes (CTLs) promote antitumor immunity, whereas CD68+ macrophages impair immunity, we hypothesized that quantification and spatial analysis of macrophages and CTLs would correlate with clinical outcome. We applied qmIF to 104 primary stage II to III melanoma tumors and found that CTLs were closer in proximity to activated (CD68+HLA-DR+) macrophages than nonactivated (CD68+HLA-DR-) macrophages (P < 0.0001). CTLs were further in proximity from proliferating SOX10+ melanoma cells than nonproliferating ones (P < 0.0001). In 64 patients with known cause of death, we found that high CTL and low macrophage density in the stroma (P = 0.0038 and P = 0.0006, respectively) correlated with disease-specific survival (DSS), but the correlation was less significant for CTL and macrophage density in the tumor (P = 0.0147 and P = 0.0426, respectively). DSS correlation was strongest for stromal HLA-DR+ CTLs (P = 0.0005). CTL distance to HLA-DR- macrophages associated with poor DSS (P = 0.0016), whereas distance to Ki67- tumor cells associated inversely with DSS (P = 0.0006). A low CTL/macrophage ratio in the stroma conferred a hazard ratio (HR) of 3.719 for death from melanoma and correlated with shortened overall survival (OS) in the complete 104 patient cohort by Cox analysis (P = 0.009) and merits further development as a biomarker for clinical application

Deep-Learning–Based Characterization of Tumor-Infiltrating Lymphocytes in Breast Cancers From Histopathology Images and Multiomics Data

Purpose: Tumor-infiltrating lymphocytes (TILs) and their spatial characterizations on whole-slide images (WSIs) of histopathology sections have become crucial in diagnosis, prognosis, and treatment response prediction for different cancers. However, fully automatic assessment of TILs on WSIs currently remains a great challenge because of the heterogeneity and large size of WSIs. We present an automatic pipeline based on a cascade-training U-net to generate high-resolution TIL maps on WSIs. Methods: We present global cell-level TIL maps and 43 quantitative TIL spatial image features for 1,000 WSIs of The Cancer Genome Atlas patients with breast cancer. For more specific analysis, all the patients were divided into three subtypes, namely, estrogen receptor (ER)-positive, ER-negative, and triple-negative groups. The associations between TIL scores and gene expression and somatic mutation were examined separately in three breast cancer subtypes. Both univariate and multivariate survival analyses were performed on 43 TIL image features to examine the prognostic value of TIL spatial patterns in different breast cancer subtypes. Results: The TIL score was in strong association with immune response pathway and genes (eg, programmed death-1 and CLTA4). Different breast cancer subtypes showed TIL score in association with mutations from different genes suggesting that different genetic alterations may lead to similar phenotypes. Spatial TIL features that represent density and distribution of TIL clusters were important indicators of the patient outcomes. Conclusion: Our pipeline can facilitate computational pathology-based discovery in cancer immunology and research on immunotherapy. Our analysis results are available for the research community to generate new hypotheses and insights on breast cancer immunology and development

Assessment of immunological features in muscle-invasive bladder cancer prognosis using ensemble learning

Funding: This research received financial support from Definiens GmbH and the Industrial Centre for AI Research in digital Diagnostics (iCAIRD) which is funded by Innovate UK on behalf of UK Research and Innovation (UKRI) [project number: 104690].The clinical staging and prognosis of muscle-invasive bladder cancer (MIBC) routinely includes the assessment of patient tissue samples by a pathologist. Recent studies corroborate the importance of image analysis in identifying and quantifying immunological markers from tissue samples that can provide further insight into patient prognosis. In this paper, we apply multiplex immunofluorescence to MIBC tissue sections to capture whole-slide images and quantify potential prognostic markers related to lymphocytes, macrophages, tumour buds, and PD-L1. We propose a machine-learning-based approach for the prediction of 5 year prognosis with different combinations of image, clinical, and spatial features. An ensemble model comprising several functionally different models successfully stratifies MIBC patients into two risk groups with high statistical significance (p value < 1×10−5). Critical to improving MIBC survival rates, our method correctly classifies 71.4% of the patients who succumb to MIBC, which is significantly more than the 28.6% of the current clinical gold standard, the TNM staging system.Publisher PDFPeer reviewe

Machine learning methods for histopathological image analysis

Abundant accumulation of digital histopathological images has led to the increased demand for their analysis, such as computer-aided diagnosis using machine learning techniques. However, digital pathological images and related tasks have some issues to be considered. In this mini-review, we introduce the application of digital pathological image analysis using machine learning algorithms, address some problems specific to such analysis, and propose possible solutions.Comment: 23 pages, 4 figure

Heterogeneity of circulating tumour cell-associated genomic gains in breast cancer and its association with the host immune response.

Tumor cells that preferentially enter circulation include the precursors of metastatic cancer. Previously, we characterized circulating tumor cells (CTC) from patients with breast cancer and identified a signature of genomic regions with recurrent copy-number gains. Through FISH, we now show that these CTC-associated regions are detected within the matched untreated primary tumors of these patients (21% to 69%, median 55.5%, n = 19). Furthermore, they are more prevalent in the metastases of patients who died from breast cancer after multiple rounds of treatment (70% to 100%, median 93%, samples n = 41). Diversity indices revealed that higher spatial heterogeneity for these regions within primary tumors is associated with increased dissemination and metastasis. An identified subclone with multiple regions gained (MRG clone) was enriched in a posttreatment primary breast carcinoma as well as multiple metastatic tumors and local breast recurrences obtained at autopsy, indicative of a distinct early subclone with the capability to resist multiple lines of treatment and eventually cause death. In addition, multiplex immunofluorescence revealed that tumor heterogeneity is significantly associated with the degree of infiltration of B lymphocytes in triple-negative breast cancer, a subtype with a large immune component. Collectively, these data reveal the functional potential of genetic subclones that comprise heterogeneous primary breast carcinomas and are selected for in CTCs and posttreatment breast cancer metastases. In addition, they uncover a relationship between tumor heterogeneity and host immune response in the tumor microenvironment. SIGNIFICANCE: As breast cancers progress, they become more heterogeneous for multiple regions amplified in circulating tumor cells, and intratumoral spatial heterogeneity is associated with the immune landscape