Spatial Organization and Molecular Correlation of Tumor-Infiltrating Lymphocytes Using Deep Learning on Pathology Images

Beyond sample curation and basic pathologic characterization, the digitized H&E-stained images of TCGA samples remain underutilized. To highlight this resource, we present mappings of tumorinfiltrating lymphocytes (TILs) based on H&E images from 13 TCGA tumor types. These TIL maps are derived through computational staining using a convolutional neural network trained to classify patches of images. Affinity propagation revealed local spatial structure in TIL patterns and correlation with overall survival. TIL map structural patterns were grouped using standard histopathological parameters. These patterns are enriched in particular T cell subpopulations derived from molecular measures. TIL densities and spatial structure were differentially enriched among tumor types, immune subtypes, and tumor molecular subtypes, implying that spatial infiltrate state could reflect particular tumor cell aberration states. Obtaining spatial lymphocytic patterns linked to the rich genomic characterization of TCGA samples demonstrates one use for the TCGA image archives with insights into the tumor-immune microenvironment

Learning Deep Neural Networks for Enhanced Prostate Histological Image Analysis

In recent years, deep convolutional neural networks (CNNs) have shown promise for improving prostate cancer diagnosis by enabling quantitative histopathology through digital pathology. However, there are a number of factors that limit the widespread adoption and clinical utility of deep learning for digital pathology. One of these limitations is the requirement for large labelled training datasets which are expensive to construct due to limited availability of the requisite expertise. Additionally, digital pathology applications typically require the digitisation of histological slides at high magnifications. This process can be challenging especially when digitising large histological slides such as prostatectomies. This work studies and addresses these issues in two important applications of digital pathology: prostate nuclei detection and cell type classification. We study the performance of CNNs at different magnifications and demonstrate that it is possible to perform nuclei detection in low magnification prostate histopathology using CNNs with minimal loss in accuracy. We then study the training of prostate nuclei detectors in the small data setting and demonstrate that although it is possible to train nuclei detectors with minimal data, the models will be sensitive to hyperparameter choice and therefore may not generalise well. Instead, we show that pre-training the CNNs with colon histology data makes them more robust to hyperparameter choice. We then study the CNN performance for prostate cell type classification using supervised, transfer and semi-supervised learning in the small data setting. Our results show that transfer learning can be detrimental to performance but semi-supervised learning is able to provide significant improvements to the learning curve, allowing the training of neural networks with modest amounts of labelled data. We then propose a novel semi-supervised learning method called Deeply-supervised Exemplar CNNs and demonstrate their ability to improve the cell type classifier learning curves at a much better rate than previous semi-supervised neural network methods

Sensor, Signal, and Imaging Informatics in 2017.

Objective To summarize significant contributions to sensor, signal, and imaging informatics literature published in 2017.Methods PubMed® and Web of Science® were searched to identify the scientific publications published in 2017 that addressed sensors, signals, and imaging in medical informatics. Fifteen papers were selected by consensus as candidate best papers. Each candidate article was reviewed by section editors and at least two other external reviewers. The final selection of the four best papers was conducted by the editorial board of the International Medical Informatics Association (IMIA) Yearbook.Results The selected papers of 2017 demonstrate the important scientific advances in management and analysis of sensor, signal, and imaging information.ConclusionThe growth of signal and imaging data and the increasing power of machine learning techniques have engendered new opportunities for research in medical informatics. This synopsis highlights cutting-edge contributions to the science of Sensor, Signal, and Imaging Informatics

Intra-operative spectroscopic assessment of surgical margins during breast conserving surgery

Background: In over 20% of breast conserving operations, postoperative pathological assessment of the excised tissue reveals positive margins, requiring additional surgery. Current techniques for intra-operative assessment of tumor margins are insufficient in accuracy or resolution to reliably detect small tumors. There is a distinct need for a fast technique to accurately identify tumors smaller than 1 mm2 in large tissue surfaces within 30 min. Methods: Multi-modal spectral histopathology (MSH), a multimodal imaging technique combining tissue auto-fluorescence and Raman spectroscopy was used to detect microscopic residual tumor at the surface of the excised breast tissue. New algorithms were developed to optimally utilize auto-fluorescence images to guide Raman measurements and achieve the required detection accuracy over large tissue surfaces (up to 4 × 6.5 cm2). Algorithms were trained on 91 breast tissue samples from 65 patients. Results: Independent tests on 121 samples from 107 patients - including 51 fresh, whole excision specimens - detected breast carcinoma on the tissue surface with 95% sensitivity and 82% specificity. One surface of each uncut excision specimen was measured in 12–24 min. The combination of high spatial-resolution auto-fluorescence with specific diagnosis by Raman spectroscopy allows reliable detection even for invasive carcinoma or ductal carcinoma in situ smaller than 1 mm2. Conclusions: This study provides evidence that this multimodal approach could provide an objective tool for intra-operative assessment of breast conserving surgery margins, reducing the risk for unnecessary second operations

Pan-tumor CAnine cuTaneous Cancer Histology (CATCH) dataset

Due to morphological similarities, the differentiation of histologic sections of cutaneous tumors into individual subtypes can be challenging. Recently, deep learning-based approaches have proven their potential for supporting pathologists in this regard. However, many of these supervised algorithms require a large amount of annotated data for robust development. We present a publicly available dataset of 350 whole slide images of seven different canine cutaneous tumors complemented by 12,424 polygon annotations for 13 histologic classes, including seven cutaneous tumor subtypes. In inter-rater experiments, we show a high consistency of the provided labels, especially for tumor annotations. We further validate the dataset by training a deep neural network for the task of tissue segmentation and tumor subtype classification. We achieve a class-averaged Jaccard coefficient of 0.7047, and 0.9044 for tumor in particular. For classification, we achieve a slide-level accuracy of 0.9857. Since canine cutaneous tumors possess various histologic homologies to human tumors the added value of this dataset is not limited to veterinary pathology but extends to more general fields of application

Differentiation of Metabolically Distinct Areas within Head and Neck Region using Dynamic 18F-FDG Positron Emission Tomography Imaging

Positron Emission Tomography (PET) using 18F-FDG is playing a vital role in the diagnosis and treatment planning of cancer. However, the most widely used radiotracer, 18F-FDG, is not specific for tumours and can also accumulate in inflammatory lesions as well as normal physiologically active tissues making diagnosis and treatment planning complicated for the physicians. Malignant, inflammatory and normal tissues are known to have different pathways for glucose metabolism which could possibly be evident from different characteristics of the time activity curves from a dynamic PET acquisition protocol. Therefore, we aimed to develop new image analysis methods, for PET scans of the head and neck region, which could differentiate between inflammation, tumour and normal tissues using this functional information within these radiotracer uptake areas. We developed different dynamic features from the time activity curves of voxels in these areas and compared them with the widely used static parameter, SUV, using Gaussian Mixture Model algorithm as well as K-means algorithm in order to assess their effectiveness in discriminating metabolically different areas. Moreover, we also correlated dynamic features with other clinical metrics obtained independently of PET imaging. The results show that some of the developed features can prove to be useful in differentiating tumour tissues from inflammatory regions and some dynamic features also provide positive correlations with clinical metrics. If these proposed methods are further explored then they can prove to be useful in reducing false positive tumour detections and developing real world applications for tumour diagnosis and contouring.Siirretty Doriast

Role of AI and digital pathology for colorectal immuno-oncology

Immunotherapy deals with therapeutic interventions to arrest the progression of tumours using the immune system. These include checkpoint inhibitors, T-cell manipulation, cytokines, oncolytic viruses and tumour vaccines. In this paper, we present a survey of the latest developments on immunotherapy in colorectal cancer (CRC) and the role of artificial intelligence (AI) in this context. Among these, microsatellite instability (MSI) is perhaps the most popular IO biomarker globally. We first discuss the MSI status of tumours, its implications for patient management, and its relationship to immune response. In recent years, several aspiring studies have used AI to predict the MSI status of patients from digital whole-slide images (WSIs) of routine diagnostic slides. We present a survey of AI literature on the prediction of MSI and tumour mutation burden from digitised WSIs of haematoxylin and eosin-stained diagnostic slides. We discuss AI approaches in detail and elaborate their contributions, limitations and key takeaways to drive future research. We further expand this survey to other IO-related biomarkers like immune cell infiltrates and alternate data modalities like immunohistochemistry and gene expression. Finally, we underline possible future directions in immunotherapy for CRC and promise of AI to accelerate this exploration for patient benefits