Longitudinal Brain Tumor Tracking, Tumor Grading, and Patient Survival Prediction Using MRI

This work aims to develop novel methods for brain tumor classification, longitudinal brain tumor tracking, and patient survival prediction. Consequently, this dissertation proposes three tasks. First, we develop a framework for brain tumor segmentation prediction in longitudinal multimodal magnetic resonance imaging (mMRI) scans, comprising two methods: feature fusion and joint label fusion (JLF). The first method fuses stochastic multi-resolution texture features with tumor cell density features, in order to obtain tumor segmentation predictions in follow-up scans from a baseline pre-operative timepoint. The second method utilizes JLF to combine segmentation labels obtained from (i) the stochastic texture feature-based and Random Forest (RF)-based tumor segmentation method; and (ii) another state-of-the-art tumor growth and segmentation method known as boosted Glioma Image Segmentation and Registration (GLISTRboost, or GB). With the advantages of feature fusion and label fusion, we achieve state-of-the-art brain tumor segmentation prediction. Second, we propose a deep neural network (DNN) learning-based method for brain tumor type and subtype grading using phenotypic and genotypic data, following the World Health Organization (WHO) criteria. In addition, the classification method integrates a cellularity feature which is derived from the morphology of a pathology image to improve classification performance. The proposed method achieves state-of-the-art performance for tumor grading following the new CNS tumor grading criteria. Finally, we investigate brain tumor volume segmentation, tumor subtype classification, and overall patient survival prediction, and then we propose a new context- aware deep learning method, known as the Context Aware Convolutional Neural Network (CANet). Using the proposed method, we participated in the Multimodal Brain Tumor Segmentation Challenge 2019 (BraTS 2019) for brain tumor volume segmentation and overall survival prediction tasks. In addition, we also participated in the Radiology-Pathology Challenge 2019 (CPM-RadPath 2019) for Brain Tumor Subtype Classification, organized by the Medical Image Computing & Computer Assisted Intervention (MICCAI) Society. The online evaluation results show that the proposed methods offer competitive performance from their use of state-of-the-art methods in tumor volume segmentation, promising performance on overall survival prediction, and state-of-the-art performance on tumor subtype classification. Moreover, our result was ranked second place in the testing phase of the CPM-RadPath 2019

The clinical utility of prostate cancer heterogeneity using texture analysis of multiparametric MRI

Purpose To determine if multiparametric MRI (mpMRI) derived filtration-histogram based texture analysis (TA) can differentiate between different Gleason scores (GS) and the D’Amico risk in prostate cancer. Methods We retrospectively studied patients whose pre-operative 1.5T mpMRI had shown a visible tumour and who subsequently underwent radical prostatectomy (RP). Guided by tumour location from the histopathology report, we drew a region of interest around the dominant visible lesion on a single axial slice on the T2, Apparent Diffusion Coefficient (ADC) map and early arterial phase post-contrast T1 image. We then performed TA with a filtration-histogram software (TexRAD -Feedback Medical Ltd, Cambridge, UK). We correlated GS and D’Amico risk with texture using the Spearman’s rank correlation test. Results We had 26 RP patients with an MR-visible tumour. Mean of positive pixels (MPP) on ADC showed a significant negative correlation with GS at coarse texture scales. MPP showed a significant negative correlation with GS without filtration and with medium filtration. MRI contrast texture without filtration showed a significant, negative correlation with D’Amico score. MR T2 texture showed a significant, negative correlation with the D’Amico risk, particularly at textures without filtration, medium texture scales and coarse texture scales. Conclusion ADC map mpMRI TA correlated negatively with GS, and T2 and post-contrast images with the D’Amico risk score. These associations may allow for better assessment of disease prognosis and a non-invasive method of follow-up for patients on surveillance. Further, identifying clinically significant prostate cancer is essential to reduce harm from over-diagnosis and over-treatment

Deep Learning in Medical Image Analysis

The accelerating power of deep learning in diagnosing diseases will empower physicians and speed up decision making in clinical environments. Applications of modern medical instruments and digitalization of medical care have generated enormous amounts of medical images in recent years. In this big data arena, new deep learning methods and computational models for efficient data processing, analysis, and modeling of the generated data are crucially important for clinical applications and understanding the underlying biological process. This book presents and highlights novel algorithms, architectures, techniques, and applications of deep learning for medical image analysis

Learning Deep Neural Networks for Enhanced Prostate Histological Image Analysis

In recent years, deep convolutional neural networks (CNNs) have shown promise for improving prostate cancer diagnosis by enabling quantitative histopathology through digital pathology. However, there are a number of factors that limit the widespread adoption and clinical utility of deep learning for digital pathology. One of these limitations is the requirement for large labelled training datasets which are expensive to construct due to limited availability of the requisite expertise. Additionally, digital pathology applications typically require the digitisation of histological slides at high magnifications. This process can be challenging especially when digitising large histological slides such as prostatectomies. This work studies and addresses these issues in two important applications of digital pathology: prostate nuclei detection and cell type classification. We study the performance of CNNs at different magnifications and demonstrate that it is possible to perform nuclei detection in low magnification prostate histopathology using CNNs with minimal loss in accuracy. We then study the training of prostate nuclei detectors in the small data setting and demonstrate that although it is possible to train nuclei detectors with minimal data, the models will be sensitive to hyperparameter choice and therefore may not generalise well. Instead, we show that pre-training the CNNs with colon histology data makes them more robust to hyperparameter choice. We then study the CNN performance for prostate cell type classification using supervised, transfer and semi-supervised learning in the small data setting. Our results show that transfer learning can be detrimental to performance but semi-supervised learning is able to provide significant improvements to the learning curve, allowing the training of neural networks with modest amounts of labelled data. We then propose a novel semi-supervised learning method called Deeply-supervised Exemplar CNNs and demonstrate their ability to improve the cell type classifier learning curves at a much better rate than previous semi-supervised neural network methods

Tumor cell load and heterogeneity estimation from diffusion-weighted MRI calibrated with histological data: an example from lung cancer

Producción CientíficaDiffusion-weighted magnetic resonance imaging (DWI) is a key non-invasive imaging technique for cancer diagnosis and tumor treatment assessment, reflecting Brownian movement of water molecules in tissues. Since densely packed cells restrict molecule mobility, tumor tissues produce usually higher signal (a.k.a. less attenuated signal) on isotropic maps compared with normal tissues. However, no general quantitative relation between DWI data and the cell density has been established. In order to link low-resolution clinical cross-sectional data with high-resolution histological information, we developed an image processing and analysis chain, which was used to study the correlation between the diffusion coefficient (D value) estimated from DWI and tumor cellularity from serial histological slides of a resected non-small cell lung cancer tumor. Color deconvolution followed by cell nuclei segmentation was performed on digitized histological images to determine local and cell-type specific 2d (two-dimensional) densities. From these, the 3d cell density was inferred by a model-based sampling technique, which is necessary for the calculation of local and global 3d tumor cell count. Next, DWI sequence information was overlaid with high-resolution CT data and the resected histology using prominent anatomical hallmarks for co-registration of histology tissue blocks and non-invasive imaging modalities' data. The integration of cell numbers information and DWI data derived from different tumor areas revealed a clear negative correlation between cell density and D value. Importantly, spatial tumor cell density can be calculated based on DWI data. In summary, our results demonstrate that tumor cell count and heterogeneity can be predicted from DWI data, which may open new opportunities for personalized diagnosis and therapy optimization

Domain Generalization in Computational Pathology: Survey and Guidelines

Deep learning models have exhibited exceptional effectiveness in Computational Pathology (CPath) by tackling intricate tasks across an array of histology image analysis applications. Nevertheless, the presence of out-of-distribution data (stemming from a multitude of sources such as disparate imaging devices and diverse tissue preparation methods) can cause \emph{domain shift} (DS). DS decreases the generalization of trained models to unseen datasets with slightly different data distributions, prompting the need for innovative \emph{domain generalization} (DG) solutions. Recognizing the potential of DG methods to significantly influence diagnostic and prognostic models in cancer studies and clinical practice, we present this survey along with guidelines on achieving DG in CPath. We rigorously define various DS types, systematically review and categorize existing DG approaches and resources in CPath, and provide insights into their advantages, limitations, and applicability. We also conduct thorough benchmarking experiments with 28 cutting-edge DG algorithms to address a complex DG problem. Our findings suggest that careful experiment design and CPath-specific Stain Augmentation technique can be very effective. However, there is no one-size-fits-all solution for DG in CPath. Therefore, we establish clear guidelines for detecting and managing DS depending on different scenarios. While most of the concepts, guidelines, and recommendations are given for applications in CPath, we believe that they are applicable to most medical image analysis tasks as well.Comment: Extended Versio

High Resolution Optical Imaging Techniques for Rapid Assessment of Breast Cancer

Breast cancer is the most prevalent and deadly cancer among women worldwide. The current standard for breast lesion diagnosis is histologic assessment with hematoxylin and eosin (H&E) staining. Histology has high diagnostic accuracy, but requires extensive time and resources to perform. The objective of this work was to improve diagnosis of early breast cancers by developing approaches to rapidly image and characterize neoplastic tissue and the tumor microenvironment in high resolution optical images. Confocal fluorescence microscopy can image optical sections of tissue without the need for extensive tissue processing. Three studies were performed to evaluate if confocal microscopy images contain sufficient information to identify neoplasia in breast tissue. In a 31 patient study, five pathologists identified neoplasia with high accuracy in confocal and histologic images. In another study, an expert pathologist estimated tumor cellularity in core biopsies with moderate agreement between confocal and histologic images. In a third study, an expert pathologist assigned diagnoses and grades to neoplastic tissue in confocal and histologic images. Limitations of these studies include recruitment of patients at a single center and data assessment by a single reader in two of three studies. Visual assessment for cancer diagnosis is limited by the potential for inter- and intra-observer error. Using a computerized algorithm to segment and quantify architectural features of breast ducts and nuclei, a decision-tree model was developed that classified confocal images of breast tissue sites as neoplastic or non-neoplastic with an overall accuracy of 90%. Another computerized algorithm was developed to segment adipocytes in confocal images and results showed significant differences in phenotypic properties of adipocytes adjacent to neoplastic and non-neoplastic tissue. High resolution microendoscopy (HRME) can be used to rapidly acquire images at a lower cost than confocal microscopy. In a study evaluating HRME and two approaches to improve image contrast, results demonstrated that HRME with structured illumination yields images with high contrast relative to HRME with standard illumination. The unique contribution of these results is the characterization of qualitative and quantitative criteria to evaluate breast tissue and classify neoplasia in optical images, although recognition of invasive lobular carcinoma was limited. The criteria developed in this research may be applied to further development of techniques for objective classification and diagnosis of breast cancer in optical images.

Patient-specific, mechanistic models of tumor growth incorporating artificial intelligence and big data

Despite the remarkable advances in cancer diagnosis, treatment, and management that have occurred over the past decade, malignant tumors remain a major public health problem. Further progress in combating cancer may be enabled by personalizing the delivery of therapies according to the predicted response for each individual patient. The design of personalized therapies requires patient-specific information integrated into an appropriate mathematical model of tumor response. A fundamental barrier to realizing this paradigm is the current lack of a rigorous, yet practical, mathematical theory of tumor initiation, development, invasion, and response to therapy. In this review, we begin by providing an overview of different approaches to modeling tumor growth and treatment, including mechanistic as well as data-driven models based on ``big data" and artificial intelligence. Next, we present illustrative examples of mathematical models manifesting their utility and discussing the limitations of stand-alone mechanistic and data-driven models. We further discuss the potential of mechanistic models for not only predicting, but also optimizing response to therapy on a patient-specific basis. We then discuss current efforts and future possibilities to integrate mechanistic and data-driven models. We conclude by proposing five fundamental challenges that must be addressed to fully realize personalized care for cancer patients driven by computational models

Advanced Computational Methods for Oncological Image Analysis

[Cancer is the second most common cause of death worldwide and encompasses highly variable clinical and biological scenarios. Some of the current clinical challenges are (i) early diagnosis of the disease and (ii) precision medicine, which allows for treatments targeted to specific clinical cases. The ultimate goal is to optimize the clinical workflow by combining accurate diagnosis with the most suitable therapies. Toward this, large-scale machine learning research can define associations among clinical, imaging, and multi-omics studies, making it possible to provide reliable diagnostic and prognostic biomarkers for precision oncology. Such reliable computer-assisted methods (i.e., artificial intelligence) together with clinicians’ unique knowledge can be used to properly handle typical issues in evaluation/quantification procedures (i.e., operator dependence and time-consuming tasks). These technical advances can significantly improve result repeatability in disease diagnosis and guide toward appropriate cancer care. Indeed, the need to apply machine learning and computational intelligence techniques has steadily increased to effectively perform image processing operations—such as segmentation, co-registration, classification, and dimensionality reduction—and multi-omics data integration.