Uncertainty driven pooling network for microvessel segmentation in routine histology images

Lymphovascular invasion (LVI) and tumor angiogenesis are correlated with metastasis, cancer recurrence and poor patient survival. In most of the cases, the LVI quantification and angiogenic analysis is based on microvessel segmentation and density estimation in immunohistochemically (IHC) stained tissues. However, in routine H&E stained images, the microvessels display a high level of heterogeneity in terms of size, shape, morphology and texture which makes microvessel segmentation a non-trivial task. Manual delineation of microvessels for biomarker analysis is labor-intensive, time consuming, irreproducible and can suffer from subjectivity among pathologists. Moreover, it is often beneficial to account for the uncertainty of a prediction when making a diagnosis. To address these challenges, we proposed a framework for microvessel segmentation in H&E stained histology images. The framework extends DeepLabV3+ by using an improved dice coefficient based custom loss function and also incorporating an uncertainty prediction mechanism. The proposed method uses an aligned Xception model, followed by atrous spatial pyramid pooling for feature extraction at multiple scales. This architecture counters the challenge of segmenting blood vessels of varying morphological appearance. To incorporate uncertainty, random transformations are introduced at test time for a superior segmentation result and simultaneous uncertainty map generation, highlighting ambiguous regions. The method is evaluated using 1167 images of size 512×512 pixels, extracted from 13 WSIs of oral squamous cell carcinoma (OSCC) tissue at 20x magnification. The proposed net-work achieves state-of-the-art performance compared to current semantic segmentation deep neural networks (FCN-8, U-Net, SegNet and DeepLabV3+)

A survey on artificial intelligence in histopathology image analysis

The increasing adoption of the whole slide image (WSI) technology in histopathology has dramatically transformed pathologists' workflow and allowed the use of computer systems in histopathology analysis. Extensive research in Artificial Intelligence (AI) with a huge progress has been conducted resulting in efficient, effective, and robust algorithms for several applications including cancer diagnosis, prognosis, and treatment. These algorithms offer highly accurate predictions but lack transparency, understandability, and actionability. Thus, explainable artificial intelligence (XAI) techniques are needed not only to understand the mechanism behind the decisions made by AI methods and increase user trust but also to broaden the use of AI algorithms in the clinical setting. From the survey of over 150 papers, we explore different AI algorithms that have been applied and contributed to the histopathology image analysis workflow. We first address the workflow of the histopathological process. We present an overview of various learning-based, XAI, and actionable techniques relevant to deep learning methods in histopathological imaging. We also address the evaluation of XAI methods and the need to ensure their reliability on the field

Deep Learning in Breast Cancer Imaging: A Decade of Progress and Future Directions

Breast cancer has reached the highest incidence rate worldwide among all malignancies since 2020. Breast imaging plays a significant role in early diagnosis and intervention to improve the outcome of breast cancer patients. In the past decade, deep learning has shown remarkable progress in breast cancer imaging analysis, holding great promise in interpreting the rich information and complex context of breast imaging modalities. Considering the rapid improvement in the deep learning technology and the increasing severity of breast cancer, it is critical to summarize past progress and identify future challenges to be addressed. In this paper, we provide an extensive survey of deep learning-based breast cancer imaging research, covering studies on mammogram, ultrasound, magnetic resonance imaging, and digital pathology images over the past decade. The major deep learning methods, publicly available datasets, and applications on imaging-based screening, diagnosis, treatment response prediction, and prognosis are described in detail. Drawn from the findings of this survey, we present a comprehensive discussion of the challenges and potential avenues for future research in deep learning-based breast cancer imaging.Comment: Survey, 41 page

Content-based Image Retrieval of Gigapixel Histopathology Scans: A Comparative Study of Convolution Neural Network, Local Binary Pattern, and Bag of visual Words

The state-of-the-art image analysis algorithms offer a unique opportunity to extract semantically meaningful features from medical images. The advantage of this approach is automation in terms of content-based image retrieval (CBIR) of medical images. Such an automation leads to more reliable diagnostic decisions by clinicians as the direct beneficiary of these algorithms. Digital pathology (DP), or whole slide imaging (WSI), is a new avenue for image-based diagnosis in histopathology. WSI technology enables the digitization of traditional glass slides to ultra high-resolution digital images (or digital slides). Digital slides are more commonly used for CBIR research than other modalities of medical images due to their enormous size, increasing adoption among hospitals, and their various benefits offered to pathologists (e.g., digital telepathology). Pathology laboratories are under constant pressure to meet increasingly complex demands from hospitals. Many diseases (such as cancer) continue to grow which creates a pressing need to utilize existing innovative machine learning schemes to harness the knowledge contained in digital slides for more effective and efficient histopathology. This thesis provides a qualitative assessment of three popular image analysis techniques, namely Local Binary Pattern (LBP), Bag of visual Words (BoW), and Convolution Neural Networks (CNN) in their abilities to extract the discriminative features from gigapixel histopathology images. LBP and BoW are well-established techniques used in different image analysis problems. Over the last 5-10 years, CNN has become a frequent research topic in computer vision. CNN offers a domain-agnostic approach for the automatic extraction of discriminative image features, used for either classification or retrieval purposes. Therefore, it is imperative that this thesis gives more emphasis to CNN as a viable approach for the analysis of DP images. A new dataset, Kimia Path24 is specially designed and developed to facilitate the research in classification and CBIR of DP images. Kimia Path24 is used to measure the quality of image features extracted from LBP, BoW, and CNN; resulting in the best accuracy values of 41.33%, 54.67%, and 56.98% respectively. The results are somewhat surprising, suggesting that the handcrafted feature extraction algorithm, i.e., LBP can reach very close to the deep features extracted from CNN. It is unanticipated, considering that CNN requires much more computational resources and efforts for designing and fine-tuning. One of the conclusions is that CNN needs to be trained for the problem with a large number of training images to realize its comprehensive benefits. However, there are many situations where large, balanced, and the labeled dataset is not available; one such area is histopathology at present

Heterogeneidad tumoral en imágenes PET-CT

Tesis inédita de la Universidad Complutense de Madrid, Facultad de Ciencias Físicas, Departamento de Estructura de la Materia, Física Térmica y Electrónica, leída el 28/01/2021Cancer is a leading cause of morbidity and mortality [1]. The most frequent cancers worldwide are non–small cell lung carcinoma (NSCLC) and breast cancer [2], being their management a challenging task [3]. Tumor diagnosis is usually made through biopsy [4]. However, medical imaging also plays an important role in diagnosis, staging, response to treatment, and recurrence assessment [5]. Tumor heterogeneity is recognized to be involved in cancer treatment failure, with worse clinical outcomes for highly heterogeneous tumors [6,7]. This leads to the existence of tumor sub-regions with different biological behavior (some more aggressive and treatment-resistant than others) [8-10]. Which are characterized by a different pattern of vascularization, vessel permeability, metabolism, cell proliferation, cell death, and other features, that can be measured by modern medical imaging techniques, including positron emission tomography/computed tomography (PET/CT) [10-12]. Thus, the assessment of tumor heterogeneity through medical images could allow the prediction of therapy response and long-term outcomes of patients with cancer [13]. PET/CT has become essential in oncology [14,15] and is usually evaluated through semiquantitative metabolic parameters, such as maximum/mean standard uptake value (SUVmax, SUVmean) or metabolic tumor volume (MTV), which are valuables as prognostic image-based biomarkers in several tumors [16-17], but these do not assess tumor heterogeneity. Likewise, fluorodeoxyglucose (18F-FDG) PET/CT is important to differentiate malignant from benign solitary pulmonary nodules (SPN), reducing so the number of patients who undergo unnecessary surgical biopsies. Several publications have shown that some quantitative image features, extracted from medical images, are suitable for diagnosis, tumor staging, the prognosis of treatment response, and long-term evolution of cancer patients [18-20]. The process of extracting and relating image features with clinical or biological variables is called “Radiomics” [9,20-24]. Radiomic parameters, such as textural features have been related directly to tumor heterogeneity [25]. This thesis investigated the relationships of the tumor heterogeneity, assessed by 18F-FDG-PET/CT texture analysis, with metabolic parameters and pathologic staging in patients with NSCLC, and explored the diagnostic performance of different metabolic, morphologic, and clinical criteria for classifying (malignant or not) of solitary pulmonary nodules (SPN). Furthermore, 18F-FDG-PET/CT radiomic features of patients with recurrent/metastatic breast cancer were used for constructing predictive models of response to the chemotherapy, based on an optimal combination of several feature selection and machine learning (ML) methods...El cáncer es una de las principales causas de morbilidad y mortalidad. Los más frecuentes son el carcinoma de pulmón de células no pequeñas (NSCLC) y el cáncer de mama, siendo su tratamiento un reto. El diagnóstico se suele realizar mediante biopsia. La heterogeneidad tumoral (HT) está implicada en el fracaso del tratamiento del cáncer, con peores resultados clínicos para tumores muy heterogéneos. Esta conduce a la existencia de subregiones tumorales con diferente comportamiento biológico (algunas más agresivas y resistentes al tratamiento); las cuales se caracterizan por diferentes patrones de vascularización, permeabilidad de los vasos sanguíneos, metabolismo, proliferación y muerte celular, que se pueden medir mediante imágenes médicas, incluida la tomografía por emisión de positrones/tomografía computarizada con fluorodesoxiglucosa (18F-FDG-PET/CT). La evaluación de la HT a través de imágenes médicas, podría mejorar la predicción de la respuesta al tratamiento y de los resultados a largo plazo, en pacientes con cáncer. La 18F-FDG-PET/CT es esencial en oncología, generalmente se evalúa con parámetros metabólicos semicuantitativos, como el valor de captación estándar máximo/medio (SUVmáx, SUVmedio) o el volumen tumoral metabólico (MTV), que tienen un gran valor pronóstico en varios tumores, pero no evalúan la HT. Asimismo, es importante para diferenciar los nódulos pulmonares solitarios (NPS) malignos de los benignos, reduciendo el número de pacientes que van a biopsias quirúrgicas innecesarias. Publicaciones recientes muestran que algunas características cuantitativas, extraídas de las imágenes médicas, son robustas para diagnóstico, estadificación, pronóstico de la respuesta al tratamiento y la evolución, de pacientes con cáncer. El proceso de extraer y relacionar estas características con variables clínicas o biológicas se denomina “Radiomica”. Algunos parámetros radiómicos, como la textura, se han relacionado directamente con la HT. Esta tesis investigó las relaciones entre HT, evaluada mediante análisis de textura (AT) de imágenes 18F-FDG-PET/CT, con parámetros metabólicos y estadificación patológica en pacientes con NSCLC, y exploró el rendimiento diagnóstico de diferentes criterios metabólicos, morfológicos y clínicos para la clasificación de NPS. Además, se usaron características radiómicas de imágenes 18F-FDG-PET/CT de pacientes con cáncer de mama recurrente/metastásico, para construir modelos predictivos de la respuesta a la quimioterapia, combinándose varios métodos de selección de características y aprendizaje automático (ML)...Fac. de Ciencias FísicasTRUEunpu

Imaging Based Prediction of Pathology in Adult Diffuse Glioma with Applications to Therapy and Prognosis

The overall aggressiveness of a glioma is measured by histologic and molecular analysis of tissue samples. However, the well-known spatial heterogeneity in gliomas limits the ability for clinicians to use that information to make spatially specific treatment decisions. Magnetic resonance imaging (MRI) visualizes and assesses the tumor. But, the exact degree to which MRI correlates with the actual underlying tissue characteristics is not known. In this work, we derive quantitative relationships between imaging and underlying pathology. These relations increase the value of MRI by allowing it to be a better surrogate for underlying pathology and they allow evaluation of the underlying biological heterogeneity via imaging. This provides an approach to answer questions about how tissue heterogeneity can affect prognosis. We estimated the local pathology within tumors using imaging data and stereotactically precise biopsy samples from an ongoing clinical imaging trial. From this data, we trained a random forest model to reliably predict tumor grade, proliferation, cellularity, and vascularity, representing tumor aggressiveness. We then made voxel-wise predictions to map the tumor heterogeneity and identify high-grade malignancy disease. Next, we used the previously trained models on a cohort of 1,850 glioma patients who previously underwent surgical resection. High contrast enhancement, proliferation, vascularity, and cellularity were associated with worse prognosis even after controlling for clinical factors. Patients that had substantial reduction in cellularity between preoperative and postoperative imaging (i.e. due to resection) also showed improved survival. We developed a clinically implementable model for predicting pathology and prognosis after surgery based on imaging. Results from imaging pathology correlations enhance our understanding of disease extent within glioma patients and the relationship between residual estimated pathology and outcome helps refine our knowledge of the interaction of tumor heterogeneity and prognosis

Automatic Esophageal Abnormality Detection and Classification

Esophageal cancer is counted as one of the deadliest cancers worldwide ranking the sixth among all types of cancers. Early esophageal cancer typically causes no symp- toms and mainly arises from overlooked/untreated premalignant abnormalities in the esophagus tube. Endoscopy is the main tool used for the detection of abnormalities, and the cell deformation stage is confirmed by taking biopsy samples. The process of detection and classification is considered challenging for several reasons such as; different types of abnormalities (including early cancer stages) can be located ran- domly throughout the esophagus tube, abnormal regions can have various sizes and appearances which makes it difficult to capture, and failure in discriminating between the columnar mucosa from the metaplastic epithelium. Although many studies have been conducted, it remains a challenging task and improving the accuracy of auto- matically classifying and detecting different esophageal abnormalities is an ongoing field. This thesis aims to develop novel automated methods for the detection and classification of the abnormal esophageal regions (precancerous and cancerous) from endoscopic images and videos. In this thesis, firstly, the abnormality stage of the esophageal cell deformation is clas- sified from confocal laser endomicroscopy (CLE) images. The CLE is an endoscopic tool that provides a digital pathology view of the esophagus cells. The classifica- tion is achieved by enhancing the internal features of the CLE image, using a novel enhancement filter that utilizes fractional integration and differentiation. Different imaging features including, Multi-Scale pyramid rotation LBP (MP-RLBP), gray level co-occurrence matrices (GLCM), fractal analysis, fuzzy LBP and maximally stable extremal regions (MSER), are calculated from the enhanced image to assure a robust classification result. The support vector machine (SVM) and random forest (RF) classifiers are employed to classify each image into its pathology stage. Secondly, we propose an automatic detection method to locate abnormality regions from high definition white light (HD-WLE) endoscopic images. We first investigate the performance of different deep learning detection methods on our dataset. Then we propose an approach that combines hand-designed Gabor features with extracted convolutional neural network features that are used by the Faster R-CNN to detect abnormal regions. Moreover, to further improve the detection performance, we pro- pose a novel two-input network named GFD-Faster RCNN. The proposed method generates a Gabor fractal image from the original endoscopic image using Gabor filters. Then features are learned separately from the endoscopic image and the gen- erated Gabor fractal image using the densely connected convolutional network to detect abnormal esophageal regions. Thirdly, we present a novel model to detect the abnormal regions from endoscopic videos. We design a 3D Sequential DenseConvLstm network to extract spatiotem- poral features from the input videos that are utilized by a region proposal network and ROI pooling layer to detect abnormality regions in each frame throughout the video. Additionally, we suggest an FS-CRF post-processing method that incorpor- ates the Conditional Random Field (CRF) on a frame-based level to recover missed abnormal regions in neighborhood frames within the same clip. The methods are evaluated on four datasets: (1) CLE dataset used for the classific- ation model, (2) Publicly available dataset named Kvasir, (3) MICCAI’15 Endovis challenge dataset, Both datasets (2) and (3) are used for the evaluation of detection model from endoscopic images. Finally, (4) Gastrointestinal Atlas dataset used for the evaluation of the video detection model. The experimental results demonstrate promising results of the different models and have outperformed the state-of-the-art methods

Quantitative PET-CT Perfusion Imaging of Prostate Cancer

Functional imaging of 18F-Fluorocholine PET holds promise in the detection of dominant prostatic lesions. Quantitative parameters from PET-CT Perfusion may be capable of measuring choline kinase activity, which could assist in identification of the dominant prostatic lesion for more accurate targeting of biopsies and radiation dose escalation. The objectives of this thesis are: 1) investigate the feasibility of using venous TACs in quantitative graphical analysis, and 2) develop and test a quantitative PET-CT Perfusion imaging technique that shows promise for identifying dominant prostatic lesions. Chapter 2 describes the effect of venous dispersion on distribution volume measurements with the Logan Plot. The dispersion of venous PET curves was simulated based on the arterio-venous transit time spectrum measured in a perfusion CT study of the human forearm. The analysis showed good agreement between distribution volume measurements produced by the arterial and venous TACs. Chapter 3 details the mathematical implementation of a linearized solution of the 3-Compartment kinetic model for hybrid PET-CT Perfusion imaging. A noise simulation determined the effect of incorporating CT perfusion parameters into the PET model on the accuracy and variability of measurements of the choline kinase activity. Results indicated that inclusion of CT perfusion parameters known a priori can significantly improve the accuracy and variability of imaging parameters measured with PET. Chapter 4 presents the implementation of PET-CT Perfusion imaging in a xenograft mouse model of human prostate cancer. Image-derived arterial TACs from the left ventricle were corrected for partial volume and spillover effects and validated by comparing to blood sampled curves. The PET-CT Perfusion imaging technique produced parametric maps of the choline kinase activity, k3. The results showed that the partial volume and spillover corrected arterial TACs agreed well with the blood sampled curves, and that k3max was significantly correlated with tumor volume, while SUV was not. In summary, this thesis establishes a solid foundation for future clinical research into 18F-fluorocholine PET imaging for the identification of dominant prostatic lesions. Quantitative PET-CT Perfusion imaging shows promise for assisting targeting of biopsy and radiation dose escalation of prostate cancer