Using spectral imaging for the analysis of abnormalities for colorectal cancer: When is it helpful?

© 2018 Awan et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. The spectral imaging technique has been shown to provide more discriminative information than the RGB images and has been proposed for a range of problems. There are many studies demonstrating its potential for the analysis of histopathology images for abnormality detection but there have been discrepancies among previous studies as well. Many multispectral based methods have been proposed for histopathology images but the significance of the use of whole multispectral cube versus a subset of bands or a single band is still arguable. We performed comprehensive analysis using individual bands and different subsets of bands to determine the effectiveness of spectral information for determining the anomaly in colorectal images. Our multispectral colorectal dataset consists of four classes, each represented by infra-red spectrum bands in addition to the visual spectrum bands. We performed our analysis of spectral imaging by stratifying the abnormalities using both spatial and spectral information. For our experiments, we used a combination of texture descriptors with an ensemble classification approach that performed best on our dataset. We applied our method to another dataset and got comparable results with those obtained using the state-of-the-art method and convolutional neural network based method. Moreover, we explored the relationship of the number of bands with the problem complexity and found that higher number of bands is required for a complex task to achieve improved performance. Our results demonstrate a synergy between infra-red and visual spectrum by improving the classification accuracy (by 6%) on incorporating the infra-red representation. We also highlight the importance of how the dataset should be divided into training and testing set for evaluating the histopathology image-based approaches, which has not been considered in previous studies on multispectral histopathology images.This publication was made possible using a grant from the Qatar National Research Fund through National Priority Research Program (NPRP) No. 6-249-1-053. The content of this publication are solely the responsibility of the authors and do not necessarily represent the official views of the Qatar National Research Fund or Qatar University

Context-aware convolutional neural network for grading of colorectal cancer histology images

Digital histology images are amenable to the application of convolutional neural networks (CNNs) for analysis due to the sheer size of pixel data present in them. CNNs are generally used for representation learning from small image patches (e.g. 224 × 224) extracted from digital histology images due to computational and memory constraints. However, this approach does not incorporate high-resolution contextual information in histology images. We propose a novel way to incorporate a larger context by a context-aware neural network based on images with a dimension of 1792 × 1792 pixels. The proposed framework first encodes the local representation of a histology image into high dimensional features then aggregates the features by considering their spatial organization to make a final prediction. We evaluated the proposed method on two colorectal cancer datasets for the task of cancer grading. Our method outperformed the traditional patch-based approaches, problem-specific methods, and existing context-based methods. We also presented a comprehensive analysis of different variants of the proposed method

Deep learning based digital cell profiles for risk stratification of urine cytology images

Urine cytology is a test for the detection of high-grade bladder cancer. In clinical practice, the pathologist would manually scan the sample under the microscope to locate atypical and malignant cells. They would assess the morphology of these cells to make a diagnosis. Accurate identification of atypical and malignant cells in urine cytology is a challenging task and is an essential part of identifying different diagnosis with low-risk and high-risk malignancy. Computer-assisted identification of malignancy in urine cytology can be complementary to the clinicians for treatment management and in providing advice for carrying out further tests. In this study, we presented a method for identifying atypical and malignant cells followed by their profiling to predict the risk of diagnosis automatically. For cell detection and classification, we employed two different deep learning-based approaches. Based on the best performing network predictions at the cell level, we identified low-risk and high-risk cases using the count of atypical cells and the total count of atypical and malignant cells. The area under the receiver operating characteristic (ROC) curve shows that a total count of atypical and malignant cells is comparably better at diagnosis as compared to the count of malignant cells only. We obtained area under the ROC curve with the count of malignant cells and the total count of atypical and malignant cells as 0.81 and 0.83, respectively. Our experiments also demonstrate that the digital risk could be a better predictor of the final histopathology-based diagnosis. We also analyzed the variability in annotations at both cell and whole slide image level and also explored the possible inherent rationales behind this variability

Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer

Importance Application of deep learning algorithms to whole-slide pathology images can potentially improve diagnostic accuracy and efficiency. Objective Assess the performance of automated deep learning algorithms at detecting metastases in hematoxylin and eosin–stained tissue sections of lymph nodes of women with breast cancer and compare it with pathologists’ diagnoses in a diagnostic setting. Design, Setting, and Participants Researcher challenge competition (CAMELYON16) to develop automated solutions for detecting lymph node metastases (November 2015-November 2016). A training data set of whole-slide images from 2 centers in the Netherlands with (n = 110) and without (n = 160) nodal metastases verified by immunohistochemical staining were provided to challenge participants to build algorithms. Algorithm performance was evaluated in an independent test set of 129 whole-slide images (49 with and 80 without metastases). The same test set of corresponding glass slides was also evaluated by a panel of 11 pathologists with time constraint (WTC) from the Netherlands to ascertain likelihood of nodal metastases for each slide in a flexible 2-hour session, simulating routine pathology workflow, and by 1 pathologist without time constraint (WOTC). Exposures Deep learning algorithms submitted as part of a challenge competition or pathologist interpretation. Main Outcomes and Measures The presence of specific metastatic foci and the absence vs presence of lymph node metastasis in a slide or image using receiver operating characteristic curve analysis. The 11 pathologists participating in the simulation exercise rated their diagnostic confidence as definitely normal, probably normal, equivocal, probably tumor, or definitely tumor. Results The area under the receiver operating characteristic curve (AUC) for the algorithms ranged from 0.556 to 0.994. The top-performing algorithm achieved a lesion-level, true-positive fraction comparable with that of the pathologist WOTC (72.4% [95% CI, 64.3%-80.4%]) at a mean of 0.0125 false-positives per normal whole-slide image. For the whole-slide image classification task, the best algorithm (AUC, 0.994 [95% CI, 0.983-0.999]) performed significantly better than the pathologists WTC in a diagnostic simulation (mean AUC, 0.810 [range, 0.738-0.884]; P < .001). The top 5 algorithms had a mean AUC that was comparable with the pathologist interpreting the slides in the absence of time constraints (mean AUC, 0.960 [range, 0.923-0.994] for the top 5 algorithms vs 0.966 [95% CI, 0.927-0.998] for the pathologist WOTC). Conclusions and Relevance In the setting of a challenge competition, some deep learning algorithms achieved better diagnostic performance than a panel of 11 pathologists participating in a simulation exercise designed to mimic routine pathology workflow; algorithm performance was comparable with an expert pathologist interpreting whole-slide images without time constraints. Whether this approach has clinical utility will require evaluation in a clinical setting

Deep learning based frameworks for patient selection

Recently, deep learning (DL) has become a spearhead for solving many problems in the computer vision domain, with computational pathology (CP) has no exception. In the CP domain, it is widely used for histological assessment of tissue for diagnosis and prognosis of cancer patients. The research community has developed an abundance of DL based CP tools, reporting state-of-the-art results, for many diverse applications. In near future, we can envisage better tools on the way forward to the clinical workflow to assist pathologists in making diagnostic and anti-cancer therapeutic decisions. In this thesis, we develop DL based frameworks defining eligibility criteria for selecting patients of two different types of cancers: bladder and colorectal cancers. We develop our first framework with the main goal to investigate an automated alternative to risk stratification of urine cytology slides. We utilised digital cell profiles for the identification of patients with low-risk and high-risk of developing bladder cancer. Our experiments demonstrate that the digital risk could be a better predictor of the final histopathology based diagnosis. We then develop our second framework for the assessment of mismatch repair (MMR) status to identify patients with microsatellite instability (MSI), known to respond well to immunotherapy. We perform multi-stain tissue analysis using slides stained for MMR protein, in addition to H&E and cytokeratin stained slides. To the best of our knowledge, it is the first time that MMR status is utilised for MSI prediction. Registration is an important pre-requisite task before this multi-stain slide analysis. To this end, we present two approaches utilising two different features, hand-crafted and data-driven features. We adopted a multi-scale and multi-stage strategy, important for improving the quality of registration. These methods are able to align the images with low registration error as compared to other hand-crafted based approaches

Investigation of 3D and 4D Feature Extraction from Echocardiography Images Using Local Phase Based Method

Ultrasound images are difficult to segment because of their noisy and low contrast nature which makes them challenging to extract the important features. Typical intensity-gradient based approaches are not suitable for these low contrast images while it has been shown that local phase based technique provides far better results than intensity based methods for ultrasound images. In this study, we adopt local phase based feature asymmetry (FA) measure using monogenic signal for 3-D and 4-D feature extraction. This paper presents our work on: (1) comparison between intensity gradient based feature detectors and phase based feature detector, (2) formulating a 4-D version of FA measure and comparing its results with 3-D FA measure, and (3) comparing quadrature pair of filters in terms of feature detection performance.</p

How divided is a cell? Eigenphase nuclei for classification of mitotic phase in cancer histology images

Detection of mitotic cells in histology images is an important but challenging process due to the resemblance of mitotic cells with other non-mitotic cells and also due to the different appearance of mitotic cells undergoing different phases of the division process. In this paper, we present an algorithm for classification of mitotic cells into its four different phases using eigenphase nuclei images - nuclear exemplars obtained separately from the eigen-decomposition of training nuclei images belonging to each of the four mitotic phases. To the best of our knowledge, ours is the first method to identify mitotic phases in cancer histology images. It is quite likely that the classification results may be negatively affected if the dataset used for training purposes does not contain sufficient number of samples for a positive class. To overcome this class imbalance problem, we present a novel method for oversampling the minority class. The proposed method generates synthetic images for training purposes by perturbing the representation of training samples belonging to the minority class in the eigenphase domain. We show that this strategy works effectively for pairwise classification of the mitotic cells - increasing the classification performance by as much as 24%. 2016 IEEE.Scopu

Glandular structure-guided classification of microscopic colorectal images using deep learning

In this work, we propose to automate the pre-cancerous tissue abnormality analysis by performing the classification of image patches using a novel two-stage convolutional neural network (CNN) based framework. Rather than training a model with features that may correlate among various classes, we propose to train a model using the features which vary across the different classes. Our framework processes the input image to locate the region of interest (glandular structures) and then feeds the processed image to a classification model for abnormality prediction. Our experiments show that our proposed approach improves the classification performance by up to 7% using CNNs and more than 10% while using texture descriptors. When testing with gland segmented images, our experiments reveal that the performance of our classification approach is dependent on the gland segmentation approach which is a key task in gland structure-guided classification