Applications of Deep Learning in Medical Image Analysis : Grading of Prostate Cancer and Detection of Coronary Artery Disease

A wide range of medical examinations are using analysis of images from different types of equipment. Using artificial intelligence, the assessments could be done automatically. This can have multiple benefits for the healthcare; reduce workload for medical doctors, decrease variations in diagnoses and cut waiting times for the patient as well as improve the performance. The aim of this thesis has been to develop such solutions for two common diseases: prostate cancer and coronary artery disease. The methods used are mainly based on deep learning, where the model teaches itself by training on large datasets.Prostate cancer is one of the most common cancer diagnoses among men. The diagnosis is most commonly determined by visual assessment of prostate biopsies in a light microscope according to the Gleason scale. Deep learning methods to automatically detect and grade the cancer areas are presented in this thesis. The methods have been adapted to improve the generalisation performance on images from different hospitals, images which have inevitable variations in e.g.\ stain appearance. The methods include the usage of digital stain normalisation, training with extensive augmentation or using models such as a domain-adversarial neural network. One Gleason grading algorithm was evaluated on a small cohort with biopsies annotated in detail by two pathologists, to compare the performance with pathologists' inter-observer variability. Another cancer detection algorithm was evaluated on a large active surveillance cohort, containing patients with small areas of low-grade cancer. The results are promising towards a future tool to facilitate grading of prostate cancer.Cardiovascular disease is the leading cause of death world-wide, whereof coronary artery disease is one of the most common diseases. One way to diagnose coronary artery disease is by using myocardial perfusion imaging, where disease in the three main arteries supplying the heart with blood can be detected. Methods based on deep learning to perform the detection automatically are presented in this thesis. Furthermore, an algorithm developed to predict the degree of coronary artery stenosis from myocardial perfusion imaging, by means of quantitative coronary angiography, has also been developed. This assessment is normally done using invasive coronary angiography. Making the prediction automatically from myocardial perfusion imaging could save suffering for patients and free resources within the healthcare system

A Multi-resolution Model for Histopathology Image Classification and Localization with Multiple Instance Learning

Histopathological images provide rich information for disease diagnosis. Large numbers of histopathological images have been digitized into high resolution whole slide images, opening opportunities in developing computational image analysis tools to reduce pathologists' workload and potentially improve inter- and intra- observer agreement. Most previous work on whole slide image analysis has focused on classification or segmentation of small pre-selected regions-of-interest, which requires fine-grained annotation and is non-trivial to extend for large-scale whole slide analysis. In this paper, we proposed a multi-resolution multiple instance learning model that leverages saliency maps to detect suspicious regions for fine-grained grade prediction. Instead of relying on expensive region- or pixel-level annotations, our model can be trained end-to-end with only slide-level labels. The model is developed on a large-scale prostate biopsy dataset containing 20,229 slides from 830 patients. The model achieved 92.7% accuracy, 81.8% Cohen's Kappa for benign, low grade (i.e. Grade group 1) and high grade (i.e. Grade group >= 2) prediction, an area under the receiver operating characteristic curve (AUROC) of 98.2% and an average precision (AP) of 97.4% for differentiating malignant and benign slides. The model obtained an AUROC of 99.4% and an AP of 99.8% for cancer detection on an external dataset.Comment: 9 pages, 6 figure

The impact of pre- and post-image processing techniques on deep learning frameworks: A comprehensive review for digital pathology image analysis

Recently, deep learning frameworks have rapidly become the main methodology for analyzing medical images. Due to their powerful learning ability and advantages in dealing with complex patterns, deep learning algorithms are ideal for image analysis challenges, particularly in the field of digital pathology. The variety of image analysis tasks in the context of deep learning includes classification (e.g., healthy vs. cancerous tissue), detection (e.g., lymphocytes and mitosis counting), and segmentation (e.g., nuclei and glands segmentation). The majority of recent machine learning methods in digital pathology have a pre- and/or post-processing stage which is integrated with a deep neural network. These stages, based on traditional image processing methods, are employed to make the subsequent classification, detection, or segmentation problem easier to solve. Several studies have shown how the integration of pre- and post-processing methods within a deep learning pipeline can further increase the model's performance when compared to the network by itself. The aim of this review is to provide an overview on the types of methods that are used within deep learning frameworks either to optimally prepare the input (pre-processing) or to improve the results of the network output (post-processing), focusing on digital pathology image analysis. Many of the techniques presented here, especially the post-processing methods, are not limited to digital pathology but can be extended to almost any image analysis field

The impact of pre- and post-image processing techniques on deep learning frameworks: A comprehensive review for digital pathology image analysis.

Recently, deep learning frameworks have rapidly become the main methodology for analyzing medical images. Due to their powerful learning ability and advantages in dealing with complex patterns, deep learning algorithms are ideal for image analysis challenges, particularly in the field of digital pathology. The variety of image analysis tasks in the context of deep learning includes classification (e.g., healthy vs. cancerous tissue), detection (e.g., lymphocytes and mitosis counting), and segmentation (e.g., nuclei and glands segmentation). The majority of recent machine learning methods in digital pathology have a pre- and/or post-processing stage which is integrated with a deep neural network. These stages, based on traditional image processing methods, are employed to make the subsequent classification, detection, or segmentation problem easier to solve. Several studies have shown how the integration of pre- and post-processing methods within a deep learning pipeline can further increase the model's performance when compared to the network by itself. The aim of this review is to provide an overview on the types of methods that are used within deep learning frameworks either to optimally prepare the input (pre-processing) or to improve the results of the network output (post-processing), focusing on digital pathology image analysis. Many of the techniques presented here, especially the post-processing methods, are not limited to digital pathology but can be extended to almost any image analysis field

Going deeper through the Gleason scoring scale: An automatic end-to-end system for histology prostate grading and cribriform pattern detection

[EN] Background and Objective: Prostate cancer is one of the most common diseases affecting men worldwide. The Gleason scoring system is the primary diagnostic and prognostic tool for prostate cancer. Further-more, recent reports indicate that the presence of patterns of the Gleason scale such as the cribriform pattern may also correlate with a worse prognosis compared to other patterns belonging to the Glea-son grade 4. Current clinical guidelines have indicated the convenience of highlight its presence during the analysis of biopsies. All these requirements suppose a great workload for the pathologist during the analysis of each sample, which is based on the pathologist's visual analysis of the morphology and or-ganisation of the glands in the tissue, a time-consuming and subjective task. In recent years, with the development of digitisation devices, the use of computer vision techniques for the analysis of biopsies has increased. However, to the best of the authors' knowledge, the development of algorithms to automatically detect individual cribriform patterns belonging to Gleason grade 4 has not yet been studied in the literature. The objective of the work presented in this paper is to develop a deep-learning-based system able to support pathologists in the daily analysis of prostate biopsies. This analysis must include the Gleason grading of local structures, the detection of cribriform patterns, and the Gleason scoring of the whole biopsy. Methods: The methodological core of this work is a patch-wise predictive model based on convolutional neural networks able to determine the presence of cancerous patterns based on the Gleason grading system. In particular, we train from scratch a simple self-design architecture with three filters and a top model with global-max pooling. The cribriform pattern is detected by retraining the set of filters of the last convolutional layer in the network. Subsequently, a biopsy-level prediction map is reconstructed by bi-linear interpolation of the patch-level prediction of the Gleason grades. In addition, from the re-constructed prediction map, we compute the percentage of each Gleason grade in the tissue to feed a multi-layer perceptron which provides a biopsy-level score. Results: In our SICAPv2 database, composed of 182 annotated whole slide images, we obtained a Cohen's quadratic kappa of 0.77 in the test set for the patch-level Gleason grading with the proposed architec-ture trained from scratch. Our results outperform previous ones reported in the literature. Furthermore, this model reaches the level of fine-tuned state-of-the-art architectures in a patient-based four groups cross validation. In the cribriform pattern detection task, we obtained an area under ROC curve of 0.82. Regarding the biopsy Gleason scoring, we achieved a quadratic Cohen's Kappa of 0.81 in the test subset. Shallow CNN architectures trained from scratch outperform current state-of-the-art methods for Gleason grades classification. Our proposed model is capable of characterising the different Gleason grades in prostate tissue by extracting low-level features through three basic blocks (i.e. convolutional layer + max pooling). The use of global-max pooling to reduce each activation map has shown to be a key factor for reducing complexity in the model and avoiding overfitting. Regarding the Gleason scoring of biopsies, a multi-layer perceptron has shown to better model the decision-making of pathologists than previous simpler models used in the literature.This work was supported by the Spanish Ministry of Economy and Competitiveness through project DPI2016-77869. The Titan V used for this research was donated by the NVIDIA Corporation.Silva-Rodríguez, J.; Colomer, A.; Sales, MA.; Molina, R.; Naranjo Ornedo, V. (2020). Going deeper through the Gleason scoring scale: An automatic end-to-end system for histology prostate grading and cribriform pattern detection. Computer Methods and Programs in Biomedicine. 195:1-18. https://doi.org/10.1016/j.cmpb.2020.105637S118195Gordetsky, J., & Epstein, J. (2016). Grading of prostatic adenocarcinoma: current state and prognostic implications. Diagnostic Pathology, 11(1). doi:10.1186/s13000-016-0478-2Epstein, J. I., Egevad, L., Amin, M. B., Delahunt, B., Srigley, J. R., & Humphrey, P. A. (2016). The 2014 International Society of Urological Pathology (ISUP) Consensus Conference on Gleason Grading of Prostatic Carcinoma. American Journal of Surgical Pathology, 40(2), 244-252. doi:10.1097/pas.0000000000000530Sharma, M., & Miyamoto, H. (2018). Percent Gleason pattern 4 in stratifying the prognosis of patients with intermediate-risk prostate cancer. Translational Andrology and Urology, 7(S4), S484-S489. doi:10.21037/tau.2018.03.20Kweldam, C. F., van der Kwast, T., & van Leenders, G. J. (2018). On cribriform prostate cancer. Translational Andrology and Urology, 7(1), 145-154. doi:10.21037/tau.2017.12.33Remotti, H. (2012). Tissue Microarrays: Construction and Use. Pancreatic Cancer, 13-28. doi:10.1007/978-1-62703-287-2_2KHOUJA, M. H., BAEKELANDT, M., SARAB, A., NESLAND, J. M., & HOLM, R. (2010). Limitations of tissue microarrays compared with whole tissue sections in survival analysis. Oncology Letters, 1(5), 827-831. doi:10.3892/ol_00000145Gertych, A., Ing, N., Ma, Z., Fuchs, T. J., Salman, S., Mohanty, S., … Knudsen, B. S. (2015). Machine learning approaches to analyze histological images of tissues from radical prostatectomies. Computerized Medical Imaging and Graphics, 46, 197-208. doi:10.1016/j.compmedimag.2015.08.002Ren, J., Sadimin, E., Foran, D. J., & Qi, X. (2017). Computer aided analysis of prostate histopathology images to support a refined Gleason grading system. Medical Imaging 2017: Image Processing. doi:10.1117/12.2253887Esteban, Á. E., López-Pérez, M., Colomer, A., Sales, M. A., Molina, R., & Naranjo, V. (2019). A new optical density granulometry-based descriptor for the classification of prostate histological images using shallow and deep Gaussian processes. Computer Methods and Programs in Biomedicine, 178, 303-317. doi:10.1016/j.cmpb.2019.07.003Lucas, M., Jansen, I., Savci-Heijink, C. D., Meijer, S. L., de Boer, O. J., van Leeuwen, T. G., … Marquering, H. A. (2019). Deep learning for automatic Gleason pattern classification for grade group determination of prostate biopsies. Virchows Archiv, 475(1), 77-83. doi:10.1007/s00428-019-02577-xArvaniti, E., Fricker, K. S., Moret, M., Rupp, N., Hermanns, T., Fankhauser, C., … Claassen, M. (2018). Automated Gleason grading of prostate cancer tissue microarrays via deep learning. Scientific Reports, 8(1). doi:10.1038/s41598-018-30535-1G. Nir, S. Hor, D. Karimi, L. Fazli, B.F. Skinnider, P. Tavassoli, D. Turbin, C.F. Villamil, G. Wang, R.S. Wilson, K.A. Iczkowski, M.S. Lucia, P.C. Black, P. Abolmaesumi, S.L. Goldenberg, S.E. Salcudean, Automatic grading of prostate cancer in digitized histopathology images: Learning from multiple experts, 2018. 10.1016/j.media.2018.09.005Nir, G., Karimi, D., Goldenberg, S. L., Fazli, L., Skinnider, B. F., Tavassoli, P., … Salcudean, S. E. (2019). Comparison of Artificial Intelligence Techniques to Evaluate Performance of a Classifier for Automatic Grading of Prostate Cancer From Digitized Histopathologic Images. JAMA Network Open, 2(3), e190442. doi:10.1001/jamanetworkopen.2019.0442García, G., Colomer, A., & Naranjo, V. (2019). First-Stage Prostate Cancer Identification on Histopathological Images: Hand-Driven versus Automatic Learning. Entropy, 21(4), 356. doi:10.3390/e21040356Ma, Y., Jiang, Z., Zhang, H., Xie, F., Zheng, Y., Shi, H., … Shi, J. (2018). Generating region proposals for histopathological whole slide image retrieval. Computer Methods and Programs in Biomedicine, 159, 1-10. doi:10.1016/j.cmpb.2018.02.020Li, W., Li, J., Sarma, K. V., Ho, K. C., Shen, S., Knudsen, B. S., … Arnold, C. W. (2019). Path R-CNN for Prostate Cancer Diagnosis and Gleason Grading of Histological Images. IEEE Transactions on Medical Imaging, 38(4), 945-954. doi:10.1109/tmi.2018.2875868Openseadragon, (http://openseadragon.github.io/), Accessed: 10-07-2018.Cohen, J. (1968). Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Psychological Bulletin, 70(4), 213-220. doi:10.1037/h0026256Swets, J. A. (1988). Measuring the Accuracy of Diagnostic Systems. Science, 240(4857), 1285-1293. doi:10.1126/science.3287615Kweldam, C. F., Nieboer, D., Algaba, F., Amin, M. B., Berney, D. M., Billis, A., … van Leenders, G. J. L. H. (2016). Gleason grade 4 prostate adenocarcinoma patterns: an interobserver agreement study among genitourinary pathologists. Histopathology, 69(3), 441-449. doi:10.1111/his.1297

Automated Detection of Cribriform Growth Patterns in Prostate Histology Images

Cribriform growth patterns in prostate carcinoma are associated with poor prognosis. We aimed to introduce a deep learning method to detect such patterns automatically. To do so, convolutional neural network was trained to detect cribriform growth patterns on 128 prostate needle biopsies. Ensemble learning taking into account other tumor growth patterns during training was used to cope with heterogeneous and limited tumor tissue occurrences. ROC and FROC analyses were applied to assess network performance regarding detection of biopsies harboring cribriform growth pattern. The ROC analysis yielded a mean area under the curve up to 0.81. FROC analysis demonstrated a sensitivity of 0.9 for regions larger than 0.0150 mm2 with on average 7.5 false positives. To benchmark method performance for intra-observer annotation variability, false positive and negative detections were re-evaluated by the pathologists. Pathologists considered 9% of the false positive regions as cribriform, and 11% as possibly cribriform; 44% of the false negative regions were not annotated as cribriform. As a final experiment, the network was also applied on a dataset of 60 biopsy regions annotated by 23 pathologists. With the cut-off reaching highest sensitivity, all images annotated as cribriform by at least 7/23 of the pathologists, were all detected as cribriform by the network and 9/60 of the images were detected as cribriform whereas no pathologist labelled them as such. In conclusion, the proposed deep learning method has high sensitivity for detecting cribriform growth patterns at the expense of a limited number of false positives. It can detect cribriform regions that are labelled as such by at least a minority of pathologists. Therefore, it could assist clinical decision making by suggesting suspicious regions.Comment: 15 pages, 6 figure