Histopathological image analysis : a review

Over the past decade, dramatic increases in computational power and improvement in image analysis algorithms have allowed the development of powerful computer-assisted analytical approaches to radiological data. With the recent advent of whole slide digital scanners, tissue histopathology slides can now be digitized and stored in digital image form. Consequently, digitized tissue histopathology has now become amenable to the application of computerized image analysis and machine learning techniques. Analogous to the role of computer-assisted diagnosis (CAD) algorithms in medical imaging to complement the opinion of a radiologist, CAD algorithms have begun to be developed for disease detection, diagnosis, and prognosis prediction to complement the opinion of the pathologist. In this paper, we review the recent state of the art CAD technology for digitized histopathology. This paper also briefly describes the development and application of novel image analysis technology for a few specific histopathology related problems being pursued in the United States and Europe

Non-Cancerous Abnormalities That Could Mimic Prostate Cancer Like Signal in Multi-Parametric MRI Images

Prostate Cancer (PCa) is the most common non-cutaneous cancer in North American men. Multi-parametric magnatic resonance imaging (mpMRI) has the potential to be used as a non-invasive procedure to predict locations and prognosis of PCa. This study aims to examine non-cancerous pathology lesions and normal histology that could mimic cancer in mpMRI signals. This study includes 19 radical prostatectomy specimens from the London Health Science Centre (LHSC) that were marked with 10 strand-shaped fiducials per specimen which were used as landmarks in histology processing and ex vivo MRI. Initial registration between fiducials on histology and MR images was performed followed by the development of an interactive digital technique for deformable registration of in vivo to ex vivo MRI with digital histopathology images. The relationship between MRI signals and non-cancerous abnormalities that could mimic PCa has not been tested previously in correlation with digital histopathology imaging. The unregistered mp-MRI images are contoured by 4 individual radiology observers according to the Prostate Imaging Reporting and Data System (PI-RADS). Analysis of the radiology data showed prostatic intraepithelial neoplasia (PIN), atrophy and benign prostatic hyperplasia (BPH) as main non-cancerous abnormalities responsible for cancer like signals on mpMRI. This study will help increase the accuracy of detecting PCa and play a role in the diagnosis and classification of confounders that mimic cancer in MR images

Histopathological image analysis: a review,”

Abstract-Over the past decade, dramatic increases in computational power and improvement in image analysis algorithms have allowed the development of powerful computer-assisted analytical approaches to radiological data. With the recent advent of whole slide digital scanners, tissue histopathology slides can now be digitized and stored in digital image form. Consequently, digitized tissue histopathology has now become amenable to the application of computerized image analysis and machine learning techniques. Analogous to the role of computer-assisted diagnosis (CAD) algorithms in medical imaging to complement the opinion of a radiologist, CAD algorithms have begun to be developed for disease detection, diagnosis, and prognosis prediction to complement the opinion of the pathologist. In this paper, we review the recent state of the art CAD technology for digitized histopathology. This paper also briefly describes the development and application of novel image analysis technology for a few specific histopathology related problems being pursued in the United States and Europe

Added benefits of computer-assisted analysis of Hematoxylin-Eosin stained breast histopathological digital slides

This thesis aims at determining if computer-assisted analysis can be used to better understand pathologists’ perception of mitotic figures on Hematoxylin-Eosin (HE) stained breast histopathological digital slides. It also explores the feasibility of reproducible histologic nuclear atypia scoring by incorporating computer-assisted analysis to cytological scores given by a pathologist. In addition, this thesis investigates the possibility of computer-assisted diagnosis for categorizing HE breast images into different subtypes of cancer or benign masses. In the first study, a data set of 453 mitoses and 265 miscounted non-mitoses within breast cancer digital slides were considered. Different features were extracted from the objects in different channels of eight colour spaces. The findings from the first research study suggested that computer-aided image analysis can provide a better understanding of image-related features related to discrepancies among pathologists in recognition of mitoses. Two tasks done routinely by the pathologists are making diagnosis and grading the breast cancer. In the second study, a new tool for reproducible nuclear atypia scoring in breast cancer histological images was proposed. The third study proposed and tested MuDeRN (MUlti-category classification of breast histopathological image using DEep Residual Networks), which is a framework for classifying hematoxylin-eosin stained breast digital slides either as benign or cancer, and then categorizing cancer and benign cases into four different subtypes each. The studies indicated that computer-assisted analysis can aid in both nuclear grading (COMPASS) and breast cancer diagnosis (MuDeRN). The results could be used to improve current status of breast cancer prognosis estimation through reducing the inter-pathologist disagreement in counting mitotic figures and reproducible nuclear grading. It can also improve providing a second opinion to the pathologist for making a diagnosis

Prostate cancer in a new dimension

The scope of this thesis is to reveal the three-dimensional morphology of prostate cancer and its benign mimickers and to investigate parameters predictive for patient outcome on prostate cancer biopsies

Going deeper through the Gleason scoring scale: An automatic end-to-end system for histology prostate grading and cribriform pattern detection

[EN] Background and Objective: Prostate cancer is one of the most common diseases affecting men worldwide. The Gleason scoring system is the primary diagnostic and prognostic tool for prostate cancer. Further-more, recent reports indicate that the presence of patterns of the Gleason scale such as the cribriform pattern may also correlate with a worse prognosis compared to other patterns belonging to the Glea-son grade 4. Current clinical guidelines have indicated the convenience of highlight its presence during the analysis of biopsies. All these requirements suppose a great workload for the pathologist during the analysis of each sample, which is based on the pathologist's visual analysis of the morphology and or-ganisation of the glands in the tissue, a time-consuming and subjective task. In recent years, with the development of digitisation devices, the use of computer vision techniques for the analysis of biopsies has increased. However, to the best of the authors' knowledge, the development of algorithms to automatically detect individual cribriform patterns belonging to Gleason grade 4 has not yet been studied in the literature. The objective of the work presented in this paper is to develop a deep-learning-based system able to support pathologists in the daily analysis of prostate biopsies. This analysis must include the Gleason grading of local structures, the detection of cribriform patterns, and the Gleason scoring of the whole biopsy. Methods: The methodological core of this work is a patch-wise predictive model based on convolutional neural networks able to determine the presence of cancerous patterns based on the Gleason grading system. In particular, we train from scratch a simple self-design architecture with three filters and a top model with global-max pooling. The cribriform pattern is detected by retraining the set of filters of the last convolutional layer in the network. Subsequently, a biopsy-level prediction map is reconstructed by bi-linear interpolation of the patch-level prediction of the Gleason grades. In addition, from the re-constructed prediction map, we compute the percentage of each Gleason grade in the tissue to feed a multi-layer perceptron which provides a biopsy-level score. Results: In our SICAPv2 database, composed of 182 annotated whole slide images, we obtained a Cohen's quadratic kappa of 0.77 in the test set for the patch-level Gleason grading with the proposed architec-ture trained from scratch. Our results outperform previous ones reported in the literature. Furthermore, this model reaches the level of fine-tuned state-of-the-art architectures in a patient-based four groups cross validation. In the cribriform pattern detection task, we obtained an area under ROC curve of 0.82. Regarding the biopsy Gleason scoring, we achieved a quadratic Cohen's Kappa of 0.81 in the test subset. Shallow CNN architectures trained from scratch outperform current state-of-the-art methods for Gleason grades classification. Our proposed model is capable of characterising the different Gleason grades in prostate tissue by extracting low-level features through three basic blocks (i.e. convolutional layer + max pooling). The use of global-max pooling to reduce each activation map has shown to be a key factor for reducing complexity in the model and avoiding overfitting. Regarding the Gleason scoring of biopsies, a multi-layer perceptron has shown to better model the decision-making of pathologists than previous simpler models used in the literature.This work was supported by the Spanish Ministry of Economy and Competitiveness through project DPI2016-77869. The Titan V used for this research was donated by the NVIDIA Corporation.Silva-Rodríguez, J.; Colomer, A.; Sales, MA.; Molina, R.; Naranjo Ornedo, V. (2020). Going deeper through the Gleason scoring scale: An automatic end-to-end system for histology prostate grading and cribriform pattern detection. Computer Methods and Programs in Biomedicine. 195:1-18. https://doi.org/10.1016/j.cmpb.2020.105637S118195Gordetsky, J., & Epstein, J. (2016). Grading of prostatic adenocarcinoma: current state and prognostic implications. Diagnostic Pathology, 11(1). doi:10.1186/s13000-016-0478-2Epstein, J. I., Egevad, L., Amin, M. B., Delahunt, B., Srigley, J. R., & Humphrey, P. A. (2016). The 2014 International Society of Urological Pathology (ISUP) Consensus Conference on Gleason Grading of Prostatic Carcinoma. American Journal of Surgical Pathology, 40(2), 244-252. doi:10.1097/pas.0000000000000530Sharma, M., & Miyamoto, H. (2018). Percent Gleason pattern 4 in stratifying the prognosis of patients with intermediate-risk prostate cancer. Translational Andrology and Urology, 7(S4), S484-S489. doi:10.21037/tau.2018.03.20Kweldam, C. F., van der Kwast, T., & van Leenders, G. J. (2018). On cribriform prostate cancer. Translational Andrology and Urology, 7(1), 145-154. doi:10.21037/tau.2017.12.33Remotti, H. (2012). Tissue Microarrays: Construction and Use. Pancreatic Cancer, 13-28. doi:10.1007/978-1-62703-287-2_2KHOUJA, M. H., BAEKELANDT, M., SARAB, A., NESLAND, J. M., & HOLM, R. (2010). Limitations of tissue microarrays compared with whole tissue sections in survival analysis. Oncology Letters, 1(5), 827-831. doi:10.3892/ol_00000145Gertych, A., Ing, N., Ma, Z., Fuchs, T. J., Salman, S., Mohanty, S., … Knudsen, B. S. (2015). Machine learning approaches to analyze histological images of tissues from radical prostatectomies. Computerized Medical Imaging and Graphics, 46, 197-208. doi:10.1016/j.compmedimag.2015.08.002Ren, J., Sadimin, E., Foran, D. J., & Qi, X. (2017). Computer aided analysis of prostate histopathology images to support a refined Gleason grading system. Medical Imaging 2017: Image Processing. doi:10.1117/12.2253887Esteban, Á. E., López-Pérez, M., Colomer, A., Sales, M. A., Molina, R., & Naranjo, V. (2019). A new optical density granulometry-based descriptor for the classification of prostate histological images using shallow and deep Gaussian processes. Computer Methods and Programs in Biomedicine, 178, 303-317. doi:10.1016/j.cmpb.2019.07.003Lucas, M., Jansen, I., Savci-Heijink, C. D., Meijer, S. L., de Boer, O. J., van Leeuwen, T. G., … Marquering, H. A. (2019). Deep learning for automatic Gleason pattern classification for grade group determination of prostate biopsies. Virchows Archiv, 475(1), 77-83. doi:10.1007/s00428-019-02577-xArvaniti, E., Fricker, K. S., Moret, M., Rupp, N., Hermanns, T., Fankhauser, C., … Claassen, M. (2018). Automated Gleason grading of prostate cancer tissue microarrays via deep learning. Scientific Reports, 8(1). doi:10.1038/s41598-018-30535-1G. Nir, S. Hor, D. Karimi, L. Fazli, B.F. Skinnider, P. Tavassoli, D. Turbin, C.F. Villamil, G. Wang, R.S. Wilson, K.A. Iczkowski, M.S. Lucia, P.C. Black, P. Abolmaesumi, S.L. Goldenberg, S.E. Salcudean, Automatic grading of prostate cancer in digitized histopathology images: Learning from multiple experts, 2018. 10.1016/j.media.2018.09.005Nir, G., Karimi, D., Goldenberg, S. L., Fazli, L., Skinnider, B. F., Tavassoli, P., … Salcudean, S. E. (2019). Comparison of Artificial Intelligence Techniques to Evaluate Performance of a Classifier for Automatic Grading of Prostate Cancer From Digitized Histopathologic Images. JAMA Network Open, 2(3), e190442. doi:10.1001/jamanetworkopen.2019.0442García, G., Colomer, A., & Naranjo, V. (2019). First-Stage Prostate Cancer Identification on Histopathological Images: Hand-Driven versus Automatic Learning. Entropy, 21(4), 356. doi:10.3390/e21040356Ma, Y., Jiang, Z., Zhang, H., Xie, F., Zheng, Y., Shi, H., … Shi, J. (2018). Generating region proposals for histopathological whole slide image retrieval. Computer Methods and Programs in Biomedicine, 159, 1-10. doi:10.1016/j.cmpb.2018.02.020Li, W., Li, J., Sarma, K. V., Ho, K. C., Shen, S., Knudsen, B. S., … Arnold, C. W. (2019). Path R-CNN for Prostate Cancer Diagnosis and Gleason Grading of Histological Images. IEEE Transactions on Medical Imaging, 38(4), 945-954. doi:10.1109/tmi.2018.2875868Openseadragon, (http://openseadragon.github.io/), Accessed: 10-07-2018.Cohen, J. (1968). Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Psychological Bulletin, 70(4), 213-220. doi:10.1037/h0026256Swets, J. A. (1988). Measuring the Accuracy of Diagnostic Systems. Science, 240(4857), 1285-1293. doi:10.1126/science.3287615Kweldam, C. F., Nieboer, D., Algaba, F., Amin, M. B., Berney, D. M., Billis, A., … van Leenders, G. J. L. H. (2016). Gleason grade 4 prostate adenocarcinoma patterns: an interobserver agreement study among genitourinary pathologists. Histopathology, 69(3), 441-449. doi:10.1111/his.1297

Multi-Features Classification of Prostate Carcinoma Observed in Histological Sections: Analysis of Wavelet-Based Texture and Colour Features

Microscopic biopsy images are coloured in nature because pathologists use the haematoxylin and eosin chemical colour dyes for biopsy examinations. In this study, biopsy images are used for histological grading and the analysis of benign and malignant prostate tissues. The following PCa grades are analysed in the present study: benign, grade 3, grade 4, and grade 5. Biopsy imaging has become increasingly important for the clinical assessment of PCa. In order to analyse and classify the histological grades of prostate carcinomas, pixel-based colour moment descriptor (PCMD) and gray-level co-occurrence matrix (GLCM) methods were used to extract the most significant features for multilayer perceptron (MLP) neural network classification. Haar wavelet transformation was carried out to extract GLCM texture features, and colour features were extracted from RGB (red/green/blue) colour images of prostate tissues. The MANOVA statistical test was performed to select significant features based on F-values and P-values using the R programming language. We obtained an average highest accuracy of 92.7% using level-1 wavelet texture and colour features. The MLP classifier performed well, and our study shows promising results based on multi-feature classification of histological sections of prostate carcinomas.ope

Computer-Assisted Characterization of Prostate Cancer on Magnetic Resonance Imaging

Prostate cancer (PCa) is one of the most prevalent cancers among men. Early diagnosis can improve survival and reduce treatment costs. Current inter-radiologist variability for detection of PCa is high. The use of multi-parametric magnetic resonance imaging (mpMRI) with machine learning algorithms has been investigated both for improving PCa detection and for PCa diagnosis. Widespread clinical implementation of computer-assisted PCa lesion characterization remains elusive; critically needed is a model that is validated against a histologic reference standard that is densely sampled in an unbiased fashion. We address this using our technique for highly accurate fusion of mpMRI with whole-mount digitized histology of the surgical specimen. In this thesis, we present models for characterization of malignant, benign and confounding tissue and aggressiveness of PCa. Further validation on a larger dataset could enable improved characterization performance, improving survival rates and enabling a more personalized treatment plan

Machine Learning for Prostate Histopathology Assessment

Pathology reporting on radical prostatectomy (RP) specimens is essential to post-surgery patient care. However, current pathology interpretation of RP sections is typically qualitative and subject to intra- and inter-observer variability, which challenges quantitative and repeatable reporting of lesion grade, size, location, and spread. Therefore, we developed and validated a software platform that can automatically detect and grade cancerous regions on whole slide images (WSIs) of whole-mount RP sections to support quantitative and visual reporting. Our study used hæmatoxylin- and eosin-stained WSIs from 299 whole-mount RP sections from 71 patients, comprising 1.2 million 480μm×480μm regions-of-interest (ROIs) covering benign and cancerous tissues which contain all clinically relevant grade groups. Each cancerous region was annotated and graded by an expert genitourinary pathologist. We used a machine learning approach with 7 different classifiers (3 non-deep learning and 4 deep learning) to classify: 1) each ROI as cancerous vs. non-cancerous, and 2) each cancerous ROI as high- vs. low-grade. Since recent studies found some subtypes beyond Gleason grade to have independent prognostic value, we also used one deep learning method to classify each cancerous ROI from 87 RP sections of 25 patients as each of eight subtypes to support further clinical pathology research on this topic. We cross-validated each system against the expert annotations. To compensate for the staining variability across different WSIs from different patients, we computed the tissue component map (TCM) using our proposed adaptive thresholding algorithm to label nucleus pixels, global thresholding to label lumen pixels, and assigning the rest as stroma/other. Fine-tuning AlexNet with ROIs of the TCM yielded the best results for prostate cancer (PCa) detection and grading, with areas under the receiver operating characteristic curve (AUCs) of 0.98 and 0.93, respectively, followed by fine-tuned AlexNet with ROIs of the raw image. For subtype grading, fine-tuning AlexNet with ROIs of the raw image yielded AUCs ≥ 0.7 for seven of eight subtypes. To conclude, deep learning approaches outperformed non-deep learning approaches for PCa detection and grading. The TCMs provided the primary cues for PCa detection and grading. Machine learning can be used for subtype grading beyond the Gleason grading system