Bertoni L.

Bianchi G.

Bolelli F.

Caramaschi S.

Grana C.

Micali S.

Pinamonti M.

Puliatti S.

Reggiani Bonetti L.

Rodriguez N.

Rozze D.

English

Archivio istituzionale della ricerca - Università di Modena e Reggio Emilia

Vol. 75 - No. 5 MiNerVa Urology aNd Nephrology 545M I N E R VA  O P I N I O N  E D I T O R I A LArtificial intelligence evaluation of confocal microscope prostate images: our preliminary experienceGiampaolo BIANCHI 1, Stefano PULIATTI 1 *, Natalia RODRIGUEZ 1, Salvatore MICALI 1, Laura BERTONI 2, Luca REGGIANI BONETTI 3, Stefania CARAMASCHI 3, Federico BOLELLI 4, Maurizio PINAMONTI 5, Davide ROZZE 5, Costantino GRANA 41Department of Urology, University of Modena and Reggio Emilia, Modena, Italy; 2Department of Surgery, Medicine, Dentistry and Morphological Sciences with Interest in Transplant, Oncology and Regenerative Medicine, Modena, Italy; 3Department of Medical and Surgical Sciences for Children and Adults, University of Modena and Reggio Emilia, Modena Polyclinic Hospital, Modena, Italy; 4Enzo Ferrari Department of Engineering, University of Modena and Reggio Emilia, Modena, Italy; 5Azienda Sanitaria Universitaria Giuliano Isontina, Trieste, Italy*Corresponding author: Stefano Puliatti, Department of Urology, University of Modena and Reggio Emilia, Modena, Italy. E-mail: stefano.puliatti@unimore.itMinerva Urology and Nephrology 2023 October;75(5):545-7DOI: 10.23736/S2724-6051.23.05538-6© 2023 EDIZIONI MINERVA MEDICAOnline version at https://www.minervamedica.itprostate cancer is a prevalent neoplasm in men, with varying aggressiveness.1, 2 To manage the disease optimally, both oncologic radicality and postoperative functional outcomes are essential.3, 4 The ‘gold standard’ pathological assessment of prostate cancer is still a conven-tional, time-consuming approach, based on his-tomorphological evaluation of bioptic or surgical tissue samples.5, 6Ex-vivo fluorescence confocal microscopy (FCM) is a promising innovative method which allows real-time evaluation of prostate tis-sue, producing images that highly overlap with the traditional hematoxylin and eosin-stained slides.6, 7 FCM has shown excellent results in evaluating healthy and cancerous prostate tissue, offering high concordance, reliability, sensitiv-ity, and specificity.6, 7The application of Artificial Intelligence (AI) on FCM images seems to have great potential. Notably, confocal microscopy allows the direct acquisition of digital images, enabling the im-mediate utilization of AI. The annotation process could be demanding in terms of expertise, cost, and time; but AI, especially neural networks, has shown potential in overcoming these obstacles, as demonstrated also in the dermatological field.8 The integration of confocal microscopy technol-ogy with neural networks represents a potential “game-changer” in the field of pathology.The aim of our study is to assess the perfor-mance of a weakly supervised approach based on graph neural networks (GNNs) for detecting prostatic tumors in digitally colored FCM images obtained from prostatic biopsies. The selected al-gorithm, DAS-MIL,9 integrates pyramidal GNN with multi-instance learning (MIL) to leverage the full potential of FCM high resolution images, without relying on pixel-level annotations (i.e., requiring only the information about the pres-ence/absence of the prostatic tumor inside the image).For the examination, 506 prostatic biopsies from 64 patients affected by adenocarcinoma or non-neoplastic conditions, were collected and analyzed using the ex-vivo FCM device (Viva-Scope® 2500 M-G4, Mavig GmbH, Munich, Germany; Caliber I.D.; Rochester NY, USA) This document is protected by international copyright laws. No additional reproduction is authorized. It is permitted for personal use to download and save only one file and print only one copy of this Article. It is not permitted to make additional copies (either sporadically or systematically, either printed or electronic) of the Article for any purpose. It is not permitted to distribute the electronic copy of the article through online internet and/or intranet file sharing systems, electronic mailing or any other means which may allow access to the Article. The use of all or any part of the Article for any Commercial Use is not permitted. The creation of derivative works from the Article is not permitted. The production of reprints for personal or commercial use is not permitted. It is not permitted to remove, cover,  overlay, obscure, block, or change any copyright notices or terms of use which the Publisher may post on the Article. It is not permitted to frame or use framing techniques to enclose any trademark, logo, or other proprietary information of the Publisher.COPYRIGHT© 2023 EDIZIONI MINERVA MEDICABIANCHI  AI EVALUATION OF CONFOCAL MICROSCOPE PROSTATE IMAGES546 MiNerVa Urology aNd Nephrology October 2023 can be also exploited to produce a heatmap high-lighting the portions of the images that contrib-uted the most to the final prediction. Since no la-bel at pixel-level is available the heatmaps can be used only for qualitative evaluation. An example is provided in Figure 1.Up until now, AI has been employed in ana-lyzing conventional pathologic prostate can-cer images directly or that were later digitized, yielding remarkably high accuracy11-13and strong concurrence with expert uropathologists’ diag-noses.14 Our study sought to utilize real-time flu-orescence confocal microscope (FCM) images, eliminating the need for specimen processing at the Department of Urology, University of Modena and Reggio Emilia, Italy. Written in-formed consent was obtained from each patient preoperatively. Fresh samples were analyzed as described elsewhere.10The 506 digitalized prostatic biopsies were split into two sets: 307 images for training DAS-MIL and 199 images for testing and validating the automatic approach. Both sets were random-ly generated while preserving the class distribu-tion (around 20% of images contained tumor tis-sue in both sets). The model was trained to dis-tinguish between images with tumor tissue and healthy ones, utilizing available ground-truth labels. The final performance was assessed using the area under the ROC curve (AUC) and accu-racy metrics. The AUC achieved a value of 0.93, and the accuracy score was 0.86. Additionally, the algorithm’s behavior was analyzed through the confusion matrix presented in Table I. This enabled the calculation of sensitivity (true posi-tive rate) and specificity (true negative rate) of the algorithm on the target dataset. DAS-MIL demonstrated a sensitivity of 0.8 and a specific-ity of 0.88.Leveraging on MIL, the final prediction of DAS-MIL is performed by calculating the criti-cality of each instance and comparing all of them with the most significant one. Such an approach Table I.— Confusion matrix summarizing DAS-MIL predictions on the selected test-set. Among the 199 im-ages available, 172 (86.43%) are predicted correctly with high sensitivity (0.8) and specificity (0.88).OutputTargetNon-tumor Tumor SumNon-tumor 14472.36%73.52%15195.36%4.64%Tumor 2010.05%2814.07%4858.33%41.67%Sum 16487.80%12.20%3580.00%20.00%172/19986.43%13.57%a BFigure 1.—A) Example heatmap produced by DAS-MIL on a positive FMC image predicted as positive also by the model. To better appre-ciate the details only a portion of the entire im-age is reported. Instances highlighted in yellowish identify more critical areas and the relevance decreases with color in-tensity. No highlighted areas are those that the automatic model consid-ers less significant for the prediction. B) The same image, obtained with confocal micros-copy, unprocessed by AI.This document is protected by international copyright laws. No additional reproduction is authorized. It is permitted for personal use to download and save only one file and print only one copy of this Article. It is not permitted to make additional copies (either sporadically or systematically, either printed or electronic) of the Article for any purpose. It is not permitted to distribute the electronic copy of the article through online internet and/or intranet file sharing systems, electronic mailing or any other means which may allow access to the Article. The use of all or any part of the Article for any Commercial Use is not permitted. The creation of derivative works from the Article is not permitted. The production of reprints for personal or commercial use is not permitted. It is not permitted to remove, cover,  overlay, obscure, block, or change any copyright notices or terms of use which the Publisher may post on the Article. It is not permitted to frame or use framing techniques to enclose any trademark, logo, or other proprietary information of the Publisher.COPYRIGHT© 2023 EDIZIONI MINERVA MEDICAAI EVALUATION OF CONFOCAL MICROSCOPE PROSTATE IMAGES BIANCHIVol. 75 - No. 5 MiNerVa Urology aNd Nephrology 547radical prostatectomy: can the approach really make a differ-ence? Minerva Urol Nephrol 2022;74:137–45. 5. Mir MC, Bancalari B, Calatrava A, Casanova J, Domin-guez Escrig JL, Ramirez-Backhaus M, et al. Ex-vivo confo-cal fluorescence microscopy for rapid evaluation of renal core biopsy. Minerva Urol Nefrol 2020;72:109–13. 6. Puliatti S, Bertoni L, Pirola GM, Azzoni P, Bevilacqua L, Eissa A, et al. Ex vivo fluorescence confocal microscopy: the first application for real-time pathological examination of prostatic tissue. BJU Int 2019;124:469–76. 7. Bertoni L, Puliatti S, Reggiani Bonetti L, Maiorana A, Eissa A, Azzoni P, et al. Ex vivo fluorescence confocal mi-croscopy: prostatic and periprostatic tissues atlas and evalu-ation of the learning curve. Virchows Arch 2020;476:511–20. 8. Sendín-Martín M, Lara-Caro M, Harris U, Moronta M, Rossi A, Lee E, et al. Classification of Basal Cell Carcinoma in Ex Vivo Confocal Microscopy Images from Freshly Ex-cised Tissues Using a Deep Learning Algorithm. J Invest Der-matol 2022;142:1291–1299.e2. 9. Bontempo G. Marta Lovino, Federico Bolelli, Anni Vir-tanen, Elisa Ficarra. Enhancing PFI Prediction with GDS-MIL: A Graph-based Dual Stream MIL Approach; 2023 [Internet]. Available from: https://iris.unimore.it/retrieve/fd6dd815-9d7e-4b5e-b64d-9b405f70fcf0/2023ICIAP_Enhancing%20PFI%20Prediction%20with%20GDS-MIL%3A%20A%20Graph-based%20Dual%20Stream%20MIL%20Approach.pdf [cited 2023, Aug 1].10. Rocco B, Sighinolfi MC, Sandri M, Spandri V, Cimada-more A, Volavsek M, et al. Digital Biopsy with Fluorescence Confocal Microscope for Effective Real-time Diagnosis of Prostate Cancer: A Prospective, Comparative Study. Eur Urol Oncol 2021;4:784–91. 11. Ström P, Kartasalo K, Olsson H, Solorzano L, Delahunt B, Berney DM, et al. Artificial intelligence for diagnosis and grading of prostate cancer in biopsies: a population-based, di-agnostic study. Lancet Oncol 2020;21:222–32. 12. Campanella G, Hanna MG, Geneslaw L, Miraflor A, Werneck Krauss Silva V, Busam KJ, et al. Clinical-grade computational pathology using weakly supervised deep learn-ing on whole slide images. Nat Med 2019;25:1301–9. 13. Wessels F, Kuntz S, Krieghoff-Henning E, Schmitt M, Braun V, Worst TS, et al. Artificial intelligence to predict onco-logical outcome directly from hematoxylin and eosin-stained slides in urology. Minerva Urol Nephrol 2022;74:538–50. 14. Nagpal K, Foote D, Tan F, Liu Y, Chen PC, Steiner DF, et al. Development and Validation of a Deep Learning Algo-rithm for Gleason Grading of Prostate Cancer From Biopsy Specimens. JAMA Oncol 2020;6:1372–80. and the following image digitalization, thereby expediting the application process. Surprisingly, the artificial neural network achieved in our pre-liminary experience an AUC value of 0.93, and an accuracy score of 0.86, even though it had not undergone prior training to recognize neoplastic lesions. On a following analysis performed by our expert pathologist (LRB) on the AI-sensitive areas, although it was a non-machine learning neural network application, those areas were largely recognized as glandular areas with fea-tures of neoplasia. In our experience, the applica-tion of AI on FCM images could allow a remark-able time advantage, thanks to a real-time intra-operative assessment that could directly address the pathologist to initially evaluate those areas flagged by the machine as neoplastic. This study constitutes a preliminary experience. By training the machine to recognize pathologic patterns, in our opinion, even more encouraging results will be possible.References1. D’Agostino D, Casablanca C, Mineo Bianchi F, Corsi P, Romagnoli D, Giampaoli M, et al. The role of magnetic reso-nance imaging-guided biopsy for diagnosis of prostate cancer; comparison between FUSION and “IN-BORE” approaches. Minerva Urol Nephrol 2021;73:90–7. 2. Nocera L, Collà Ruvolo C, Stolzenbach LF, Deuker M, Tian Z, Gandaglia G, et al. Improving the stratification of intermediate risk prostate cancer. Minerva Urol Nephrol 2022;74:590–8. 3. Eissa A, Zoeir A, Sighinolfi MC, Puliatti S, Bevilacqua L, Del Prete C, et al. “Real-time” Assessment of Surgical Mar-gins During Radical Prostatectomy: State-of-the-Art. Clin Genitourin Cancer 2020;18:95–104. 4. Albisinni S, Dasnoy C, Diamand R, Mjaess G, Aoun F, Esperto F, et al. Anterior vs. Retzius-sparing robotic assisted Conflicts of interestThe authors certify that there is no conflict of interest with any financial organization regarding the material discussed in the manu-script.Authors’ contributionsGiampaolo Bianchi and Costantino Grana conceptualized the study. Stefano Puliatti, Federico Bolelli and Salvatore Micali designed the study. Laura Bertoni acquired FCM images. Federico Bolelli developed and tested the graph neural network algorithms. Luca Reggiani Bonetti, Stefania Caramaschi, Davide Rozze, and Maurizio Pinamonti were responsible for the pathologic evaluation of the FCM images. Federico Bolelli was responsible for data analysis. Natali Rodriguez, Laura Bertoni, and Federico Bolelli wrote the draft of the paper. Giampaolo Bianchi, Stefano Puliatti, Luca Reggiani Bonetti, Costantino Grana reviewed the paper. All authors read and approved the final version of the manuscript.HistoryManuscript accepted: July 31, 2023. - Manuscript received: July 31, 2023.(Cite this article as: Bianchi G, Puliatti S, Rodriguez N, Micali S, Bertoni L, Reggiani Bonetti L, et al. Artificial intelligence evalu-ation of confocal microscope prostate images: our preliminary experience. Minerva Urol Nephrol 2023;75:545-7. DOI: 10.23736/S2724-6051.23.05538-6)This document is protected by international copyright laws. No additional reproduction is authorized. It is permitted for personal use to download and save only one file and print only one copy of this Article. It is not permitted to make additional copies (either sporadically or systematically, either printed or electronic) of the Article for any purpose. It is not permitted to distribute the electronic copy of the article through online internet and/or intranet file sharing systems, electronic mailing or any other means which may allow access to the Article. The use of all or any part of the Article for any Commercial Use is not permitted. The creation of derivative works from the Article is not permitted. The production of reprints for personal or commercial use is not permitted. It is not permitted to remove, cover,  overlay, obscure, block, or change any copyright notices or terms of use which the Publisher may post on the Article. It is not permitted to frame or use framing techniques to enclose any trademark, logo, or other proprietary information of the Publisher.COPYRIGHT© 2023 EDIZIONI MINERVA MEDICA

Artificial intelligence evaluation of confocal microscope prostate images: our preliminary experience

https://iris.unimore.it/bitstream/11380/1328830/1/2023minervaurology.pdf

Artificial intelligence evaluation of confocal microscope prostate images: our preliminary experience

Abstract

Similar works

Full text

Available Versions

Archivio istituzionale della ricerca - Università di Modena e Reggio Emilia