Optical diagnosis of colorectal polyp images using a newly developed computer-aided diagnosis system (CADx) compared with intuitive optical diagnosis

Background Optical diagnosis of colorectal polyps remains challenging. Image-enhancement techniques such as narrow-band imaging and blue-light imaging (BLI) can improve optical diagnosis. We developed and prospectively validated a computer-aided diagnosis system (CADx) using high-definition white-light (HDWL) and BLI images, and compared the system with the optical diagnosis of expert and novice endoscopists.Methods CADx characterized colorectal polyps by exploiting artificial neural networks. Six experts and 13 novices optically diagnosed 60 colorectal polyps based on intuition. After 4 weeks, the same set of images was permuted and optically diagnosed using the BLI Adenoma Serrated International Classification (BASIC).Results CADx had a diagnostic accuracy of 88.3% using HDWL images and 86.7% using BLI images. The overall diagnostic accuracy combining HDWL and BLI (multimodal imaging) was 95.0%, which was significantly higher than that of experts (81.7%, P =0.03) and novices (66.7%, P <0.001). Sensitivity was also higher for CADx (95.6% vs. 61.1% and 55.4%), whereas specificity was higher for experts compared with CADx and novices (95.6% vs. 93.3% and 93.2%). For endoscopists, diagnostic accuracy did not increase when using BASIC, either for experts (intuition 79.5% vs. BASIC 81.7%, P =0.14) or for novices (intuition 66.7% vs. BASIC 66.5%, P =0.95).Conclusion CADx had a significantly higher diagnostic accuracy than experts and novices for the optical diagnosis of colorectal polyps. Multimodal imaging, incorporating both HDWL and BLI, improved the diagnostic accuracy of CADx. BASIC did not increase the diagnostic accuracy of endoscopists compared with intuitive optical diagnosis

Effect of domain-specific self-supervised pretraining on predictive uncertainty for colorectal polyp characterization

Colorectal polyps (CRPs) are potential precursors of colorectal cancer (CRC), one of the most common types of cancer worldwide. Computer-Aided Diagnosis (CADx) systems can play a crucial role as a second opinion for endoscopists in characterizing CRPs and contribute to the diagnostic performance of colonoscopies. Despite their potential, deep neural network-based systems often tend to overestimate the confidence about their decisions and provide predictive probabilities that are poorly related to their classification accuracy. Quantifying uncertainty of such supportive systems is crucial for optimal clinical workflow integration and physician's acceptance. Thus, a trustworthy CADx system is expected to provide accurate and well-calibrated classification confidence. Transfer learning from either natural image datasets, such as ImageNet, or other datasets with similar modalities, has been widely used for improving the accuracy of deep learning-based systems in medical image classification. In this paper, we study the impact of domain-specific pretraining on the calibration and the overall performance of a CADx system for CRP characterization. We evaluate our hypothesis on a fully deterministic and a hybrid Bayesian version of each approach using a generic ResNet50 architecture. Experimental results demonstrate the effectiveness of domain-specific pretraining in achieving a higher overall characterization AUC. Additionally, the in-domain and out-of-domain pretrained models portray similar calibration error rates, however, their corresponding hybrid Bayesian models offer higher robustness with improved calibration performance. A hybrid Bayesian version of a domain-specific pretraining approach has shown to significantly improve the accuracy and reliability of CADx systems used for CRP characterization and similar positive effects may be expected for other medical imaging applications

A novel clinical gland feature for detection of early Barrett’s neoplasia using volumetric laser endomicroscopy

Volumetric laser endomicroscopy (VLE) is an advanced imaging system offering a promising solution for the detection of early Barrett’s esophagus (BE) neoplasia. BE is a known precursor lesion for esophageal adenocarcinoma and is often missed during regular endoscopic surveillance of BE patients. VLE provides a circumferential scan of near-microscopic resolution of the esophageal wall up to 3-mm depth, yielding a large amount of data that is hard to interpret in real time. In a preliminary study on an automated analysis system for ex vivo VLE scans, novel quantitative image features were developed for two previously identified clinical VLE features predictive for BE neoplasia, showing promising results. This paper proposes a novel quantitative image feature for a missing third clinical VLE feature. The novel gland-based image feature called “gland statistics” (GS), is compared to several generic image analysis features and the most promising clinically-inspired feature “layer histogram” (LH). All features are evaluated on a clinical, validated data set consisting of 88 non-dysplastic BE and 34 neoplastic in vivo VLE images for eight different widely-used machine learning methods. The new clinically-inspired feature has on average superior classification accuracy (0.84 AUC) compared to the generic image analysis features (0.61 AUC), as well as comparable performance to the LH feature (0.86 AUC). Also, the LH feature achieves superior classification accuracy compared to the generic image analysis features in vivo, confirming previous ex vivo results. Combining the LH and the novel GS features provides even further improvement of the performance (0.88 AUC), showing great promise for the clinical utility of this algorithm to detect early BE neoplasia

Computer-aided classification of colorectal polyps using blue-light and linked-color imaging

Colorectal cancer (CRC) is one of the leading causes of cancer-related deaths. Since most CRCs develop from colorectal polyps (CRPs), accurate endoscopic differentiation facilitates decision making on resection of CRPs, thereby increasing cost-efficiency and reducing patient risk. Current classification systems based on whitelight imaging (WLI) or narrow-band imaging (NBI) have limited predictive power, or they do not consider sessile serrated adenomas/polyps (SSA/Ps), although these cause up to 30% of all CRCs. To better differentiate adenomas, hyperplastic polyps, and SSA/Ps, this paper explores the feasibility of two approaches: (1) an accurate computer-aided diagnosis (CADx) system for automated diagnosis of CRPs, and (2) novel endoscopic imaging techniques like blue-light imaging (BLI) and linked-color imaging (LCI). Two methods are explored to predict histology: (1) direct classification using a support vector machine (SVM) classifier, and (2) classification via a clinical classification model (WASP classification) combined with an SVM. The use of probabilistic features of SVM facilitates objective quantification of the detailed classification process. Automated differentiation of colonic polyp subtypes reaches accuracies of 78−96%, thereby improving medical expert results by 4−20%. Diagnostic accuracy for directly predicting adenomatous from hyperplastic histology reaches 93% and 87−90% using NBI and the novel BLI and LCI techniques, respectively, thus improving medical expert results by 26% and 20−23%, respectively. Predicting adenomatous histology in diminutive polyps with high confidence yields NPVs of 100%, clearly satisfying the PIVI guideline recommendation on endoscopic innovations (≥90% NPV). Our CADx system outperforms clinicians, while the novel BLI technique adds performance value

Ensemble of deep convolutional neural networks for classification of early Barrett’s neoplasia using volumetric laser endomicroscopy

Barrett’s esopaghagus (BE) is a known precursor of esophageal adenocarcinoma (EAC). Patients with BE undergo regular surveillance to early detect stages of EAC. Volumetric laser endomicroscopy (VLE) is a novel technology incorporating a second-generation form of optical coherence tomography and is capable of imaging the inner tissue layers of the esophagus over a 6 cm length scan. However, interpretation of full VLE scans is still a challenge for human observers. In this work, we train an ensemble of deep convolutional neural networks to detect neoplasia in 45 BE patients, using a dataset of images acquired with VLE in a multi-center study. We achieve an area under the receiver operating characteristic curve (AUC) of 0.96 on the unseen test dataset and we compare our results with previous work done with VLE analysis, where only AUC of 0.90 was achieved via cross-validation on 18 BE patients. Our method for detecting neoplasia in BE patients facilitates future advances on patient treatment and provides clinicians with new assisting solutions to process and better understand VLE data

A novel clinical gland feature for detection of early Barrett’s neoplasia using volumetric laser endomicroscopy

Volumetric laser endomicroscopy (VLE) is an advanced imaging system offering a promising solution for the detection of early Barrett’s esophagus (BE) neoplasia. BE is a known precursor lesion for esophageal adenocarcinoma and is often missed during regular endoscopic surveillance of BE patients. VLE provides a circumferential scan of near-microscopic resolution of the esophageal wall up to 3-mm depth, yielding a large amount of data that is hard to interpret in real time. In a preliminary study on an automated analysis system for ex vivo VLE scans, novel quantitative image features were developed for two previously identified clinical VLE features predictive for BE neoplasia, showing promising results. This paper proposes a novel quantitative image feature for a missing third clinical VLE feature. The novel gland-based image feature called “gland statistics” (GS), is compared to several generic image analysis features and the most promising clinically-inspired feature “layer histogram” (LH). All features are evaluated on a clinical, validated data set consisting of 88 non-dysplastic BE and 34 neoplastic in vivo VLE images for eight different widely-used machine learning methods. The new clinically-inspired feature has on average superior classification accuracy (0.84 AUC) compared to the generic image analysis features (0.61 AUC), as well as comparable performance to the LH feature (0.86 AUC). Also, the LH feature achieves superior classification accuracy compared to the generic image analysis features in vivo, confirming previous ex vivo results. Combining the LH and the novel GS features provides even further improvement of the performance (0.88 AUC), showing great promise for the clinical utility of this algorithm to detect early BE neoplasia

Deep principal dimension encoding for the classification of early neoplasia in Barrett's Esophagus with volumetric laser endomicroscopy

Barrett cancer is a treatable disease when detected at an early stage. However, current screening protocols are often not effective at finding the disease early. Volumetric Laser Endomicroscopy (VLE) is a promising new imaging tool for finding dysplasia in Barrett's esophagus (BE) at an early stage, by acquiring cross-sectional images of the microscopic structure of BE up to 3-mm deep. However, interpretation of VLE scans is difficult for medical doctors due to both the size and subtlety of the gray-scale data. Therefore, algorithms that can accurately find cancerous regions are very valuable for the interpretation of VLE data. In this study, we propose a fully-automatic multi-step Computer-Aided Detection (CAD) algorithm that optimally leverages the effectiveness of deep learning strategies by encoding the principal dimension in VLE data. Additionally, we show that combining the encoded dimensions with conventional machine learning techniques further improves results while maintaining interpretability. Furthermore, we train and validate our algorithm on a new histopathologically validated set of in-vivo VLE snapshots. Additionally, an independent test set is used to assess the performance of the model. Finally, we compare the performance of our algorithm against previous state-of-the-art systems. With the encoded principal dimension, we obtain an Area Under the Curve (AUC) and F1 score of 0.93 and 87.4% on the test set respectively. We show this is a significant improvement compared to the state-of-the-art of 0.89 and 83.1%, respectively, thereby demonstrating the effectiveness of our approach

Automatic textual description of colorectal polyp features: explainable artificial intelligence based on the BASIC classification

Aims Computer-aided diagnosis (CADx-)systems could improve optical diagnosis of colorectal polyps (CRPs) by endoscopists. For integration into clinical practice, better understanding of artificial intelligence (AI) is needed. A branch of deep learning and explainable AI is automatically generating textual descriptions from images to improve understanding. We aimed to develop a CADx-system generating automatic textual descriptions for CRPs based on Blue Light Imaging (BLI) Adenoma Serrated International Classification (BASIC)[1]. Methods Training data contained 35 hyperplastic polyps, 12 sessile serrated lesions (SSLs) and 48 adenomas, with 6525 corresponding textual descriptions by endoscopists. Testing data contained 15 hyperplastic polyps, three SSLs, 36 adenomas, and one colorectal carcinoma. Both databases consisted of High Definition White Light (HDWL), BLI, and Linked Color Imaging (LCI) images. CADx’s 165 generated descriptions were compared to 1857 descriptions from nineteen endoscopists. References not matching histological diagnoses were excluded. The Recall Oriented Understudy for Gisting Evaluation Longest common subsequence (ROUGE-L) score measured the longest word segment in generated descriptions corresponding with reference descriptions. Results A CADx-system generating automatic textual descriptions of CRP features was successfully developed ([Figure 1]). ROUGE-L scores (%) per category were: Complete sentence 83%, BASIC descriptors 70%, Morphology & size 89%, Surface 92%, Pit pattern 85%, and Vessels 59

Real-time characterization of colorectal polyps using artificial intelligence: A prospective pilot study comparing two computer-aided diagnosis systems and one expert endoscopist

Aims Artificial intelligence (AI) has great potential in gastrointestinal endoscopy. Aim was to evaluate real-time diagnostic performances of our Artificial Intelligence for ColoRectal Polyps (AI4CRP) computer-aided diagnosis system for optical diagnosis of diminutive colorectal polyps (CRPs) and compare it with CAD EYE and an expert endoscopist. Methods AI4CRP was developed using convolutional neural networks and previously trained and tested. In this prospective real-time pilot study, AI4CRP was compared with CAD EYE© (Fujifilm, Tokyo, Japan) and one expert endoscopist unaware of AI-output. Blue light imaging was used for characterization and histopathology as gold standard. CRPs were characterized as hyperplastic (hyperplastic polyps) or neoplastic (adenomas, sessile serrated lesions[SSLs]) by AI4CRP and the endoscopist, and as hyperplastic (hyperplastic polyps, SSLs) or neoplastic (adenomas) by CAD EYE. CAD EYE’s inconclusive diagnoses were excluded. Enabling self-critical AI4CRP, post-hoc analysis excluded low confidence scores. Results Real-time testing included 30 patients with 51 CRPs (32 adenomas, 6 SSLs, 12 hyperplastic polyps). AI4CRP had a diagnostic accuracy of 80.4%, sensitivity of 82.1%, and specificity of 75.0%. For self-critical AI4CRP (n=37) the diagnostic accuracy was 89.2%, sensitivity 89.7%, and specificity 87.5%. CAD EYE (n=49) had a diagnostic accuracy of 83.7%, sensitivity of 74.2%, and specificity of 100.0%. For the expert endoscopist the diagnostic accuracy was 88.2%, sensitivity 94.9%, and specificity 66.7%