Barrett's lesion detection using a minimal integer-based neural network for embedded systems integration

Embedded processing architectures are often integrated into devices to develop novel functions in a cost-effective medical system. In order to integrate neural networks in medical equipment, these models require specialized optimizations for preparing their integration in a high-efficiency and power-constrained environment. In this paper, we research the feasibility of quantized networks with limited memory for the detection of Barrett’s neoplasia. An Efficientnet-lite1+Deeplabv3 architecture is proposed, which is trained using a quantization-aware training scheme, in order to achieve an 8-bit integer-based model. The performance of the quantized model is comparable with float32 precision models. We show that the quantized model with only 5-MB memory is capable of reaching the same performance scores with 95% Area Under the Curve (AUC), compared to a fullprecision U-Net architecture, which is 10× larger. We have also optimized the segmentation head for efficiency and reduced the output to a resolution of 32×32 pixels. The results show that this resolution captures sufficient segmentation detail to reach a DICE score of 66.51%, which is comparable to the full floating-point model. The proposed lightweight approach also makes the model quite energy-efficient, since it can be real-time executed on a 2-Watt Coral Edge TPU. The obtained low power consumption of the lightweight Barrett’s esophagus neoplasia detection and segmentation system enables the direct integration into standard endoscopic equipment

The use of a real-time computer-aided detection system for visible lesions in the Barrett's esophagus during live endoscopic procedures: a pilot study (with video)

Background and aims: This pilot study evaluated the performance of a recently developed computer-aided detection (CADe) system for Barrett's neoplasia during live endoscopic procedures. Methods: Fifteen patients with a visible lesion and 15 without were included in this study. A CAD-assisted workflow was used that included a slow pullback video recording of the entire Barrett's segment with live CADe assistance, followed by CADe-assisted level-based video recordings every 2 cm of the Barrett's segment. Outcomes were per-patient and per-level diagnostic accuracy of the CAD-assisted workflow, in which the primary outcome was per-patient in vivo CADe sensitivity. Results: In the per-patient analyses, the CADe system detected all visible lesions (sensitivity 100%). Per-patient CADe specificity was 53%. Per-level sensitivity and specificity of the CADe assisted workflow were 100% and 73%, respectively. Conclusions: In this pilot study, detection by the CADe system of all potentially neoplastic lesions in Barrett's esophagus was comparable to that of an expert endoscopist. Continued refinement of the system may improve specificity. External validation in larger multicenter studies is planned. (Clinical trial registration number: NCT05628441.

Towards a robust and compact deep learning system for primary detection of early Barrett’s neoplasia: Initial image-based results of training on a multi-center retrospectively collected data set

Introduction: Endoscopic detection of early neoplasia in Barrett's esophagus is difficult. Computer Aided Detection (CADe) systems may assist in neoplasia detection. The aim of this study was to report the first steps in the development of a CADe system for Barrett's neoplasia and to evaluate its performance when compared with endoscopists. Methods: This CADe system was developed by a consortium, consisting of the Amsterdam University Medical Center, Eindhoven University of Technology, and 15 international hospitals. After pretraining, the system was trained and validated using 1.713 neoplastic (564 patients) and 2.707 non-dysplastic Barrett's esophagus (NDBE; 665 patients) images. Neoplastic lesions were delineated by 14 experts. The performance of the CADe system was tested on three independent test sets. Test set 1 (50 neoplastic and 150 NDBE images) contained subtle neoplastic lesions representing challenging cases and was benchmarked by 52 general endoscopists. Test set 2 (50 neoplastic and 50 NDBE images) contained a heterogeneous case-mix of neoplastic lesions, representing distribution in clinical practice. Test set 3 (50 neoplastic and 150 NDBE images) contained prospectively collected imagery. The main outcome was correct classification of the images in terms of sensitivity. Results: The sensitivity of the CADe system on test set 1 was 84%. For general endoscopists, sensitivity was 63%, corresponding to a neoplasia miss-rate of one-third of neoplastic lesions and a potential relative increase in neoplasia detection of 33% for CADe-assisted detection. The sensitivity of the CADe system on test sets 2 and 3 was 100% and 88%, respectively. The specificity of the CADe system varied for the three test sets between 64% and 66%. Conclusion: This study describes the first steps towards the establishment of an unprecedented data infrastructure for using machine learning to improve the endoscopic detection of Barrett's neoplasia. The CADe system detected neoplasia reliably and outperformed a large group of endoscopists in terms of sensitivity

Evaluating Self-Supervised Learning Methods for Downstream Classification of Neoplasia in Barrett’s Esophagus

A major problem in applying machine learning for the medical domain is the scarcity of labeled data, which results in the demand for methods that enable high-quality models trained with little to no labels. Self-supervised learning methods present a plausible solution to this problem, enabling the use of large sets of unlabeled data for model pretraining. In this study, multiple of these methods and training strategies are employed on a large dataset of endoscopic images from the gastrointestinal tract (GastroNet). The suitability of these methods is assessed for an intra-domain downstream classification task on a small endoscopic dataset, involving neoplasia in Barrett’s esophagus. The classification performances are compared against pretraining on ImageNet and training from scratch. This yields promising results for domain-specific self-supervised methods, where super-resolution outperforms pretraining on ImageNet with a mean classification accuracy of 83.8% (cf. 79.2%). This implies that the large amounts of unlabeled data in hospitals could be employed in combination with self-supervised learning methods to improve models for downstream tasks

Multi-stage domain-specific pretraining for improved detection and localization of Barrett's neoplasia: A comprehensive clinically validated study

Patients suffering from Barrett's Esophagus (BE) are at an increased risk of developing esophageal adenocarcinoma and early detection is crucial for a good prognosis. To aid the endoscopists with the early detection for this preliminary stage of esophageal cancer, this work concentrates on the development and extensive evaluation of a state-of-the-art computer-aided classification and localization algorithm for dysplastic lesions in BE. To this end, we have employed a large-scale endoscopic data set, consisting of 494,355 images, in combination with a novel semi-supervised learning algorithm to pretrain several instances of the proposed neural network architecture. Next, several Barrett-specific data sets that are increasingly closer to the target domain with significantly more data compared to other related work, were used in a multi-stage transfer learning strategy. Additionally, the algorithm was evaluated on two prospectively gathered external test sets and compared against 53 medical professionals. Finally, the model was also evaluated in a live setting without interfering with the current biopsy protocol. Results from the performed experiments show that the proposed model improves on the state-of-the-art on all measured metrics. More specifically, compared to the best performing state-of-the-art model, the specificity is improved by more than 20% points while simultaneously preserving high sensitivity and reducing the false positive rate substantially. Our algorithm yields similar scores on the localization metrics, where the intersection of all experts is correctly indicated in approximately 92% of the cases. Furthermore, the live pilot study shows great performance in a clinical setting with a patient level accuracy, sensitivity, and specificity of 90%. Finally, the proposed algorithm outperforms each individual medical expert by at least 5% and the average assessor by more than 10% over all assessor groups with respect to accuracy

Real-time Barrett's neoplasia characterization in NBI videos using an int8-based quantized neural network

Computer-Aided Diagnosis (CADx) systems for characterization of Narrow-Band Imaging (NBI) videos of suspected lesions in Barrett’s Esophagus (BE) can assist endoscopists during endoscopic surveillance. The real clinical value and application of such CADx systems lies in real-time analysis of endoscopic videos inside the endoscopy suite, placing demands on robustness in decision making and insightful classification matching with the clinical opinions. In this paper, we propose a lightweight int8-based quantized neural network architecture supplemented with an efficient stability function on the output for real-time classification of NBI videos. The proposed int8-architecture has low-memory footprint (4.8 MB), enabling operation on a range of edge devices and even existing endoscopy equipment. Moreover, the stability function ensures robust inclusion of temporal information from the video to provide a continuously stable video classification. The algorithm is trained, validated and tested with a total of 3,799 images and 284 videos of in total 598 patients, collected from 7 international centers. Several stability functions are experimented with, some of them being clinically inspired by weighing low-confidence predictions. For the detection of early BE neoplasia, the proposed algorithm achieves a performance of 92.8% accuracy, 95.7% sensitivity, and 91.4% specificity, while only 5.6% of the videos are without a final video classification. This work shows a robust, lightweight and effective deep learning-based CADx system for accurate automated real-time endoscopic video analysis, suited for embedding in endoscopy clinical practice

Blue-light imaging and linked-color imaging improve visualization of Barrett's neoplasia by nonexpert endoscopists

BACKGROUND AND AIMS: Endoscopic recognition of early Barrett's neoplasia is challenging. Blue-light imaging (BLI) and linked-color imaging (LCI) may assist endoscopists in appreciation of neoplasia. Our aim was to evaluate BLI and LCI for visualization of Barrett's neoplasia in comparison with white-light endoscopy (WLE) alone, when assessed by nonexpert endoscopists. METHODS: In this web-based assessment, corresponding WLE, BLI, and LCI images of 30 neoplastic Barrett's lesions were delineated by 3 expert endoscopists to establish ground truth. These images were then scored and delineated by 76 nonexpert endoscopists from 3 countries and with different levels of expertise, in 4 separate assessment phases with a washout period of 2 weeks. Assessments were as follows: assessment 1, WLE only; assessment 2, WLE + BLI; assessment 3, WLE + LCI; assessment 4, WLE + BLI + LCI. The outcomes were (1) appreciation of macroscopic appearance and ability to delineate lesions (visual analog scale [VAS] scores); (2) preferred technique (ordinal scores); and (3) assessors' delineation performance in terms of overlap with expert ground truth. RESULTS: Median VAS scores for phases 2 to 4 were significantly higher than in phase 1 (P < .001). Assessors preferred BLI and LCI over WLE for appreciation of macroscopic appearance (P < .001) and delineation (P < .001). Linear mixed-effect models showed that delineation performance increased significantly in phase 4. CONCLUSIONS: The use of BLI and LCI has significant additional value for the visualization of Barrett's neoplasia when used by nonexpert endoscopists. Assessors appreciated the addition of BLI and LCI better than the use of WLE alone. Furthermore, this addition led to improved delineation performance, thereby allowing for better acquisition of targeted biopsy samples. (The Netherlands Trial Registry number: NL7541.).status: publishe

A CAD System for Real-Time Characterization of Neoplasia in Barrett's Esophagus NBI Videos

Barrett’s Esophagus (BE) is a well-known precursor for Esophageal Adenocarcinoma (EAC). Endoscopic detection and diagnosisof early BE neoplasia is performed in two steps: primary detection of asuspected lesion in overview and a targeted and detailed inspection of thespecific area using Narrow-Band Imaging (NBI). Despite the improvedvisualization of tissue by NBI and clinical classification systems, endoscopists have difficulties with correct characterization of the imagery.Computer-aided Diagnosis (CADx) may assist endoscopists in the classification of abnormalities in NBI imagery. We propose an endoscopydriven pre-trained deep learning-based CADx, for the characterization of NBI imagery of BE. We evaluate the performance of the algorithm on images as well as on videos, for which we use several post-hoc and real-time video analysis methods. The proposed real-time methods outperform the post-hoc methods on average by 1.2% and 2.3% for accuracy and specificity, respectively. The obtained results show promising methods towards real-time endoscopic video analysis and identifies steps forfurther development

Comparing Training Strategies Using Multi-Assessor Segmentation Labels for Barrett’s Neoplasia Detection

In medical imaging, segmentation ground truths generally suffer from large inter-observer variability. When multiple observers are used, simple fusion techniques are typically employed to combine multiple delineations into one consensus ground truth. However, in this process, potentially valuable information is discarded and it is yet unknown what strategy leads to optimal segmentation results. In this work, we compare several ground-truth types to train a U-net and apply it to the clinical use case of Barrett’s neoplasia detection. To this end, we have invited 14 international Barrett’s experts to delineate 2,851 neoplastic images derived from 812 patients into a higher- and lower-likelihood neoplasia areas. Five different ground-truths techniques along with four different training losses are compared with each other using the Area-under-the-curve (AUC) value for Barrett’s neoplasia detection. The value used to generate this curve is the maximum pixel value in the raw segmentation map, and the histologically proven ground truth of the image. The experiments show that random sampling of the four neoplastic areas together with a compound loss Binary Cross-entropy and DICE yields the highest value of 94.12%, while fusion-based ground truth clearly performs lower. The results show that researchers should incorporate measures for uncertainty in their design of networks

Blue-light imaging and linked-color imaging improve visualization of Barrett's neoplasia by nonexpert endoscopists

Background and Aims: Endoscopic recognition of early Barrett's neoplasia is challenging. Blue-light imaging (BLI) and linked-color imaging (LCI) may assist endoscopists in appreciation of neoplasia. Our aim was to evaluate BLI and LCI for visualization of Barrett's neoplasia in comparison with white-light endoscopy (WLE) alone, when assessed by nonexpert endoscopists. Methods: In this web-based assessment, corresponding WLE, BLI, and LCI images of 30 neoplastic Barrett's lesions were delineated by 3 expert endoscopists to establish ground truth. These images were then scored and delineated by 76 nonexpert endoscopists from 3 countries and with different levels of expertise, in 4 separate assessment phases with a washout period of 2 weeks. Assessments were as follows: assessment 1, WLE only; assessment 2, WLE + BLI; assessment 3, WLE + LCI; assessment 4, WLE + BLI + LCI. The outcomes were (1) appreciation of macroscopic appearance and ability to delineate lesions (visual analog scale [VAS] scores); (2) preferred technique (ordinal scores); and (3) assessors’ delineation performance in terms of overlap with expert ground truth. Results: Median VAS scores for phases 2 to 4 were significantly higher than in phase 1 (P < .001). Assessors preferred BLI and LCI over WLE for appreciation of macroscopic appearance (P < .001) and delineation (P < .001). Linear mixed-effect models showed that delineation performance increased significantly in phase 4. Conclusions: The use of BLI and LCI has significant additional value for the visualization of Barrett's neoplasia when used by nonexpert endoscopists. Assessors appreciated the addition of BLI and LCI better than the use of WLE alone. Furthermore, this addition led to improved delineation performance, thereby allowing for better acquisition of targeted biopsy samples. (The Netherlands Trial Registry number: NL7541.