Barrett's lesion detection using a minimal integer-based neural network for embedded systems integration

Embedded processing architectures are often integrated into devices to develop novel functions in a cost-effective medical system. In order to integrate neural networks in medical equipment, these models require specialized optimizations for preparing their integration in a high-efficiency and power-constrained environment. In this paper, we research the feasibility of quantized networks with limited memory for the detection of Barrett’s neoplasia. An Efficientnet-lite1+Deeplabv3 architecture is proposed, which is trained using a quantization-aware training scheme, in order to achieve an 8-bit integer-based model. The performance of the quantized model is comparable with float32 precision models. We show that the quantized model with only 5-MB memory is capable of reaching the same performance scores with 95% Area Under the Curve (AUC), compared to a fullprecision U-Net architecture, which is 10× larger. We have also optimized the segmentation head for efficiency and reduced the output to a resolution of 32×32 pixels. The results show that this resolution captures sufficient segmentation detail to reach a DICE score of 66.51%, which is comparable to the full floating-point model. The proposed lightweight approach also makes the model quite energy-efficient, since it can be real-time executed on a 2-Watt Coral Edge TPU. The obtained low power consumption of the lightweight Barrett’s esophagus neoplasia detection and segmentation system enables the direct integration into standard endoscopic equipment

Evaluating Self-Supervised Learning Methods for Downstream Classification of Neoplasia in Barrett’s Esophagus

A major problem in applying machine learning for the medical domain is the scarcity of labeled data, which results in the demand for methods that enable high-quality models trained with little to no labels. Self-supervised learning methods present a plausible solution to this problem, enabling the use of large sets of unlabeled data for model pretraining. In this study, multiple of these methods and training strategies are employed on a large dataset of endoscopic images from the gastrointestinal tract (GastroNet). The suitability of these methods is assessed for an intra-domain downstream classification task on a small endoscopic dataset, involving neoplasia in Barrett’s esophagus. The classification performances are compared against pretraining on ImageNet and training from scratch. This yields promising results for domain-specific self-supervised methods, where super-resolution outperforms pretraining on ImageNet with a mean classification accuracy of 83.8% (cf. 79.2%). This implies that the large amounts of unlabeled data in hospitals could be employed in combination with self-supervised learning methods to improve models for downstream tasks

Multi-stage domain-specific pretraining for improved detection and localization of Barrett's neoplasia: A comprehensive clinically validated study

Patients suffering from Barrett's Esophagus (BE) are at an increased risk of developing esophageal adenocarcinoma and early detection is crucial for a good prognosis. To aid the endoscopists with the early detection for this preliminary stage of esophageal cancer, this work concentrates on the development and extensive evaluation of a state-of-the-art computer-aided classification and localization algorithm for dysplastic lesions in BE. To this end, we have employed a large-scale endoscopic data set, consisting of 494,355 images, in combination with a novel semi-supervised learning algorithm to pretrain several instances of the proposed neural network architecture. Next, several Barrett-specific data sets that are increasingly closer to the target domain with significantly more data compared to other related work, were used in a multi-stage transfer learning strategy. Additionally, the algorithm was evaluated on two prospectively gathered external test sets and compared against 53 medical professionals. Finally, the model was also evaluated in a live setting without interfering with the current biopsy protocol. Results from the performed experiments show that the proposed model improves on the state-of-the-art on all measured metrics. More specifically, compared to the best performing state-of-the-art model, the specificity is improved by more than 20% points while simultaneously preserving high sensitivity and reducing the false positive rate substantially. Our algorithm yields similar scores on the localization metrics, where the intersection of all experts is correctly indicated in approximately 92% of the cases. Furthermore, the live pilot study shows great performance in a clinical setting with a patient level accuracy, sensitivity, and specificity of 90%. Finally, the proposed algorithm outperforms each individual medical expert by at least 5% and the average assessor by more than 10% over all assessor groups with respect to accuracy

Real-time Barrett's neoplasia characterization in NBI videos using an int8-based quantized neural network

Computer-Aided Diagnosis (CADx) systems for characterization of Narrow-Band Imaging (NBI) videos of suspected lesions in Barrett’s Esophagus (BE) can assist endoscopists during endoscopic surveillance. The real clinical value and application of such CADx systems lies in real-time analysis of endoscopic videos inside the endoscopy suite, placing demands on robustness in decision making and insightful classification matching with the clinical opinions. In this paper, we propose a lightweight int8-based quantized neural network architecture supplemented with an efficient stability function on the output for real-time classification of NBI videos. The proposed int8-architecture has low-memory footprint (4.8 MB), enabling operation on a range of edge devices and even existing endoscopy equipment. Moreover, the stability function ensures robust inclusion of temporal information from the video to provide a continuously stable video classification. The algorithm is trained, validated and tested with a total of 3,799 images and 284 videos of in total 598 patients, collected from 7 international centers. Several stability functions are experimented with, some of them being clinically inspired by weighing low-confidence predictions. For the detection of early BE neoplasia, the proposed algorithm achieves a performance of 92.8% accuracy, 95.7% sensitivity, and 91.4% specificity, while only 5.6% of the videos are without a final video classification. This work shows a robust, lightweight and effective deep learning-based CADx system for accurate automated real-time endoscopic video analysis, suited for embedding in endoscopy clinical practice

Blue-light imaging and linked-color imaging improve visualization of Barrett's neoplasia by nonexpert endoscopists

BACKGROUND AND AIMS: Endoscopic recognition of early Barrett's neoplasia is challenging. Blue-light imaging (BLI) and linked-color imaging (LCI) may assist endoscopists in appreciation of neoplasia. Our aim was to evaluate BLI and LCI for visualization of Barrett's neoplasia in comparison with white-light endoscopy (WLE) alone, when assessed by nonexpert endoscopists. METHODS: In this web-based assessment, corresponding WLE, BLI, and LCI images of 30 neoplastic Barrett's lesions were delineated by 3 expert endoscopists to establish ground truth. These images were then scored and delineated by 76 nonexpert endoscopists from 3 countries and with different levels of expertise, in 4 separate assessment phases with a washout period of 2 weeks. Assessments were as follows: assessment 1, WLE only; assessment 2, WLE + BLI; assessment 3, WLE + LCI; assessment 4, WLE + BLI + LCI. The outcomes were (1) appreciation of macroscopic appearance and ability to delineate lesions (visual analog scale [VAS] scores); (2) preferred technique (ordinal scores); and (3) assessors' delineation performance in terms of overlap with expert ground truth. RESULTS: Median VAS scores for phases 2 to 4 were significantly higher than in phase 1 (P < .001). Assessors preferred BLI and LCI over WLE for appreciation of macroscopic appearance (P < .001) and delineation (P < .001). Linear mixed-effect models showed that delineation performance increased significantly in phase 4. CONCLUSIONS: The use of BLI and LCI has significant additional value for the visualization of Barrett's neoplasia when used by nonexpert endoscopists. Assessors appreciated the addition of BLI and LCI better than the use of WLE alone. Furthermore, this addition led to improved delineation performance, thereby allowing for better acquisition of targeted biopsy samples. (The Netherlands Trial Registry number: NL7541.).status: publishe

A CAD System for Real-Time Characterization of Neoplasia in Barrett's Esophagus NBI Videos

Barrett’s Esophagus (BE) is a well-known precursor for Esophageal Adenocarcinoma (EAC). Endoscopic detection and diagnosisof early BE neoplasia is performed in two steps: primary detection of asuspected lesion in overview and a targeted and detailed inspection of thespecific area using Narrow-Band Imaging (NBI). Despite the improvedvisualization of tissue by NBI and clinical classification systems, endoscopists have difficulties with correct characterization of the imagery.Computer-aided Diagnosis (CADx) may assist endoscopists in the classification of abnormalities in NBI imagery. We propose an endoscopydriven pre-trained deep learning-based CADx, for the characterization of NBI imagery of BE. We evaluate the performance of the algorithm on images as well as on videos, for which we use several post-hoc and real-time video analysis methods. The proposed real-time methods outperform the post-hoc methods on average by 1.2% and 2.3% for accuracy and specificity, respectively. The obtained results show promising methods towards real-time endoscopic video analysis and identifies steps forfurther development

Comparing Training Strategies Using Multi-Assessor Segmentation Labels for Barrett’s Neoplasia Detection

In medical imaging, segmentation ground truths generally suffer from large inter-observer variability. When multiple observers are used, simple fusion techniques are typically employed to combine multiple delineations into one consensus ground truth. However, in this process, potentially valuable information is discarded and it is yet unknown what strategy leads to optimal segmentation results. In this work, we compare several ground-truth types to train a U-net and apply it to the clinical use case of Barrett’s neoplasia detection. To this end, we have invited 14 international Barrett’s experts to delineate 2,851 neoplastic images derived from 812 patients into a higher- and lower-likelihood neoplasia areas. Five different ground-truths techniques along with four different training losses are compared with each other using the Area-under-the-curve (AUC) value for Barrett’s neoplasia detection. The value used to generate this curve is the maximum pixel value in the raw segmentation map, and the histologically proven ground truth of the image. The experiments show that random sampling of the four neoplastic areas together with a compound loss Binary Cross-entropy and DICE yields the highest value of 94.12%, while fusion-based ground truth clearly performs lower. The results show that researchers should incorporate measures for uncertainty in their design of networks

Blue-light imaging and linked-color imaging improve visualization of Barrett's neoplasia by nonexpert endoscopists

Background and Aims: Endoscopic recognition of early Barrett's neoplasia is challenging. Blue-light imaging (BLI) and linked-color imaging (LCI) may assist endoscopists in appreciation of neoplasia. Our aim was to evaluate BLI and LCI for visualization of Barrett's neoplasia in comparison with white-light endoscopy (WLE) alone, when assessed by nonexpert endoscopists. Methods: In this web-based assessment, corresponding WLE, BLI, and LCI images of 30 neoplastic Barrett's lesions were delineated by 3 expert endoscopists to establish ground truth. These images were then scored and delineated by 76 nonexpert endoscopists from 3 countries and with different levels of expertise, in 4 separate assessment phases with a washout period of 2 weeks. Assessments were as follows: assessment 1, WLE only; assessment 2, WLE + BLI; assessment 3, WLE + LCI; assessment 4, WLE + BLI + LCI. The outcomes were (1) appreciation of macroscopic appearance and ability to delineate lesions (visual analog scale [VAS] scores); (2) preferred technique (ordinal scores); and (3) assessors’ delineation performance in terms of overlap with expert ground truth. Results: Median VAS scores for phases 2 to 4 were significantly higher than in phase 1 (P < .001). Assessors preferred BLI and LCI over WLE for appreciation of macroscopic appearance (P < .001) and delineation (P < .001). Linear mixed-effect models showed that delineation performance increased significantly in phase 4. Conclusions: The use of BLI and LCI has significant additional value for the visualization of Barrett's neoplasia when used by nonexpert endoscopists. Assessors appreciated the addition of BLI and LCI better than the use of WLE alone. Furthermore, this addition led to improved delineation performance, thereby allowing for better acquisition of targeted biopsy samples. (The Netherlands Trial Registry number: NL7541.

CNNs vs. Transformers:Performance and Robustness in Endoscopic Image Analysis

In endoscopy, imaging conditions are often challenging due to organ movement, user dependence, fluctuations in video quality and real-time processing, which pose requirements on the performance, robustness and complexity of computer-based analysis techniques. This paper poses the question whether Transformer-based architectures, which are capable to directly capture global contextual information, can handle the aforementioned endoscopic conditions and even outperform the established Convolutional Neural Networks (CNNs) for this task. To this end, we evaluate and compare clinically relevant performance and robustness of CNNs and Transformers for neoplasia detection in Barrett’s esophagus. We have selected several top performing CNN and Transformers on endoscopic benchmarks, which we have trained and validated on a total of 10,208 images (2,079 patients), and tested on a total of 4,661 images (743 patients), divided over a high-quality test set and three different robustness test sets. Our results show that Transformers generally perform better on classification and segmentation for the high-quality challenging test set, and show on-par or increased robustness to various clinically relevant input data variations, while requiring comparable model complexity. This robustness against challenging video-related conditions and equipment variations over the hospitals is an essential trait for adoption in clinical practice. The code is made publicly available at: https://github.com/BONS-AI-VCA-AMC/Endoscopy-CNNs-vs-Transformers

Efficient endoscopic frame informativeness assessment by reusing the encoder of the primary CAD task

The majority of the encouraging experimental results published on AI-based endoscopic Computer-Aided Detection (CAD) systems have not yet been reproduced in clinical settings, mainly due to highly curated datasets used throughout the experimental phase of the research. In a realistic clinical environment, these necessary high image-quality standards cannot be guaranteed, and the CAD system performance may degrade. While several studies have previously presented impressive outcomes with Frame Informativeness Assessment (FIA) algorithms, the current-state of the art implies sequential use of FIA and CAD systems, affecting the time performance of both algorithms. Since these algorithms are often trained on similar datasets, we hypothesise that part of the learned feature representations can be leveraged for both systems, enabling a more efficient implementation. This paper explores this case for early Barrett cancer detection by integrating the FIA algorithm within the CAD system. Sharing the weights between two tasks reduces the number of parameters from 16 to 11 million and the number of floating-point operations from 502 to 452 million. Due to the lower complexity of the architecture, the proposed model leads to inference time up to 2 times faster than the state-of-The-Art sequential implementation while retaining the classification performance