140 research outputs found
Recommended from our members
Artificial Intelligence in Gastrointestinal Endoscopy.
Artificial intelligence (AI) is rapidly integrating into modern technology and clinical practice. Although in its nascency, AI has become a hot topic of investigation for applications in clinical practice. Multiple fields of medicine have embraced the possibility of a future with AI assisting in diagnosis and pathology applications. In the field of gastroenterology, AI has been studied as a tool to assist in risk stratification, diagnosis, and pathologic identification. Specifically, AI has become of great interest in endoscopy as a technology with substantial potential to revolutionize the practice of a modern gastroenterologist. From cancer screening to automated report generation, AI has touched upon all aspects of modern endoscopy. Here, we review landmark AI developments in endoscopy. Starting with broad definitions to develop understanding, we will summarize the current state of AI research and its potential applications. With innovation developing rapidly, this article touches upon the remarkable advances in AI-assisted endoscopy since its initial evaluation at the turn of the millennium, and the potential impact these AI models may have on the modern clinical practice. As with any discussion of new technology, its limitations must also be understood to apply clinical AI tools successfully
μμμ κΈ° ν₯μμ μν λ₯λ¬λ κΈ°λ² μ°κ΅¬: λμ₯λ΄μκ²½ μ§λ¨ λ° λ‘λ΄μμ μ κΈ° νκ°μ μ μ©
νμλ
Όλ¬Έ (λ°μ¬) -- μμΈλνκ΅ λνμ : 곡과λν νλκ³Όμ μμ©μ체곡νμ 곡, 2020. 8. κΉν¬μ°¬.This paper presents deep learning-based methods for improving performance of clinicians. Novel methods were applied to the following two clinical cases and the results were evaluated.
In the first study, a deep learning-based polyp classification algorithm for improving clinical performance of endoscopist during colonoscopy diagnosis was developed. Colonoscopy is the main method for diagnosing adenomatous polyp, which can multiply into a colorectal cancer and hyperplastic polyps. The classification algorithm was developed using convolutional neural network (CNN), trained with colorectal polyp images taken by a narrow-band imaging colonoscopy. The proposed method is built around an automatic machine learning (AutoML) which searches for the optimal architecture of CNN for colorectal polyp image classification and trains the weights of the architecture. In addition, gradient-weighted class activation mapping technique was used to overlay the probabilistic basis of the prediction result on the polyp location to aid the endoscopists visually. To verify the improvement in diagnostic performance, the efficacy of endoscopists with varying proficiency levels were compared with or without the aid of the proposed polyp classification algorithm. The results confirmed that, on average, diagnostic accuracy was improved and diagnosis time was shortened in all proficiency groups significantly.
In the second study, a surgical instruments tracking algorithm for robotic surgery video was developed, and a model for quantitatively evaluating the surgeons surgical skill based on the acquired motion information of the surgical instruments was proposed. The movement of surgical instruments is the main component of evaluation for surgical skill. Therefore, the focus of this study was develop an automatic surgical instruments tracking algorithm, and to overcome the limitations presented by previous methods. The instance segmentation framework was developed to solve the instrument occlusion issue, and a tracking framework composed of a tracker and a re-identification algorithm was developed to maintain the type of surgical instruments being tracked in the video. In addition, algorithms for detecting the tip position of instruments and arm-indicator were developed to acquire the movement of devices specialized for the robotic surgery video. The performance of the proposed method was evaluated by measuring the difference between the predicted tip position and the ground truth position of the instruments using root mean square error, area under the curve, and Pearsons correlation analysis. Furthermore, motion metrics were calculated from the movement of surgical instruments, and a machine learning-based robotic surgical skill evaluation model was developed based on these metrics. These models were used to evaluate clinicians, and results were similar in the developed evaluation models, the Objective Structured Assessment of Technical Skill (OSATS), and the Global Evaluative Assessment of Robotic Surgery (GEARS) evaluation methods.
In this study, deep learning technology was applied to colorectal polyp images for a polyp classification, and to robotic surgery videos for surgical instruments tracking. The improvement in clinical performance with the aid of these methods were evaluated and verified.λ³Έ λ
Όλ¬Έμ μλ£μ§μ μμμ κΈ° λ₯λ ₯μ ν₯μμν€κΈ° μνμ¬ μλ‘μ΄ λ₯λ¬λ κΈ°λ²λ€μ μ μνκ³ λ€μ λ κ°μ§ μ€λ‘μ λν΄ μ μ©νμ¬ κ·Έ κ²°κ³Όλ₯Ό νκ°νμλ€.
첫 λ²μ§Έ μ°κ΅¬μμλ λμ₯λ΄μκ²½μΌλ‘ κ΄ν μ§λ¨ μ, λ΄μκ²½ μ λ¬Έμμ μ§λ¨ λ₯λ ₯μ ν₯μμν€κΈ° μνμ¬ λ₯λ¬λ κΈ°λ°μ μ©μ’
λΆλ₯ μκ³ λ¦¬μ¦μ κ°λ°νκ³ , λ΄μκ²½ μ λ¬Έμμ μ§λ¨ λ₯λ ₯ ν₯μ μ¬λΆλ₯Ό κ²μ¦νκ³ μ νμλ€. λμ₯λ΄μκ²½ κ²μ¬λ‘ μμ’
μΌλ‘ μ¦μν μ μλ μ μ’
κ³Ό κ³Όμ¦μμ± μ©μ’
μ μ§λ¨νλ κ²μ μ€μνλ€. λ³Έ μ°κ΅¬μμλ νλμ μμ λ΄μκ²½μΌλ‘ 촬μν λμ₯ μ©μ’
μμμΌλ‘ ν©μ±κ³± μ κ²½λ§μ νμ΅νμ¬ λΆλ₯ μκ³ λ¦¬μ¦μ κ°λ°νμλ€. μ μνλ μκ³ λ¦¬μ¦μ μλ κΈ°κ³νμ΅ (AutoML) λ°©λ²μΌλ‘, λμ₯ μ©μ’
μμμ μ΅μ νλ ν©μ±κ³± μ κ²½λ§ κ΅¬μ‘°λ₯Ό μ°Ύκ³ μ κ²½λ§μ κ°μ€μΉλ₯Ό νμ΅νμλ€. λν κΈ°μΈκΈ°-κ°μ€μΉ ν΄λμ€ νμ±ν 맡ν κΈ°λ²μ μ΄μ©νμ¬ κ°λ°ν ν©μ±κ³± μ κ²½λ§ κ²°κ³Όμ νλ₯ μ κ·Όκ±°λ₯Ό μ©μ’
μμΉμ μκ°μ μΌλ‘ λνλλλ‘ ν¨μΌλ‘ λ΄μκ²½ μ λ¬Έμμ μ§λ¨μ λλλ‘ νμλ€. λ§μ§λ§μΌλ‘, μλ ¨λ κ·Έλ£Ήλ³λ‘ λ΄μκ²½ μ λ¬Έμκ° μ©μ’
λΆλ₯ μκ³ λ¦¬μ¦μ κ²°κ³Όλ₯Ό μ°Έκ³ νμμ λ μ§λ¨ λ₯λ ₯μ΄ ν₯μλμλμ§ λΉκ΅ μ€νμ μ§ννμκ³ , λͺ¨λ κ·Έλ£Ήμμ μ μλ―Ένκ² μ§λ¨ μ νλκ° ν₯μλκ³ μ§λ¨ μκ°μ΄ λ¨μΆλμμμ νμΈνμλ€.
λ λ²μ§Έ μ°κ΅¬μμλ λ‘λ΄μμ λμμμμ μμ λꡬ μμΉ μΆμ μκ³ λ¦¬μ¦μ κ°λ°νκ³ , νλν μμ λꡬμ μμ§μ μ 보λ₯Ό λ°νμΌλ‘ μμ μμ μλ ¨λλ₯Ό μ λμ μΌλ‘ νκ°νλ λͺ¨λΈμ μ μνμλ€. μμ λꡬμ μμ§μμ μμ μμ λ‘λ΄μμ μλ ¨λλ₯Ό νκ°νκΈ° μν μ£Όμν μ 보μ΄λ€. λ°λΌμ λ³Έ μ°κ΅¬λ λ₯λ¬λ κΈ°λ°μ μλ μμ λꡬ μΆμ μκ³ λ¦¬μ¦μ κ°λ°νμμΌλ©°, λ€μ λκ°μ§ μ νμ°κ΅¬μ νκ³μ μ 극볡νμλ€. μΈμ€ν΄μ€ λΆν (Instance Segmentation) νλ μμμ κ°λ°νμ¬ νμ (Occlusion) λ¬Έμ λ₯Ό ν΄κ²°νμκ³ , μΆμ κΈ° (Tracker)μ μ¬μλ³ν (Re-Identification) μκ³ λ¦¬μ¦μΌλ‘ ꡬμ±λ μΆμ νλ μμμ κ°λ°νμ¬ λμμμμ μΆμ νλ μμ λꡬμ μ’
λ₯κ° μ μ§λλλ‘ νμλ€. λν λ‘λ΄μμ λμμμ νΉμμ±μ κ³ λ €νμ¬ μμ λꡬμ μμ§μμ νλνκΈ°μν΄ μμ λꡬ λ μμΉμ λ‘λ΄ ν-μΈλμΌμ΄ν° (Arm-Indicator) μΈμ μκ³ λ¦¬μ¦μ κ°λ°νμλ€. μ μνλ μκ³ λ¦¬μ¦μ μ±λ₯μ μμΈ‘ν μμ λꡬ λ μμΉμ μ λ΅ μμΉ κ°μ νκ· μ κ³±κ·Ό μ€μ°¨, 곑μ μλ λ©΄μ , νΌμ΄μ¨ μκ΄λΆμμΌλ‘ νκ°νμλ€. λ§μ§λ§μΌλ‘, μμ λꡬμ μμ§μμΌλ‘λΆν° μμ§μ μ§νλ₯Ό κ³μ°νκ³ μ΄λ₯Ό λ°νμΌλ‘ κΈ°κ³νμ΅ κΈ°λ°μ λ‘λ΄μμ μλ ¨λ νκ° λͺ¨λΈμ κ°λ°νμλ€. κ°λ°ν νκ° λͺ¨λΈμ κΈ°μ‘΄μ Objective Structured Assessment of Technical Skill (OSATS), Global Evaluative Assessment of Robotic Surgery (GEARS) νκ° λ°©λ²κ³Ό μ μ¬ν μ±λ₯μ 보μμ νμΈνμλ€.
λ³Έ λ
Όλ¬Έμ μλ£μ§μ μμμ κΈ° λ₯λ ₯μ ν₯μμν€κΈ° μνμ¬ λμ₯ μ©μ’
μμκ³Ό λ‘λ΄μμ λμμμ λ₯λ¬λ κΈ°μ μ μ μ©νκ³ κ·Έ μ ν¨μ±μ νμΈνμμΌλ©°, ν₯νμ μ μνλ λ°©λ²μ΄ μμμμ μ¬μ©λκ³ μλ μ§λ¨ λ° νκ° λ°©λ²μ λμμ΄ λ κ²μΌλ‘ κΈ°λνλ€.Chapter 1 General Introduction 1
1.1 Deep Learning for Medical Image Analysis 1
1.2 Deep Learning for Colonoscipic Diagnosis 2
1.3 Deep Learning for Robotic Surgical Skill Assessment 3
1.4 Thesis Objectives 5
Chapter 2 Optical Diagnosis of Colorectal Polyps using Deep Learning with Visual Explanations 7
2.1 Introduction 7
2.1.1 Background 7
2.1.2 Needs 8
2.1.3 Related Work 9
2.2 Methods 11
2.2.1 Study Design 11
2.2.2 Dataset 14
2.2.3 Preprocessing 17
2.2.4 Convolutional Neural Networks (CNN) 21
2.2.4.1 Standard CNN 21
2.2.4.2 Search for CNN Architecture 22
2.2.4.3 Searched CNN Training 23
2.2.4.4 Visual Explanation 24
2.2.5 Evaluation of CNN and Endoscopist Performances 25
2.3 Experiments and Results 27
2.3.1 CNN Performance 27
2.3.2 Results of Visual Explanation 31
2.3.3 Endoscopist with CNN Performance 33
2.4 Discussion 45
2.4.1 Research Significance 45
2.4.2 Limitations 47
2.5 Conclusion 49
Chapter 3 Surgical Skill Assessment during Robotic Surgery by Deep Learning-based Surgical Instrument Tracking 50
3.1 Introduction 50
3.1.1 Background 50
3.1.2 Needs 51
3.1.3 Related Work 52
3.2 Methods 56
3.2.1 Study Design 56
3.2.2 Dataset 59
3.2.3 Instance Segmentation Framework 63
3.2.4 Tracking Framework 66
3.2.4.1 Tracker 66
3.2.4.2 Re-identification 68
3.2.5 Surgical Instrument Tip Detection 69
3.2.6 Arm-Indicator Recognition 71
3.2.7 Surgical Skill Prediction Model 71
3.3 Experiments and Results 78
3.3.1 Performance of Instance Segmentation Framework 78
3.3.2 Performance of Tracking Framework 82
3.3.3 Evaluation of Surgical Instruments Trajectory 83
3.3.4 Evaluation of Surgical Skill Prediction Model 86
3.4 Discussion 90
3.4.1 Research Significance 90
3.4.2 Limitations 92
3.5 Conclusion 96
Chapter 4 Summary and Future Works 97
4.1 Thesis Summary 97
4.2 Limitations and Future Works 98
Bibliography 100
Abstract in Korean 116
Acknowledgement 119Docto
Endoscopic Polyp Segmentation Using a Hybrid 2D/3D CNN
Colonoscopy is the gold standard for early diagnosis and pre-emptive treatment of colorectal cancer by detecting and removing colonic polyps. Deep learning approaches to polyp detection have shown potential for enhancing polyp detection rates. However, the majority of these systems are developed and evaluated on static images from colonoscopies, whilst applied treatment is performed on a real-time video feed. Non-curated video data includes a high proportion of low-quality frames in comparison to selected images but also embeds temporal information that can be used for more stable predictions. To exploit this, a hybrid 2D/3D convolutional neural network architecture is presented. The network is used to improve polyp detection by encompassing spatial and temporal correlation of the predictions while preserving real-time detections. Extensive experiments show that the hybrid method outperforms a 2D baseline. The proposed architecture is validated on videos from 46 patients. The results show that real-world clinical implementations of automated polyp detection can benefit from the hybrid algorithm
Rethinking the transfer learning for FCN based polyp segmentation in colonoscopy
Besides the complex nature of colonoscopy frames with intrinsic frame
formation artefacts such as light reflections and the diversity of polyp
types/shapes, the publicly available polyp segmentation training datasets are
limited, small and imbalanced. In this case, the automated polyp segmentation
using a deep neural network remains an open challenge due to the overfitting of
training on small datasets. We proposed a simple yet effective polyp
segmentation pipeline that couples the segmentation (FCN) and classification
(CNN) tasks. We find the effectiveness of interactive weight transfer between
dense and coarse vision tasks that mitigates the overfitting in learning. And
It motivates us to design a new training scheme within our segmentation
pipeline. Our method is evaluated on CVC-EndoSceneStill and Kvasir-SEG
datasets. It achieves 4.34% and 5.70% Polyp-IoU improvements compared to the
state-of-the-art methods on the EndoSceneStill and Kvasir-SEG datasets,
respectively.Comment: 11 pages, 10 figures, submit versio
Enhancing endoscopic navigation and polyp detection using artificial intelligence
Colorectal cancer (CRC) is one most common and deadly forms of cancer. It has a very high mortality rate if the disease advances to late stages however early diagnosis and treatment can be curative is hence essential to enhancing disease management. Colonoscopy is considered the gold standard for CRC screening and early therapeutic treatment. The effectiveness of colonoscopy is highly dependent on the operatorβs skill, as a high level of hand-eye coordination is required to control the endoscope and fully examine the colon wall. Because of this, detection rates can vary between different gastroenterologists and technology have been proposed as solutions to assist disease detection and standardise detection rates. This thesis focuses on developing artificial intelligence algorithms to assist gastroenterologists during colonoscopy with the potential to ensure a baseline standard of quality in CRC screening. To achieve such assistance, the technical contributions develop deep learning methods and architectures for automated endoscopic image analysis to address both the detection of lesions in the endoscopic image and the 3D mapping of the endoluminal environment. The proposed detection models can run in real-time and assist visualization of different polyp types. Meanwhile the 3D reconstruction and mapping models developed are the basis for ensuring that the entire colon has been examined appropriately and to support quantitative measurement of polyp sizes using the image during a procedure. Results and validation studies presented within the thesis demonstrate how the developed algorithms perform on both general scenes and on clinical data. The feasibility of clinical translation is demonstrated for all of the models on endoscopic data from human participants during CRC screening examinations
Decomposition of color wavelet with higher order statistical texture and convolutional neural network features set based classification of colorectal polyps from video endoscopy
Gastrointestinal cancer is one of the leading causes of death across the world. The gastrointestinal polyps are considered as the precursors of developing this malignant cancer. In order to condense the probability of cancer, early detection and removal of colorectal polyps can be cogitated. The most used diagnostic modality for colorectal polyps is video endoscopy. But the accuracy of diagnosis mostly depends on doctors' experience that is crucial to detect polyps in many cases. Computer-aided polyp detection is promising to reduce the miss detection rate of the polyp and thus improve the accuracy of diagnosis results. The proposed method first detects polyp and non-polyp then illustrates an automatic polyp classification technique from endoscopic video through color wavelet with higher-order statistical texture feature and Convolutional Neural Network (CNN). Gray Level Run Length Matrix (GLRLM) is used for higher-order statistical texture features of different directions (Ζ = 0o, 45o, 90o, 135o). The features are fed into a linear support vector machine (SVM) to train the classifier. The experimental result demonstrates that the proposed approach is auspicious and operative with residual network architecture, which triumphs the best performance of accuracy, sensitivity, and specificity of 98.83%, 97.87%, and 99.13% respectively for classification of colorectal polyps on standard public endoscopic video databases
Classification of Colorectal Cancer Polyps via Transfer Learning and Vision-Based Tactile Sensing
In this study, to address the current high earlydetection miss rate of
colorectal cancer (CRC) polyps, we explore the potentials of utilizing transfer
learning and machine learning (ML) classifiers to precisely and sensitively
classify the type of CRC polyps. Instead of using the common colonoscopic
images, we applied three different ML algorithms on the 3D textural image
outputs of a unique vision-based surface tactile sensor (VS-TS). To collect
realistic textural images of CRC polyps for training the utilized ML
classifiers and evaluating their performance, we first designed and additively
manufactured 48 types of realistic polyp phantoms with different hardness,
type, and textures. Next, the performance of the used three ML algorithms in
classifying the type of fabricated polyps was quantitatively evaluated using
various statistical metrics.Comment: Accepted to IEEE Sensors 2022 Conferenc
Assessing generalisability of deep learning-based polyp detection and segmentation methods through a computer vision challenge
Polyps are well-known cancer precursors identified by colonoscopy. However, variability in their size, appearance, and location makes the detection of polyps challenging. Moreover, colonoscopy surveillance and removal of polyps are highly operator-dependent procedures and occur in a highly complex organ topology. There exists a high missed detection rate and incomplete removal of colonic polyps. To assist in clinical procedures and reduce missed rates, automated methods for detecting and segmenting polyps using machine learning have been achieved in past years. However, the major drawback in most of these methods is their ability to generalise to out-of-sample unseen datasets from different centres, populations, modalities, and acquisition systems. To test this hypothesis rigorously, we, together with expert gastroenterologists, curated a multi-centre and multi-population dataset acquired from six different colonoscopy systems and challenged the computational expert teams to develop robust automated detection and segmentation methods in a crowd-sourcing Endoscopic computer vision challenge. This work put forward rigorous generalisability tests and assesses the usability of devised deep learning methods in dynamic and actual clinical colonoscopy procedures. We analyse the results of four top performing teams for the detection task and five top performing teams for the segmentation task. Our analyses demonstrate that the top-ranking teams concentrated mainly on accuracy over the real-time performance required for clinical applicability. We further dissect the devised methods and provide an experiment-based hypothesis that reveals the need for improved generalisability to tackle diversity present in multi-centre datasets and routine clinical procedures
- β¦