Automatic Segmentation of Polyps in Colonoscopic Narrow-Band Imaging Data

Colorectal cancer is the third most common type of cancer worldwide. However, this disease can be prevented by detection and removal of precursor adenomatous polyps during optical colonoscopy (OC). During OC, the endoscopist looks for colon polyps. While hyperplastic polyps are benign lesions, adenomatous polyps are likely to become cancerous. Hence, it is a common practice to remove all identified polyps and send them to subsequent histological analysis. But removal of hyperplastic polyps poses unnecessary risk to patients and incurs unnecessary costs for histological analysis. In this paper, we develop the first part of a novel optical biopsy application based on narrow-band imaging (NBI). A barrier to an automatic system is that polyp classification algorithms require manual segmentations of the polyps, so we automatically segment polyps in colonoscopic NBI data. We propose an algorithm, Shape-UCM, which is an extension of the gPb-OWT-UCM algorithm, a state-of-the-art algorithm for boundary detection and segmentation. Shape-UCM solves the intrinsic scale selection problem of gPb-OWT-UCM by including prior knowledge about the shape of the polyps. Shape-UCM outperforms previous methods with a specificity of 92%, a sensitivity of 71%, and an accuracy of 88% for automatic segmentation of a test set of 87 images

From the surface to the single cell: Novel endoscopic approaches in inflammatory bowel disease

Inflammatory bowel diseases (IBD) comprise the two major entities Crohn's disease and ulcerative colitis and endoscopic imaging of the gastrointestinal tract has always been an integral and central part in the management of IBD patients. Within the recent years, mucosal healing emerged as a key treatment goal in IBD that substantially decides about the clinical outcome of IBD patients, thereby demanding for a precise, timely and detailed endoscopic assessment of the mucosal inflammation associated with IBD. Further, molecular imaging has tremendously expanded the clinical utility and applications of modern endoscopy, now encompassing not only diagnosis, surveillance, and treatment but also the prediction of individual therapy response. Within this review we describe novel endoscopic approaches and advanced endoscopic imaging methods for the diagnosis, treatment and surveillance of IBD patients. We begin by providing an overview over novel and advanced imaging techniques such as magnification endoscopy and dye-based and dye-less chromoendoscopy, endomicroscopy and endocytoscopy. We then describe how these techniques can be utilized for the precise and ultrastructural assessment of mucosal inflammation and dysplasia development associated with IBD and outline how they have enabled the endoscopist to gain insight onto the cellular level in real-time. Finally, we provide an outlook on how molecular imaging has rapidly evolved in the recent past and can be used to make individual predictions about the therapeutic response towards biological treatment

Advanced gastrointestinal endoscopic imaging for inflammatory bowel diseases

Gastrointestinal luminal endoscopy is of paramount importance for diagnosis, monitoring and dysplasia surveillance in patients with both, Crohn's disease and ulcerative colitis. Moreover, with the recent recognition that mucosal healing is directly linked to the clinical outcome of patients with inflammatory bowel disorders, a growing demand exists for the precise, timely and detailed endoscopic assessment of superficial mucosal layer. Further, the novel field of molecular imaging has tremendously expanded the clinical utility and applications of modern endoscopy, now encompassing not only diagnosis, surveillance, and treatment but also the prediction of individual therapeutic responses. Within this review, we describe how novel endoscopic approaches and advanced endoscopic imaging methods such as high definition and high magnification endoscopy, dye-based and dye-less chromoendoscopy, confocal laser endomicroscopy, endocytoscopy and molecular imaging now allow for the precise and ultrastructural assessment of mucosal inflammation and describe the potential of these techniques for dysplasia detection

SCALING ARTIFICIAL INTELLIGENCE IN ENDOSCOPY: FROM MODEL DEVELOPMENT TO MACHINE LEARNING OPERATIONS FRAMEWORKS

Questa tesi esplora l'integrazione dell'intelligenza artificiale (IA) in Otorinolaringoiatria – Chirurgia di Testa e Collo, concentrandosi sui progressi della computer vision per l’endoscopia e le procedure chirurgiche. La ricerca inizia con una revisione completa dello stato dell’arte dell'IA e della computer vision in questo campo, identificando aree per ulteriori sviluppi. L'obiettivo principale è stato quello di sviluppare un sistema di computer vision per l'analisi di immagini e video endoscopici. La ricerca ha coinvolto la progettazione di strumenti per la rilevazione e segmentazione di neoplasie nelle vie aerodigestive superiori (VADS) e la valutazione della motilità delle corde vocali, cruciale nella stadiazione del carcinoma laringeo. Inoltre, lo studio si è focalizzato sul potenziale dei foundation vision models, vision transformers basati su self-supervised learning, per ridurre la necessità di annotazione da parte di esperti, approccio particolarmente vantaggioso in campi con dati limitati. Inoltre, la ricerca ha incluso lo sviluppo di un'applicazione web per migliorare e velocizzare il processo di annotazione in endoscopia delle VADS, nell’ambito generale delle tecniche di MLOps. La tesi copre varie fasi della ricerca, a partire dalla definizione del quadro concettuale e della metodologia, denominata "Videomics". Include una revisione della letteratura sull'IA in endoscopia clinica, focalizzata sulla Narrow Band Imaging (NBI) e sulle reti neurali convoluzionali (CNN). Lo studio progredisce attraverso diverse fasi, dalla valutazione della qualità delle immagini endoscopiche alla caratterizzazione approfondita delle lesioni neoplastiche. Si affronta anche la necessità di standard nel reporting degli studi di computer vision in ambito medico e si valuta l'applicazione dell'IA in setting dinamici come la motilità delle corde vocali. Una parte significativa della ricerca indaga l'uso di algoritmi di computer vision generalizzati (“foundation models”) e la “commoditization” degli algoritmi di machine learning, utilizzando polipi nasali e il carcinoma orofaringeo come casi studio. Infine, la tesi discute lo sviluppo di ENDO-CLOUD, un sistema basato su cloud per l’analisi della videolaringoscopia, evidenziando le sfide e le soluzioni nella gestione dei dati e l’utilizzo su larga scala di modelli di IA nell'imaging medico.This thesis explores the integration of artificial intelligence (AI) in Otolaryngology – Head and Neck Surgery, focusing on advancements in computer vision for endoscopy and surgical procedures. It begins with a comprehensive review of AI and computer vision advancements in this field, identifying areas for further exploration. The primary aim was to develop a computer vision system for endoscopy analysis. The research involved designing tools for detecting and segmenting neoplasms in the upper aerodigestive tract (UADT) and assessing vocal fold motility, crucial in laryngeal cancer staging. Further, the study delves into the potential of vision foundation models, like vision transformers trained via self-supervision, to reduce the need for expert annotations, particularly beneficial in fields with limited cases. Additionally, the research includes the development of a web application for enhancing and speeding up the annotation process in UADT endoscopy, under the umbrella of Machine Learning Operations (MLOps). The thesis covers various phases of research, starting with defining the conceptual framework and methodology, termed "Videomics". It includes a literature review on AI in clinical endoscopy, focusing on Narrow Band Imaging (NBI) and convolutional neural networks (CNNs). The research progresses through different stages, from quality assessment of endoscopic images to in-depth characterization of neoplastic lesions. It also addresses the need for standards in medical computer vision study reporting and evaluates the application of AI in dynamic vision scenarios like vocal fold motility. A significant part of the research investigates the use of "general purpose" vision algorithms and the commoditization of machine learning algorithms, using nasal polyps and oropharyngeal cancer as case studies. Finally, the thesis discusses the development of ENDO-CLOUD, a cloud-based system for videolaryngoscopy, highlighting the challenges and solutions in data management and the large-scale deployment of AI models in medical imaging

Advances in the Endoscopic Assessment of Inflammatory Bowel Diseases: Cooperation between Endoscopic and Pathologic Evaluations

Endoscopic assessment has a crucial role in the management of inflammatory bowel disease (IBD). It is particularly useful for the assessment of IBD disease extension, severity, and neoplasia surveillance. Recent advances in endoscopic imaging techniques have been revolutionized over the past decades, progressing from conventional white light endoscopy to novel endoscopic techniques using molecular probes or electronic filter technologies. These new technologies allow for visualization of the mucosa in detail and monitor for inflammation/dysplasia at the cellular or sub-cellular level. These techniques may enable us to alter the IBD surveillance paradigm from four quadrant random biopsy to targeted biopsy and diagnosis. High definition endoscopy and dye-based chromoendoscopy can improve the detection rate of dysplasia and evaluate inflammatory changes with better visualization. Dye-less chromoendoscopy, including narrow band imaging, iScan, and autofluorescence imaging can also enhance surveillance in comparison to white light endoscopy with optical or electronic filter technologies. Moreover, confocal laser endomicroscopy or endocytoscopy have can achieve real-time histology evaluation in vivo and have greater accuracy in comparison with histology. These new technologies could be combined with standard endoscopy or further histologic confirmation in patients with IBD. This review offers an evidence-based overview of new endoscopic techniques in patients with IBD.ope

Using Fluorescence – Polarization Endoscopy in Detection of Precancerous and Cancerous Lesions in Colon and Pancreatic Cancer

Colitis-associated cancer (CAC) arises from premalignant flat lesions of the colon, which are difficult to detect with current endoscopic screening approaches. We have developed a complementary fluorescence and polarization reporting strategy that combines the unique biochemical and physical properties of dysplasia and cancer for real time detection of these lesions. Utilizing a new thermoresponsive sol-gel formulation with targeted molecular probe allowed topical application and detection of precancerous and cancerous lesions during endoscopy. Incorporation of nanowire-filtered polarization imaging into NIR fluorescence endoscopy served as a validation strategy prior to obtaining biopsies. In order to reduce repeat surgeries arising from incomplete tumor resection, we demonstrated the efficacy of the targeted molecular probe towards margins of sporadic colorectal cancer (SCC). Fluorescence-polarization microscopy using circular polarized (CP) light served as a rapid, supplementary tool for assessment and validation of excised tissue to ensure complete tumor resection for examining tumor margins prior to H&E-based pathological diagnosis. We extended our platform towards non-invasive directed detection of pancreatic cancer utilizing fluorescence molecular tomography (FMT) and NIR laparoscopy using identified targeted molecular probe. We were able to non-invasively distinguished between pancreatitis and pancreatic cancer and guide pancreatic tumor resection using NIR laparoscopy

画像強調内視鏡による潰瘍性大腸炎関連腫瘍の観察-自家蛍光イメージングの定量化

久留米大学2015年

Beyond white light:optical enhancement in conjunction with magnification colonoscopy for the assessment of mucosal healing in ulcerative colitis

Abstract Background and study aim The I-SCAN optical enhancement (OE) system with magnification is a recently introduced combination of optical and digital electronic virtual chromoendoscopy, which enhances mucosal and vascular details. The aim of this pilot study was to investigate the use of I-SCAN OE in the assessment of inflammatory changes in ulcerative colitis (UC). Patients and methods A total of 41 consecutive patients with UC and 9 control patients were examined by I-SCAN OE (Pentax Medical, Tokyo, Japan). Targeted biopsies of the imaged areas were obtained. A new optical enhancement score focusing on mucosal and vascular changes was developed. The diagnostic accuracy of I-SCAN OE was calculated against histology using two UC histological scores – Robarts Histopathology Index (RHI) and ECAP (Extent, Chronicity, Activity, Plus additional findings). Results The overall I-SCAN OE score correlated with ECAP (r = 0.70; P &lt; 0.001). The accuracy of the overall I-SCAN OE score to detect abnormalities by ECAP was 80 % (sensitivity 78 %, specificity 100 %). I-SCAN OE vascular and mucosal scores correlated with ECAP (r = 0.65 and 0.71, respectively; P &lt; 0.001). The correlation between overall I-SCAN OE score and RHI was r = 0.61 (P &lt; 0.01), and the accuracy to detect abnormalities by RHI was 68 % (sensitivity 78 %, specificity 50 %). The majority of patients with Mayo 0 had abnormalities on I-SCAN OE. Conclusion In UC, the new I-SCAN OE technology accurately identified mucosal inflammation, and correlated well with histological scores of chronic and acute changes.</jats:p

No more hide and seek:strategies to optimize diagnosis and endoscopic treatment of complex colorectal neoplasms

Large non-pedunculated colorectal polyps and the occurrence of post-colonoscopy colorectal cancer are the subjects of the thesis “No more hide and seek”. The thesis shows that non-pedunculated colorectal cancers consist of a morphologically heterogeneous group (a group with high variety in tissue characteristics), with a different risk of cancer cells at diagnosis for each subtype. Accurate prior assessment of this risk could prevent suboptimal treatment. Furthermore, this thesis shows that patients with large flat polyps develop more polyps in the future than patients with other polyp types. Flat polyps are harder to detect than other polyps. It is suggested that they can be more easily missed during colonoscopy with the risk of malignant transformation afterwards, which should be prevented by colonoscopy. This could result in the occurrence of so-called post-colonoscopy colorectal cancer. Faster growth by different mutations was hypothesized as another factor in post-colonoscopy colorectal cancer occurrence. This thesis examined the genetic profile of post-colonoscopy colorectal cancers in comparison with common colorectal cancer. The results showed no unique mutations in post-colonoscopy colorectal cancers, but they did show more often the features as seen in a more subtle and flat category of flat bowel polyps. Improvements in detection of these polyps remain important in post-colonoscopy colorectal cancer prevention