Ptu-024 - photometric stereo reconstruction for surface analysis of mucosal tissue

This paper provides a novel approach for real-time detection of polyps. Using a photometric stereo sensor for endoscopy imaging in a porcine model, the 3D surface geometry of a porcine gut is recovered. Shape features are extracted from the 3D surface data and analysed to detect and identify regions that are locally spherical, suggestive of polyps to aid polyp detection

Multimodal endoscopic system based on multispectral and photometric stereo imaging and analysis

We propose a multimodal endoscopic system based on white light (WL), multispectral (MS), and photometric stereo (PS) imaging for the examination of colorectal cancer (CRC). Recently, the enhancement of the diagnostic accuracy of CRC colonoscopy has been reported; however, tumor diagnosis for a variety of lesion types remains challenging using current endoscopy. In this study, we demonstrate that our developed system can simultaneously discriminate tumor distributions and provide three-dimensional (3D) morphological information about the colon surface using the WL, MS, and PS imaging modalities. The results demonstrate that the proposed system has considerable potential for CRC diagnosis. © 2019, OSA - The Optical Society. All rights reserved.1

Optical Imaging Technology In Colonoscopy - Is There A Role For Photometric Stereo

AbstractColonoscopy screening for the detection and removal of colonic adenomas is central to efforts to reduce the morbidity and mortality of colorectal cancer. However, up to a third of adenomas may be missed at colonoscopy, and the majority of post-colonoscopy colorectal cancers are thought to arise from these. Adenomas have three-dimensional surface topographic features that differentiate them from adjacent normal mucosa. However, these topographic features are not enhanced by white light colonoscopy, and the endoscopist must infer these from two-dimensional cues. This may contribute to the number of missed lesions. A variety of optical imaging technologies have been developed commercially to enhance surface topography. However, existing techniques enhance surface topography indirectly, and in two dimensions, and the evidence does not wholly support their use in routine clinical practice. In this narrative review, co-authored by gastroenterologists and engineers, we summarise the evidence for the impact of established optical imaging technologies on adenoma detection rate, and review the development of photometric stereo (PS) for colonoscopy. PS is a machine vision technique able to capture a dense array of surface normals to render three-dimensional reconstructions of surface topography. This imaging technique has several potential clinical applications in colonoscopy, including adenoma detection, polyp classification, and facilitating polypectomy, an inherently three-dimensional task. However, the development of PS for colonoscopy is at an early stage. We consider the progress that has been made with PS to date and identify the obstacles that need to be overcome prior to clinical application

Integrated multipoint-laser endoscopic airway measurements by transoral approach

Objectives: Optical and technical characteristics usually do not allow objective endoscopic distance measurements. So far no standardized method for endoscopic distance measurement is available. The aim of this study was to evaluate the feasibility and accuracy of transoral airway measurements with a multipoint-laser endoscope. Methods: The semirigid endoscope includes a multipoint laser measurement system that projects 49 laser points (wavelength 639 nm, power < 5mW) into the optical axis of the endoscopic view. Distances, areas, and depths can be measured in real-time. Transoral endoscopic airway measurements were performed on nine human cadavers, which were correlated with CT measurements. Results: The preliminary experiment showed an optimum distance between the endoscope tip and the object of 5 to 6 cm. There was a mean measurement error of 3.26% ± 2.53%. A Spearman correlation coefficient of 0.95

3D Textured Surface Reconstruction from Endoscopic Video

Endoscopy enables high-resolution visualization of tissue texture and is a critical step in many clinical workflows, including diagnosis of infections, tumors or diseases and treatment planning for cancers. This includes my target problems of radiation treatment planning in the nasopharynx and pre-cancerous polyps screening and treatment in colonoscopy. However, an endoscopic video does not provide its information in 3D space, making it difficult to use for tumor localization, and it is inefficient to review. In addition, when there are incomplete camera observations of the organ surface, full surface coverage cannot be guaranteed in an endoscopic procedure, and unsurveyed regions can hardly be noticed in a continuous first-person perspective. This dissertation introduces a new imaging approach that we call endoscopography: an endoscopic video is reconstructed into a full 3D textured surface, which we call an endoscopogram. In this dissertation, I present two endoscopography techniques. One method is a combination of a frame-by-frame algorithmic 3D reconstruction method and a groupwise deformable surface registration method. My contribution is the innovative combination of the two methods that improves the temporal consistency of the frame-by-frame 3D reconstruction algorithm and eliminates the manual intervention that was needed in the deformable surface registration method. The combined method reconstructs an endoscopogram in an offline manner, and the information contained in the tissue texture in the endoscopogram can be transferred to a 3D image such as CT through a surface-to-surface registration. Then, through an interactive tool, the physician can draw directly on the endoscopogram surface to specify a tumor, which then can be automatically transferred to CT slices to aid tumor localization. The second method is a novel deep-learning-driven dense SLAM (simultaneous localization and mapping) system, called RNN-SLAM, that in real time can produce an endoscopogram with display of the unsurveyed regions. In particular, my contribution is the deep learning system in the RNN-SLAM, called RNN-DP. RNN-DP is a novel multi-view dense depth map and odometry estimation method that uses Recurrent Neural Networks (RNN) and trains utilizing multi-view image reprojection and forward-backward flow-consistency losses.Doctor of Philosoph

Appearance Modelling and Reconstruction for Navigation in Minimally Invasive Surgery

Minimally invasive surgery is playing an increasingly important role for patient care. Whilst its direct patient benefit in terms of reduced trauma, improved recovery and shortened hospitalisation has been well established, there is a sustained need for improved training of the existing procedures and the development of new smart instruments to tackle the issue of visualisation, ergonomic control, haptic and tactile feedback. For endoscopic intervention, the small field of view in the presence of a complex anatomy can easily introduce disorientation to the operator as the tortuous access pathway is not always easy to predict and control with standard endoscopes. Effective training through simulation devices, based on either virtual reality or mixed-reality simulators, can help to improve the spatial awareness, consistency and safety of these procedures. This thesis examines the use of endoscopic videos for both simulation and navigation purposes. More specifically, it addresses the challenging problem of how to build high-fidelity subject-specific simulation environments for improved training and skills assessment. Issues related to mesh parameterisation and texture blending are investigated. With the maturity of computer vision in terms of both 3D shape reconstruction and localisation and mapping, vision-based techniques have enjoyed significant interest in recent years for surgical navigation. The thesis also tackles the problem of how to use vision-based techniques for providing a detailed 3D map and dynamically expanded field of view to improve spatial awareness and avoid operator disorientation. The key advantage of this approach is that it does not require additional hardware, and thus introduces minimal interference to the existing surgical workflow. The derived 3D map can be effectively integrated with pre-operative data, allowing both global and local 3D navigation by taking into account tissue structural and appearance changes. Both simulation and laboratory-based experiments are conducted throughout this research to assess the practical value of the method proposed

A Review of Indocyanine Green Fluorescent Imaging in Surgery

The purpose of this paper is to give an overview of the recent surgical intraoperational applications of indocyanine green fluorescence imaging methods, the basics of the technology, and instrumentation used. Well over 200 papers describing this technique in clinical setting are reviewed. In addition to the surgical applications, other recent medical applications of ICG are briefly examined