Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

A new audio-visual analysis approach and tools for parsing colonoscopy videos

Colonoscopy is an important screening tool for colorectal cancer. During a colonoscopic procedure, a tiny video camera at the tip of the endoscope generates a video signal of the internal mucosa of the colon. The video data are displayed on a monitor for real-time analysis by the endoscopist. We call videos captured from colonoscopic procedures colonoscopy videos. Because these videos possess unique characteristics, new types of semantic units and parsing techniques are required. In this paper, we introduce a new analysis approach that includes (a) a new definition of semantic unit - scene (a segment of visual and audio data that correspond to an endoscopic segment of the colon); (b) a novel scene segmentation algorithm using audio and visual analysis to recognize scene boundaries. We design a prototype system to implement the proposed approach. This system also provides the tools for video/image browsing. The tools enable the users to quickly locate and browse scenes of interest. Experiments on real colonoscopy videos show the effectiveness of our algorithms. The proposed techniques and software are useful (1) for post-procedure reviews, (2) for developing an effective content-based retrieval system for colonoscopy videos to facilitate endoscopic research and education, and (3) for development of a systematic approach to assess endoscopists\u27 procedural skills

Automated quantification and classification of human kidney microstructures obtained by optical coherence tomography

Optical coherence tomography (OCT) is a rapidly emerging imaging modality that can non-invasively provide cross-sectional, high-resolution images of tissue morphology such as kidney in situ and in real-time. Because the viability of a donor kidney is closely correlated with its tubular morphology, and a large amount of image datasets are expected when using OCT to scan the entire kidney, it is necessary to develop automated image analysis methods to quantify the spatially-resolved morphometric parameters such as tubular diameter, and to classify various microstructures. In this study, we imaged the human kidney in vitro, quantified the diameters of hollow structures such as blood vessels and uriniferous tubules, and classified those structures automatically. The quantification accuracy was validated. This work can enable studies to determine the clinical utility of OCT for kidney imaging, as well as studies to evaluate kidney morphology as a biomarker for assessing kidney's viability prior to transplantation

Mobility, Navigation and Localization Towards Robotic Endoscopy

With significant progress being made towards improving endoscope technology such as capsule endoscopy and robotic endoscopy, the development of advanced strategies for manipulating, controlling and more generally, easing the accessibility of these devices for physicians is an important next step. This work presents the development of several robotic platforms for experimentally testing navigation and localization strategies in robotic endoscopy followed by the development and testing of navigation strategies using these devices. Finally, visual and visual inertial localization and mapping is explored on two of these robotic systems. We first present a detailed description on the state-of-the-art with regard to minimally invasive robotic surgery and then follow this with in-depth description of our design and validation of two important systems, the Robotic Endoscope Platform (REP) and the Modular Endoscopy Simulation Apparatus (MESA), for exploring some of the challenges in robotic endoscopy. Following these descriptions we present a technique for autonomous navigation of the REP within the MESA as well as an attempt at applying Simultaneous Localization and Mapping (SLAM) to allow for the real-time localization of this system. Finally, we transition these techniques to the Endoculus, a complete robotic endoscope suitable for in vivo testing, and demonstrate both autonomous navigation for this device, and the implementation of three different SLAM systems for localization and mapping of the Endoculus system in real-time. Throughout these experiments we demonstrate the potential for advanced methods in computer vision along with other sensory techniques to substantially benefit endoscopy, enabling greater and greater autonomy of these systems and furthering the case for robotic endoscopy as a whole.</p

A Robust and Fast System for CTC Computer-Aided Detection of Colorectal Lesions

We present a complete, end-to-end computer-aided detection (CAD) system for identifying lesions in the colon, imaged with computed tomography (CT). This system includes facilities for colon segmentation, candidate generation, feature analysis, and classification. The algorithms have been designed to offer robust performance to variation in image data and patient preparation. By utilizing efficient 2D and 3D processing, software optimizations, multi-threading, feature selection, and an optimized cascade classifier, the CAD system quickly determines a set of detection marks. The colon CAD system has been validated on the largest set of data to date, and demonstrates excellent performance, in terms of its high sensitivity, low false positive rate, and computational efficiency

New Techniques in Gastrointestinal Endoscopy

As result of progress, endoscopy has became more complex, using more sophisticated devices and has claimed a special form. In this moment, the gastroenterologist performing endoscopy has to be an expert in macroscopic view of the lesions in the gut, with good skills for using standard endoscopes, with good experience in ultrasound (for performing endoscopic ultrasound), with pathology experience for confocal examination. It is compulsory to get experience and to have patience and attention for the follow-up of thousands of images transmitted during capsule endoscopy or to have knowledge in physics necessary for autofluorescence imaging endoscopy. Therefore, the idea of an endoscopist has changed. Examinations mentioned need a special formation, a superior level of instruction, accessible to those who have already gained enough experience in basic diagnostic endoscopy. This is the reason for what these new issues of endoscopy are presented in this book of New techniques in Gastrointestinal Endoscopy

Cardiovascular computed tomography in cardiovascular disease: An overview of its applications from diagnosis to prediction

Cardiovascular computed tomography angiography (CTA) is a widely used imaging modality in the diagnosis of cardiovascular disease. Advancements in CT imaging technology have further advanced its applications from high diagnostic value to minimising radiation exposure to patients. In addition to the standard application of assessing vascular lumen changes, CTA-derived applications including 3D printed personalised models, 3D visualisations such as virtual endoscopy, virtual reality, augmented reality and mixed reality, as well as CT-derived hemodynamic flow analysis and fractional flow reserve (FFRCT) greatly enhance the diagnostic performance of CTA in cardiovascular disease. The widespread application of artificial intelligence in medicine also significantly contributes to the clinical value of CTA in cardiovascular disease. Clinical value of CTA has extended from the initial diagnosis to identification of vulnerable lesions, and prediction of disease extent, hence improving patient care and management. In this review article, as an active researcher in cardiovascular imaging for more than 20 years, I will provide an overview of cardiovascular CTA in cardiovascular disease. It is expected that this review will provide readers with an update of CTA applications, from the initial lumen assessment to recent developments utilising latest novel imaging and visualisation technologies. It will serve as a useful resource for researchers and clinicians to judiciously use the cardiovascular CT in clinical practice

Detection of Intestinal Bleeding in Wireless Capsule Endoscopy using Machine Learning Techniques

Gastrointestinal (GI) bleeding is very common in humans, which may lead to fatal consequences. GI bleeding can usually be identified using a flexible wired endoscope. In 2001, a newer diagnostic tool, wireless capsule endoscopy (WCE) was introduced. It is a swallow-able capsule-shaped device with a camera that captures thousands of color images and wirelessly sends those back to a data recorder. After that, the physicians analyze those images in order to identify any GI abnormalities. But it takes a longer screening time which may increase the danger of the patients in emergency cases. It is therefore necessary to use a real-time detection tool to identify bleeding in the GI tract. Each material has its own spectral ‘signature’ which shows distinct characteristics in specific wavelength of light [33]. Therefore, by evaluating the optical characteristics, the presence of blood can be detected. In the study, three main hardware designs were presented: one using a two-wavelength based optical sensor and others using two six-wavelength based spectral sensors with AS7262 and AS7263 chips respectively to determine the optical characteristics of the blood and non-blood samples. The goal of the research is to develop a machine learning model to differentiate blood samples (BS) and non-blood samples (NBS) by exploring their optical properties. In this experiment, 10 levels of crystallized bovine hemoglobin solutions were used as BS and 5 food colors (red, yellow, orange, tan and pink) with different concentrations totaling 25 non-blood samples were used as NBS. These blood and non-blood samples were also combined with pig’s intestine to mimic in-vivo experimental environment. The collected samples were completely separated into training and testing data. Different spectral features are analyzed to obtain the optical information about the samples. Based on the performance on the selected most significant features of the spectral wavelengths, k-nearest neighbors algorithm (k-NN) is finally chosen for the automated bleeding detection. The proposed k-NN classifier model has been able to distinguish the BS and NBS with an accuracy of 91.54% using two wavelengths features and around 89% using three combined wavelengths features in the visible and near-infrared spectral regions. The research also indicates that it is possible to deploy tiny optical detectors to detect GI bleeding in a WCE system which could eliminate the need of time-consuming image post-processing steps