Unsupervised Odometry and Depth Learning for Endoscopic Capsule Robots

In the last decade, many medical companies and research groups have tried to convert passive capsule endoscopes as an emerging and minimally invasive diagnostic technology into actively steerable endoscopic capsule robots which will provide more intuitive disease detection, targeted drug delivery and biopsy-like operations in the gastrointestinal(GI) tract. In this study, we introduce a fully unsupervised, real-time odometry and depth learner for monocular endoscopic capsule robots. We establish the supervision by warping view sequences and assigning the re-projection minimization to the loss function, which we adopt in multi-view pose estimation and single-view depth estimation network. Detailed quantitative and qualitative analyses of the proposed framework performed on non-rigidly deformable ex-vivo porcine stomach datasets proves the effectiveness of the method in terms of motion estimation and depth recovery.Comment: submitted to IROS 201

Multiscale texture descriptors for automatic small bowel tumors detection in capsule endoscopy

Conventional endoscopic exams do not allow the entire visualization of the gastrointestinal (GI) tract. Push enteroscopy (PE) is an effective diagnostic and therapeutic procedure, although it only allows exploration of the proximal small bowel (Pennazio et al., 1995). Simultaneously, convetional colonoscopy is limited at the terminal ileum. Therefore, prior to the wireless capsule endoscopy era, the small intestine was the conventional endoscopy’s last frontier, because it could not be internally visualized directly or in it’s entirely by any method (Herrerías & Mascarenhas-Saraiva, 2007). The small intestine accounts for 75% of the total length and 90% of the surface area of the gastrointestinal tract. In adults it measures about 570 cm at post mortem, which is substantially longer than conventional video endoscopes (100-180 cm) (Swain & Fritscher-Ravens, 2004). Intraoperative enteroscopy is the most complete but also the most invasive means of examining the small bowel (Gay et al., 1998). Given the technical and medical improvements introduced on the assessment of the gastrointestinal (GI) tract, Capsule Endoscopy (CE) is considered as the first major technological innovation in GI diagnostic medicine since the flexible endoscope (Kaffes, 2009). More recently, a new technique, the double-balloon enteroscopy (DBE), has been introduced into clinical practice (Yamamoto & Kita, 2006). DBE has the potential to examine the entire length of the small bowel with biopsy and therapeutic capability. Nevertheless, it is a time consuming procedure that requires specialist training for the operating physician. We should note that DBE and CE are complementary tools and not competitive (Chen et al., 2007). Hence, the diagnostic ease of CE can be complemented with a targeted and often therapeutic DBE (Kaffes, 2009). Therefore, CE can be used as a first line diagnosis method, while DBE can be used as a confirmatory or therapeutic modality for lesions first visualized by CE (Pennazio, 2006). The endoscopic capsule is a pill-like device, with only 11mm x 26 mm, and includes a miniaturized camera, a light source and circuits for the acquisition and wireless transmission of signals (Iddan et al., 2000). As the capsule moves through GI tract, propelled exclusively by peristalsis, it acquires images at a rate of two per second and sends them to a hard disk receiver that is worn in the belt of the patient, in a wireless communication scheme. The acquisition of images is limited by the battery life of the device, usually around eight hours, which imply that in a single CE exam more than 50000 images are acquired. If no complications arise, the capsule should be in the patient’s stool, usually within 24-48 h, and not reused (Pennazio, 2006). Capsule endoscopy has evolved in a few short years to become a first-line, noninvasive diagnostic technique for the small bowel. CE is now being utilized worldwide to assess patients for obscure gastrointestinal bleeding, possible Crohn’s disease, celiac disease and small bowel tumors (Lee & Eisen, 2010). It is now available in over 4500 practice sites around the world (Munoz-Navas, 2009). The time required to a physician to analyze the resulting video is, on average, 40-60 min (Pennazio, 2006). The reading time and interpretation of CE exams is very time consuming given that more than 50,000 images have to be reviewed (Delvaux & Gay, 2006; Mergener et al., 2007), which contributes to the high cost of a CE exam (Westerhof et al., 2009). Thus, a computer assisted diagnosis tool to help the physicians to evaluate CE exams faster and more accurately is an important technical challenge and an excellent economical opportunity.Centre Algoritm

Multidimensional Optical Sensing and Imaging Systems (MOSIS): From Macro to Micro Scales

Multidimensional optical imaging systems for information processing and visualization technologies have numerous applications in fields such as manufacturing, medical sciences, entertainment, robotics, surveillance, and defense. Among different three-dimensional (3-D) imaging methods, integral imaging is a promising multiperspective sensing and display technique. Compared with other 3-D imaging techniques, integral imaging can capture a scene using an incoherent light source and generate real 3-D images for observation without any special viewing devices. This review paper describes passive multidimensional imaging systems combined with different integral imaging configurations. One example is the integral-imaging-based multidimensional optical sensing and imaging systems (MOSIS), which can be used for 3-D visualization, seeing through obscurations, material inspection, and object recognition from microscales to long range imaging. This system utilizes many degrees of freedom such as time and space multiplexing, depth information, polarimetric, temporal, photon flux and multispectral information based on integral imaging to record and reconstruct the multidimensionally integrated scene. Image fusion may be used to integrate the multidimensional images obtained by polarimetric sensors, multispectral cameras, and various multiplexing techniques. The multidimensional images contain substantially more information compared with two-dimensional (2-D) images or conventional 3-D images. In addition, we present recent progress and applications of 3-D integral imaging including human gesture recognition in the time domain, depth estimation, mid-wave-infrared photon counting, 3-D polarimetric imaging for object shape and material identification, dynamic integral imaging implemented with liquid-crystal devices, and 3-D endoscopy for healthcare applications.B. Javidi wishes to acknowledge support by the National Science Foundation (NSF) under Grant NSF/IIS-1422179, and DARPA and US Army under contract number W911NF-13-1-0485. The work of P. Latorre Carmona, A. Martínez-Uso, J. M. Sotoca and F. Pla was supported by the Spanish Ministry of Economy under the project ESP2013-48458-C4-3-P, and by MICINN under the project MTM2013-48371-C2-2-PDGI, by Generalitat Valenciana under the project PROMETEO-II/2014/062, and by Universitat Jaume I through project P11B2014-09. The work of M. Martínez-Corral and G. Saavedra was supported by the Spanish Ministry of Economy and Competitiveness under the grant DPI2015-66458-C2-1R, and by the Generalitat Valenciana, Spain under the project PROMETEOII/2014/072

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

Standardization and implementation of fluorescence molecular endoscopy in the clinic

Significance: Near-infrared fluorescence molecular endoscopy (NIR-FME) is an innovative technique allowing for in vivo visualization of molecular processes in hollow organs. Despite its potential for clinical translation, NIR-FME still faces challenges, for example, the lack of consensus in performing quality control and standardization of procedures and systems. This may hamper the clinical approval of the technology by authorities and its acceptance by endoscopists. Until now, several clinical trials using NIR-FME have been performed. However, most of these trials had different study designs, making comparison difficult. Aim: We describe the need for standardization in NIR-FME, provide a pathway for setting up a standardized clinical study, and describe future perspectives for NIR-FME. Body: Standardization is challenging due to many parameters. Invariable parameters refer to the hardware specifications. Variable parameters refer to movement or tissue optical properties. Phantoms can be of aid when defining the influence of these variables or when standardizing a procedure. Conclusion: There is a need for standardization in NIR-FME and hurdles still need to be overcome before a widespread clinical implementation of NIR-FME can be realized. When these hurdles are overcome, clinical outcomes can be compared and systems can be benchmarked, enabling clinical implementation

Automatic Small Bowel Tumor Diagnosis by Using Multi-Scale Wavelet-Based Analysis in Wireless Capsule Endoscopy Images

BACKGROUND: Wireless capsule endoscopy has been introduced as an innovative, non-invasive diagnostic technique for evaluation of the gastrointestinal tract, reaching places where conventional endoscopy is unable to. However, the output of this technique is an 8 hours video, whose analysis by the expert physician is very time consuming. Thus, a computer assisted diagnosis tool to help the physicians to evaluate CE exams faster and more accurately is an important technical challenge and an excellent economical opportunity. METHOD: The set of features proposed in this paper to code textural information is based on statistical modeling of second order textural measures extracted from co-occurrence matrices. To cope with both joint and marginal non-Gaussianity of second order textural measures, higher order moments are used. These statistical moments are taken from the two-dimensional color-scale feature space, where two different scales are considered. Second and higher order moments of textural measures are computed from the co-occurrence matrices computed from images synthesized by the inverse wavelet transform of the wavelet transform containing only the selected scales for the three color channels. The dimensionality of the data is reduced by using Principal Component Analysis. RESULTS: The proposed textural features are then used as the input of a classifier based on artificial neural networks. Classification performances of 93.1% specificity and 93.9% sensitivity are achieved on real data. These promising results open the path towards a deeper study regarding the applicability of this algorithm in computer aided diagnosis systems to assist physicians in their clinical practice