Unsupervised Odometry and Depth Learning for Endoscopic Capsule Robots

In the last decade, many medical companies and research groups have tried to convert passive capsule endoscopes as an emerging and minimally invasive diagnostic technology into actively steerable endoscopic capsule robots which will provide more intuitive disease detection, targeted drug delivery and biopsy-like operations in the gastrointestinal(GI) tract. In this study, we introduce a fully unsupervised, real-time odometry and depth learner for monocular endoscopic capsule robots. We establish the supervision by warping view sequences and assigning the re-projection minimization to the loss function, which we adopt in multi-view pose estimation and single-view depth estimation network. Detailed quantitative and qualitative analyses of the proposed framework performed on non-rigidly deformable ex-vivo porcine stomach datasets proves the effectiveness of the method in terms of motion estimation and depth recovery.Comment: submitted to IROS 201

A magnetic internal mechanism for precise orientation of the camera in wireless endoluminal applications

Background and study aims: The use of magnetic fields to control operative devices has been recently described in endoluminal and transluminal surgical applications. The exponential decrease of magnetic field strength with distance has major implications for precision of the remote control. We aimed to assess the feasibility and functionality of a novel wireless miniaturized mechanism, based on magnetic forces, for precise orientation of the camera. Materials and methods: A remotely controllable endoscopic capsule was developed as proof of concept. Two intracapsular moveable permanent magnets allow fine positioning, and an externally applied magnetic field permits gross movement and stabilization. Performance was assessed in ex vivo and in vivo bench tests, using porcine upper and lower gastrointestinal tracts. Results: Fine control of capsule navigation and rotation was achieved in all tests with an external magnet held steadily about 15 cm from the capsule. The camera could be rotated in steps of 1.8°. This was confirmed by ex vivo tests; the mechanism could adjust the capsule view at 40 different locations in a gastrointestinal tract phantom model. Full 360° viewing was possible in the gastric cavity, while the maximal steering in the colonwas 45° in total. In vivo, a similar performance was verified, where the mechanism was successfully operated every 5 cm for 40 cm in the colon, visually sweeping from side to side of the lumen; 360° views were obtained in the gastric fundus and body, while antrally the luminal walls prevented full rotation. Conclusions: We report the feasibility and effectiveness of the combined use of external static magnetic fields and internal actuation to move small permanent intracapsular magnets to achieve wirelessly controllable and precise camera steering. The concept is applicable to capsule endoscopy as to other instrumentation for laparoscopic, endoluminal, or transluminal procedures

Seeing the Big Picture: System Architecture Trends in Endoscopy and LED-Based hyperspectral Subsystem Intergration

Early-stage colorectal lesions remain difficult to detect. Early development of neoplasia tends to be small (less than 10 mm) and flat and difficult to distinguish from surrounding mucosa. Additionally, optical diagnosis of neoplasia as benign or malignant is problematic. Low rates of detection of these lesions allow for continued growth in the colorectum and increased risk of cancer formation. Therefore, it is crucial to detect neoplasia and other non-neoplastic lesions to determine risk and guide future treatment. Technology for detection needs to enhance contrast of subtle tissue differences in the colorectum and track multiple biomarkers simultaneously. This work implements one such technology with the potential to achieve the desired multi-contrast outcome for endoscopic screenings: hyperspectral imaging. Traditional endoscopic imaging uses a white light source and a RGB detector to visualize the colorectum using reflected light. Hyperspectral imaging (HSI) acquires an image over a range of individual wavelength bands to create an image hypercube with a wavelength dimension much deeper and more sensitive than that of an RGB image. A hypercube can consist of reflectance or fluorescence (or both) spectra depending on the filtering optics involved. Prior studies using HSI in endoscopy have normally involved ex vivo tissues or xiv optics that created a trade-off between spatial resolution, spectral discrimination and temporal sampling. This dissertation describes the systems design of an alternative HSI endoscopic imaging technology that can provide high spatial resolution, high spectral distinction and video-rate acquisition in vivo. The hyperspectral endoscopic system consists of a novel spectral illumination source for image acquisition dependent on the fluorescence excitation (instead of emission). Therefore, this work represents a novel contribution to the field of endoscopy in combining excitation-scanning hyperspectral imaging and endoscopy. This dissertation describes: 1) systems architecture of the endoscopic system in review of previous iterations and theoretical next-generation options, 2) feasibility testing of a LED-based hyperspectral endoscope system and 3) another LED-based spectral illuminator on a microscope platform to test multi-spectral contrast imaging. The results of the architecture point towards an endoscopic system with more complex imaging and increased computational capabilities. The hyperspectral endoscope platform proved feasibility of a LED-based spectral light source with a multi-furcated solid light guide. Another LED-based design was tested successfully on a microscope platform with a dual mirror array similar to telescope designs. Both feasibility tests emphasized optimization of coupling optics and combining multiple diffuse light sources to a common output. These results should lead to enhanced imagery for endoscopic tissue discrimination and future optical diagnosis for routine colonoscopy

A Non-Rigid Map Fusion-Based RGB-Depth SLAM Method for Endoscopic Capsule Robots

In the gastrointestinal (GI) tract endoscopy field, ingestible wireless capsule endoscopy is considered as a minimally invasive novel diagnostic technology to inspect the entire GI tract and to diagnose various diseases and pathologies. Since the development of this technology, medical device companies and many groups have made significant progress to turn such passive capsule endoscopes into robotic active capsule endoscopes to achieve almost all functions of current active flexible endoscopes. However, the use of robotic capsule endoscopy still has some challenges. One such challenge is the precise localization of such active devices in 3D world, which is essential for a precise three-dimensional (3D) mapping of the inner organ. A reliable 3D map of the explored inner organ could assist the doctors to make more intuitive and correct diagnosis. In this paper, we propose to our knowledge for the first time in literature a visual simultaneous localization and mapping (SLAM) method specifically developed for endoscopic capsule robots. The proposed RGB-Depth SLAM method is capable of capturing comprehensive dense globally consistent surfel-based maps of the inner organs explored by an endoscopic capsule robot in real time. This is achieved by using dense frame-to-model camera tracking and windowed surfelbased fusion coupled with frequent model refinement through non-rigid surface deformations

Capsule endoscopy system with novel imaging algorithms

Wireless capsule endoscopy (WCE) is a state-of-the-art technology to receive images of human intestine for medical diagnostics. In WCE, the patient ingests a specially designed electronic capsule which has imaging and wireless transmission capabilities inside it. While the capsule travels through the gastrointestinal (GI) tract, it captures images and sends them wirelessly to an outside data logger unit. The data logger stores the image data and then they are transferred to a personal computer (PC) where the images are reconstructed and displayed for diagnosis. The key design challenge in WCE is to reduce the area and power consumption of the capsule while maintaining acceptable image reconstruction. In this research, the unique properties of WCE images are identified by analyzing hundreds of endoscopic images and video frames, and then these properties are used to develop novel and low complexity compression algorithms tailored for capsule endoscopy. The proposed image compressor consists of a new YEF color space converter, lossless prediction coder, customizable chrominance sub-sampler and an efficient Golomb-Rice encoder. The scheme has both lossy and lossless modes and is further customized to work with two lighting modes – conventional white light imaging (WLI) and emerging narrow band imaging (NBI). The average compression ratio achieved using the proposed lossy compression algorithm is 80.4% for WBI and 79.2% for NBI with high reconstruction quality index for both bands. Two surveys have been conducted which show that the reconstructed images have high acceptability among medical imaging doctors and gastroenterologists. The imaging algorithms have been realized in hardware description language (HDL) and their functionalities have been verified in field programmable gate array (FPGA) board. Later it was implemented in a 0.18 μm complementary metal oxide semiconductor (CMOS) technology and the chip was fabricated. Due to the low complexity of the core compressor, it consumes only 43 µW of power and 0.032 mm2 of area. The compressor is designed to work with commercial low-power image sensor that outputs image pixels in raster scan fashion, eliminating the need of significant input buffer memory. To demonstrate the advantage, a prototype of the complete WCE system including an FPGA based electronic capsule, a microcontroller based data logger unit and a Windows based image reconstruction software have been developed. The capsule contains the proposed low complexity image compressor and can generate both lossy and lossless compressed bit-stream. The capsule prototype also supports both white light imaging (WLI) and narrow band imaging (NBI) imaging modes and communicates with the data logger in full duplex fashion, which enables configuring the image size and imaging mode in real time during the examination. The developed data logger is portable and has a high data rate wireless connectivity including Bluetooth, graphical display for real time image viewing with state-of-the-art touch screen technology. The data are logged in micro SD cards and can be transferred to PC or Smartphone using card reader, USB interface, or Bluetooth wireless link. The workstation software can decompress and show the reconstructed images. The images can be navigated, marked, zoomed and can be played as video. Finally, ex-vivo testing of the WCE system has been done in pig's intestine to validate its performance

Technology of swallowable capsule for medical applications

Medical technology has undergone major breakthroughs in recent years, especially in the area of the examination tools for diagnostic purposes. This paper reviews the swallowable capsule technology in the examination of the gastrointestinal system for various diseases. The wireless camera pill has created a more advanced method than many traditional examination methods for the diagnosis of gastrointestinal diseases such as gastroscopy by the use of an endoscope. After years of great innovation, commercial swallowable pills have been produced and applied in clinical practice. These smart pills can cover the examination of the gastrointestinal system and not only provide to the physicians a lot more useful data that is not available from the traditional methods, but also eliminates the use of the painful endoscopy procedure. In this paper, the key state-of-the-art technologies in the existing Wireless Capsule Endoscopy (WCE) systems are fully reported and the recent research progresses related to these technologies are reviewed. The paper ends by further discussion on the current technical bottlenecks and future research in this area

Frontiers of robotic endoscopic capsules: a review

Digestive diseases are a major burden for society and healthcare systems, and with an aging population, the importance of their effective management will become critical. Healthcare systems worldwide already struggle to insure quality and affordability of healthcare delivery and this will be a significant challenge in the midterm future. Wireless capsule endoscopy (WCE), introduced in 2000 by Given Imaging Ltd., is an example of disruptive technology and represents an attractive alternative to traditional diagnostic techniques. WCE overcomes conventional endoscopy enabling inspection of the digestive system without discomfort or the need for sedation. Thus, it has the advantage of encouraging patients to undergo gastrointestinal (GI) tract examinations and of facilitating mass screening programmes. With the integration of further capabilities based on microrobotics, e.g. active locomotion and embedded therapeutic modules, WCE could become the key-technology for GI diagnosis and treatment. This review presents a research update on WCE and describes the state-of-the-art of current endoscopic devices with a focus on research-oriented robotic capsule endoscopes enabled by microsystem technologies. The article also presents a visionary perspective on WCE potential for screening, diagnostic and therapeutic endoscopic procedures