Adaptive Wireless Biomedical Capsule Localization and Tracking

Wireless capsule endoscopy systems have been shown as a gold step to develop future wireless biomedical multitask robotic capsules, which will be utilized in micro surgery, drug delivery, biopsy and multitasks of the endoscopy. In such wireless capsule endoscopy systems, one of the most challenging problems is accurate localization and tracking of the capsule inside the human body. In this thesis, we focus on robotic biomedical capsule localization and tracking using range measurements via electromagetic wave and magnetic strength based sensors. First, a literature review of existing localization techniques with their merits and limitations is presented. Then, a novel geometric environmental coefficient estimation technique is introduced for time of flight (TOF) and received signal strength (RSS) based range measurement. Utilizing the proposed environmental coefficient estimation technique, a 3D wireless biomedical capsule localization and tracking scheme is designed based on a discrete adaptive recursive least square algorithm with forgetting factor. The comparison between localization with novel coefficient estimation technique and localization with known coefficient is provided to demonstrate the proposed technique’s efficiency. Later, as an alternative to TOF and RSS based sensors, use of magnetic strength based sensors is considered. We analyze and demonstrate the performance of the proposed techniques and designs in various scenarios simulated in Matlab/Simulink environment

Learning monocular visual odometry with dense 3D mapping from dense 3D flow

This paper introduces a fully deep learning approach to monocular SLAM, which can perform simultaneous localization using a neural network for learning visual odometry (L-VO) and dense 3D mapping. Dense 2D flow and a depth image are generated from monocular images by sub-networks, which are then used by a 3D flow associated layer in the L-VO network to generate dense 3D flow. Given this 3D flow, the dual-stream L-VO network can then predict the 6DOF relative pose and furthermore reconstruct the vehicle trajectory. In order to learn the correlation between motion directions, the Bivariate Gaussian modelling is employed in the loss function. The L-VO network achieves an overall performance of 2.68% for average translational error and 0.0143 deg/m for average rotational error on the KITTI odometry benchmark. Moreover, the learned depth is fully leveraged to generate a dense 3D map. As a result, an entire visual SLAM system, that is, learning monocular odometry combined with dense 3D mapping, is achieved.Comment: International Conference on Intelligent Robots and Systems(IROS 2018

Seeing the Big Picture: System Architecture Trends in Endoscopy and LED-Based hyperspectral Subsystem Intergration

Early-stage colorectal lesions remain difficult to detect. Early development of neoplasia tends to be small (less than 10 mm) and flat and difficult to distinguish from surrounding mucosa. Additionally, optical diagnosis of neoplasia as benign or malignant is problematic. Low rates of detection of these lesions allow for continued growth in the colorectum and increased risk of cancer formation. Therefore, it is crucial to detect neoplasia and other non-neoplastic lesions to determine risk and guide future treatment. Technology for detection needs to enhance contrast of subtle tissue differences in the colorectum and track multiple biomarkers simultaneously. This work implements one such technology with the potential to achieve the desired multi-contrast outcome for endoscopic screenings: hyperspectral imaging. Traditional endoscopic imaging uses a white light source and a RGB detector to visualize the colorectum using reflected light. Hyperspectral imaging (HSI) acquires an image over a range of individual wavelength bands to create an image hypercube with a wavelength dimension much deeper and more sensitive than that of an RGB image. A hypercube can consist of reflectance or fluorescence (or both) spectra depending on the filtering optics involved. Prior studies using HSI in endoscopy have normally involved ex vivo tissues or xiv optics that created a trade-off between spatial resolution, spectral discrimination and temporal sampling. This dissertation describes the systems design of an alternative HSI endoscopic imaging technology that can provide high spatial resolution, high spectral distinction and video-rate acquisition in vivo. The hyperspectral endoscopic system consists of a novel spectral illumination source for image acquisition dependent on the fluorescence excitation (instead of emission). Therefore, this work represents a novel contribution to the field of endoscopy in combining excitation-scanning hyperspectral imaging and endoscopy. This dissertation describes: 1) systems architecture of the endoscopic system in review of previous iterations and theoretical next-generation options, 2) feasibility testing of a LED-based hyperspectral endoscope system and 3) another LED-based spectral illuminator on a microscope platform to test multi-spectral contrast imaging. The results of the architecture point towards an endoscopic system with more complex imaging and increased computational capabilities. The hyperspectral endoscope platform proved feasibility of a LED-based spectral light source with a multi-furcated solid light guide. Another LED-based design was tested successfully on a microscope platform with a dual mirror array similar to telescope designs. Both feasibility tests emphasized optimization of coupling optics and combining multiple diffuse light sources to a common output. These results should lead to enhanced imagery for endoscopic tissue discrimination and future optical diagnosis for routine colonoscopy

Doctor of Philosophy

dissertationClosed-loop control of wireless capsule endoscopes is an active area of research because it would drastically improve screening of the gastrointestinal tract. Traditional endoscopic procedures are unable to view the entire gastrointestinal tract and current commercial wireless capsule endoscopes are limited in their effectiveness due to their passive nature. This dissertation advances the field of active capsule endoscopy by developing methods to localize the full six-degree-of-freedom (6-DOF) pose of a screw-type magnetic capsule while it is being propelled through a lumen (such as the small intestines) using an external rotating magnetic dipole. The same external magnetic dipole is utilized for both propulsion and localization. Hardware was designed and constructed to enable testing of the magnetic localization and propulsion methods, including a robotic end-effector used as the external actuator magnet, and a prototype capsule embedded with Hall-effect sensors. Due to the use of a rotating magnetic field for propulsion, at any given time, the capsule can be in one of three regimes: synchronously rotating with the applied field, in "step-out" where it is free to move but the external field is rotating too quickly for the capsule to remain synchronously rotating, or completely stationary. We show that it is only necessary to distinguish whether or not the capsule is synchronously rotating (i.e., a single localization method can be used for a capsule in either the step-out or stationary regimes). Two magnetic localization methods are developed. The first uses nonlinear least squares to estimate the capsule's pose when it has no (or approximately no) net motion (e.g., to find the initial capsule pose or when it is stuck in an intestinal fold). The second method estimates the 6-DOF capsule pose as it synchronously rotates with the applied magnetic field using a square-root variant of the Unscented Kalman filter. A simple process model is adopted that restricts the capsule's movement to translation along and rotation about its principle axis. The capsule is actively propelled forward or backward, but it is not actively steered, rather, steering is provided by the lumen. The propulsion parameters that transform magnetic force and torque to the capsule's spatial velocity and angular velocity are estimated with an additional square-root Unscented Kalman filter to enable the capsule to navigate heterogeneous environments such as the small intestines. An optimized localization-propulsion system is described using the two localization algorithms and prior work in screw-type magnetic capsule propulsion with a single rotating dipole field. The capsule's regime is determined and the corresponding localization method is employed. Based on the capsule's estimated pose and the current estimates of its propulsion parameters, the actuator magnet's pose relative to the capsule is optimized to maximize the capsule's forward propulsion. Using this system, our prototype magnetic capsule successfully completed U-shaped and S-shaped trajectories in fresh bovine intestines with an average forward velocity of 5.5mm/s and 3.5 mm/s, respectively. At this rate it would take approximately 18-30 minutes to traverse the 6 meters of a typical human small intestine

Learning Monocular Visual Odometry with Dense 3D Mapping from Dense 3D Flow

This paper introduces a fully deep learning approach to monocular SLAM, which can perform simultaneous localization using a neural network for learning visual odometry (L-VO) and dense 3D mapping. Dense 2D flow and a depth image are generated from monocular images by sub-networks, which are then used by a 3D flow associated layer in the L-VO network to generate dense 3D flow. Given this 3D flow, the dual-stream L-VO network can then predict the 6DOF relative pose and furthermore reconstruct the vehicle trajectory. In order to learn the correlation between motion directions, the Bivariate Gaussian modeling is employed in the loss function. The L-VO network achieves an overall performance of 2.68 % for average translational error and 0.0143°/m for average rotational error on the KITTI odometry benchmark. Moreover, the learned depth is leveraged to generate a dense 3D map. As a result, an entire visual SLAM system, that is, learning monocular odometry combined with dense 3D mapping, is achieved

New Techniques in Gastrointestinal Endoscopy

As result of progress, endoscopy has became more complex, using more sophisticated devices and has claimed a special form. In this moment, the gastroenterologist performing endoscopy has to be an expert in macroscopic view of the lesions in the gut, with good skills for using standard endoscopes, with good experience in ultrasound (for performing endoscopic ultrasound), with pathology experience for confocal examination. It is compulsory to get experience and to have patience and attention for the follow-up of thousands of images transmitted during capsule endoscopy or to have knowledge in physics necessary for autofluorescence imaging endoscopy. Therefore, the idea of an endoscopist has changed. Examinations mentioned need a special formation, a superior level of instruction, accessible to those who have already gained enough experience in basic diagnostic endoscopy. This is the reason for what these new issues of endoscopy are presented in this book of New techniques in Gastrointestinal Endoscopy

Localization and Tracking of Intestinal Paths for Wireless Capsule Endoscopy

Wireless capsule endoscopy (WCE) is a non-invasive technology used for visual inspection of the human gastrointestinal (GI) tract. Localization of the capsule is a vital component of the system, as this enables physicians to identify the position of abnormalities. Several approaches exist that use the received signal strength (RSS) of the radio frequency (RF) signals for localization. However, few of these utilize the sparseness of the signals. Due to intestinal motility, the capsule positions will change with time. The distance travelled by the capsule in the intestine, however, remains more or less constant with time. In this thesis, a compressive sensing (CS) based localization algorithm is presented, that utilize signal sparsity in the RSS measurements. Different L1-minimization algorithms are used to find the sparse location vector. The performance is evaluated by electromagnetic (EM) simulations performed on a human voxel model, using narrow-band (NB) and ultra wide-band (UWB) signals. From intestinal positions, the distance the capsule has travelled is estimated by use of Kalman- and particle filters. It was found that localization accuracy of a few millimeters is possible under ideal conditions, when the RSS measurements are generated from a path loss model. When using path loss data from the EM simulations, localization accuracy on the order of 20-30 mm was achievable for NB signals. Use of UWB signals resulted in localization errors between 35-60 mm, depending on frequency range and bandwidth. From generated intestinal positions, the travelled distance was estimated with a minimum accuracy of a few millimeters, when using a VNL Kalman filter and moderate amounts of observation noise. The results are found from a limited amount of data. In order to increase the confidence in the presented results, the performance of the localization algorithm and the filters should be evaluated with a larger number of datasets