A Magnetic Localization Technique Designed for use with Magnetic Levitation Systems.

M.S. Thesis. University of Hawaiʻi at Mānoa 2017

Unsupervised Odometry and Depth Learning for Endoscopic Capsule Robots

In the last decade, many medical companies and research groups have tried to convert passive capsule endoscopes as an emerging and minimally invasive diagnostic technology into actively steerable endoscopic capsule robots which will provide more intuitive disease detection, targeted drug delivery and biopsy-like operations in the gastrointestinal(GI) tract. In this study, we introduce a fully unsupervised, real-time odometry and depth learner for monocular endoscopic capsule robots. We establish the supervision by warping view sequences and assigning the re-projection minimization to the loss function, which we adopt in multi-view pose estimation and single-view depth estimation network. Detailed quantitative and qualitative analyses of the proposed framework performed on non-rigidly deformable ex-vivo porcine stomach datasets proves the effectiveness of the method in terms of motion estimation and depth recovery.Comment: submitted to IROS 201

Doctor of Philosophy

dissertationClosed-loop control of wireless capsule endoscopes is an active area of research because it would drastically improve screening of the gastrointestinal tract. Traditional endoscopic procedures are unable to view the entire gastrointestinal tract and current commercial wireless capsule endoscopes are limited in their effectiveness due to their passive nature. This dissertation advances the field of active capsule endoscopy by developing methods to localize the full six-degree-of-freedom (6-DOF) pose of a screw-type magnetic capsule while it is being propelled through a lumen (such as the small intestines) using an external rotating magnetic dipole. The same external magnetic dipole is utilized for both propulsion and localization. Hardware was designed and constructed to enable testing of the magnetic localization and propulsion methods, including a robotic end-effector used as the external actuator magnet, and a prototype capsule embedded with Hall-effect sensors. Due to the use of a rotating magnetic field for propulsion, at any given time, the capsule can be in one of three regimes: synchronously rotating with the applied field, in "step-out" where it is free to move but the external field is rotating too quickly for the capsule to remain synchronously rotating, or completely stationary. We show that it is only necessary to distinguish whether or not the capsule is synchronously rotating (i.e., a single localization method can be used for a capsule in either the step-out or stationary regimes). Two magnetic localization methods are developed. The first uses nonlinear least squares to estimate the capsule's pose when it has no (or approximately no) net motion (e.g., to find the initial capsule pose or when it is stuck in an intestinal fold). The second method estimates the 6-DOF capsule pose as it synchronously rotates with the applied magnetic field using a square-root variant of the Unscented Kalman filter. A simple process model is adopted that restricts the capsule's movement to translation along and rotation about its principle axis. The capsule is actively propelled forward or backward, but it is not actively steered, rather, steering is provided by the lumen. The propulsion parameters that transform magnetic force and torque to the capsule's spatial velocity and angular velocity are estimated with an additional square-root Unscented Kalman filter to enable the capsule to navigate heterogeneous environments such as the small intestines. An optimized localization-propulsion system is described using the two localization algorithms and prior work in screw-type magnetic capsule propulsion with a single rotating dipole field. The capsule's regime is determined and the corresponding localization method is employed. Based on the capsule's estimated pose and the current estimates of its propulsion parameters, the actuator magnet's pose relative to the capsule is optimized to maximize the capsule's forward propulsion. Using this system, our prototype magnetic capsule successfully completed U-shaped and S-shaped trajectories in fresh bovine intestines with an average forward velocity of 5.5mm/s and 3.5 mm/s, respectively. At this rate it would take approximately 18-30 minutes to traverse the 6 meters of a typical human small intestine

Determining the Position and Orientation of In-body Medical Instruments Using Reactive Magnetic Field Mapping

There has been a huge demand for localizing in-body medical instruments (IBMI), such as wireless capsule endoscope (WCE) and nasogastric tube (NGT). Some stud ies have been conducted to solve this issue over the last three decades. In these studies, they either used a permanent magnet (PM), a static current source (SCS), radio frequency (RF) fields or integration of two of these. The PM is a stable and reliable magnetic field source. However, due to the size restriction of the NGT and the WCE, only a small PM can be used. Subsequently, the small size issue causes low power delivery at the larger tracking distance. Also, the PM field is very susceptible to ambient noise, and the PM-based localization is not possible in ap plications requiring robotic actuation. Even though an SCS can be used to replace the permanent magnet, and thus the current level can be varied in relation to the dis tance for optimized power delivery, it requires a relatively high power to generate a higher strength magnetic field. Consequently, a more powerful and larger battery is needed to feed the circuit.Radio frequency field sources require high frequencies to achieve sufficient precision, but these frequencies undergo high attenuation in the body. Therefore, the low-frequency RF field is preferred 1 . In the near-field 2 , plane wave assumption of the far-field fails for localization methods since the waves in this region are spherical. Hence, the wave-front has to be formulated by both the range and the direction of arrival (DOA). The DOA requires the phase difference of neighbouring sensors to be calculated. However, if the operating wavelength is much larger than the distance between the source and the receiver, it is not feasible to compute the phase difference between the neigh bouring sensors. Thus, there are numerous algorithms in the literature to overcome these issues, such as MUSIC or ESPRIT which are either complicated or computa tionally expensive. In RF-based localization, generally the time of arrival (TA), the time differ ence of arrival (TDA), the angle of arrival (AOA) and the received signal strength (RSS) are widely used for localization. However, the TA and TDA require accu rate knowledge of field speed and good time synchronization. It is not possible to accurately know while travelling through the body tissues due to complexity of the tissues. The AOA is also impractical for intra-body applications owing to multiple reflections signal from the tissues, commonly known as the multipath effect. The RSS precision is dependent on good knowledge of power loss in complex body tis sues. Also, the RSS method requires accurate knowledge of the transmitted signal strength. However, the power of transmitted frequencies may vary due to the ca pacitive effect of human tissue on Resonant frequency of source, hence RSS-based techniques prove difficult in practice. Therefore, a novel method of mapping the magnetic field vector in the near field region is proposed. This magnetic field mapping (MFM) uses single-axis coils placed orthogonally with respect to a sensor plane (SP). These single-axis sensors pick up only the orthogonal component of the magnetic field, which varies as a function of the orientation of the source and distance to the source. Thus, using this information, the field strength captured by each sensor is mapped to its correspond ing position on the SP as pixels. Next, these field strengths with known positions are used to detect the location and orientation of the field source relative to the SP. MATLAB and CST Microwave simulation were conducted, and many laboratory experiments were performed, and we show that the novel technique not only over comes the issues faced in the methods mentioned above but also accomplishes an accurate source positioning with a precision of better than ± 0.5 cm in 3-D and orientation with a maximum error of ±5◦

Endoscopic Tactile Capsule for Non-Polypoid Colorectal Tumour Detection

An endoscopic tactile robotic capsule, embedding miniaturized MEMS force sensors, is presented. The capsule is conceived to provide automatic palpation of non-polypoid colorectal tumours during colonoscopy, since it is characterized by high degree of dysplasia, higher invasiveness and lower detection rates with respect to polyps. A first test was performed employing a silicone phantom that embedded inclusions with variable hardness and curvature. A hardness-based classification was implemented, demonstrating detection robustness to curvature variation. By comparing a set of supervised classification algorithms, a weighted 3-nearest neighbor classifier was selected. A bias force normalization model was introduced in order to make different acquisition sets consistent. Parameters of this model were chosen through a particle swarm optimization method. Additionally, an ex-vivo test was performed to assess the capsule detection performance when magnetically-driven along a colonic tissue. Lumps were identified as voltage peaks with a prominence depending on the total magnetic force applied to the capsule. Accuracy of 94 % in hardness classification was achieved, while a 100 % accuracy is obtained for the lump detection within a tolerance of 5 mm from the central path described by the capsule. In real application scenario, we foresee our device aiding physicians to detect tumorous tissues

Advanced Endoscopic Navigation:Surgical Big Data,Methodology,and Applications

随着科学技术的飞速发展，健康与环境问题日益成为人类面临的最重大问题之一。信息科学、计算机技术、电子工程与生物医学工程等学科的综合应用交叉前沿课题，研究现代工程技术方法，探索肿瘤癌症等疾病早期诊断、治疗和康复手段。本论文综述了计算机辅助微创外科手术导航、多模态医疗大数据、方法论及其临床应用：从引入微创外科手术导航概念出发，介绍了医疗大数据的术前与术中多模态医学成像方法、阐述了先进微创外科手术导航的核心流程包括计算解剖模型、术中实时导航方案、三维可视化方法及交互式软件技术，归纳了各类微创外科手术方法的临床应用。同时，重点讨论了全球各种手术导航技术在临床应用中的优缺点，分析了目前手术导航领域内的最新技术方法。在此基础上，提出了微创外科手术方法正向数字化、个性化、精准化、诊疗一体化、机器人化以及高度智能化的发展趋势。【Abstract】Interventional endoscopy (e.g., bronchoscopy, colonoscopy, laparoscopy, cystoscopy) is a widely performed procedure that involves either diagnosis of suspicious lesions or guidance for minimally invasive surgery in a variety of organs within the body cavity. Endoscopy may also be used to guide the introduction of certain items (e.g., stents) into the body. Endoscopic navigation systems seek to integrate big data with multimodal information (e.g., computed tomography, magnetic resonance images, endoscopic video sequences, ultrasound images, external trackers) relative to the patient's anatomy, control the movement of medical endoscopes and surgical tools, and guide the surgeon's actions during endoscopic interventions. Nevertheless, it remains challenging to realize the next generation of context-aware navigated endoscopy. This review presents a broad survey of various aspects of endoscopic navigation, particularly with respect to the development of endoscopic navigation techniques. First, we investigate big data with multimodal information involved in endoscopic navigation. Next, we focus on numerous methodologies used for endoscopic navigation. We then review different endoscopic procedures in clinical applications. Finally, we discuss novel techniques and promising directions for the development of endoscopic navigation.X.L. acknowledges funding from the Fundamental Research Funds for the Central Universities. T.M.P. acknowledges funding from the Canadian Foundation for Innovation, the Canadian Institutes for Health Research, the National Sciences and Engineering Research Council of Canada, and a grant from Intuitive Surgical Inc

Capsule endoscopy of the future: What's on the horizon?

Capsule endoscopes have evolved from passively moving diagnostic devices to actively moving systems with potential therapeutic capability. In this review, we will discuss the state of the art, define the current shortcomings of capsule endoscopy, and address research areas that aim to overcome said shortcomings. Developments in capsule mobility schemes are emphasized in this text, with magnetic actuation being the most promising endeavor. Research groups are working to integrate sensor data and fuse it with robotic control to outperform today's standard invasive procedures, but in a less intrusive manner. With recent advances in areas such as mobility, drug delivery, and therapeutics, we foresee a translation of interventional capsule technology from the bench-top to the clinical setting within the next 10 years