Closed Loop Control of a Tethered Magnetic Capsule Endoscope

Magnetic field gradients have repeatedly been shown to be the most feasible mechanism for gastrointestinal capsule endoscope actuation. An inverse quartic magnetic force variation with distance results in large force gradients induced by small movements of a driving magnet; this necessitates robotic actuation of magnets to implement stable control of the device. A typical system consists of a serial robot with a permanent magnet at its end effector that actuates a capsule with an embedded permanent magnet. We present a tethered capsule system where a capsule with an embedded magnet is closed loop controlled in 2 degree-of-freedom in position and 2 degree-offreedom in orientation. Capitalizing on the magnetic field of the external driving permanent magnet, the capsule is localized in 6- D allowing for both position and orientation feedback to be used in a control scheme. We developed a relationship between the serial robot’s joint parameters and the magnetic force and torque that is exerted onto the capsule. Our methodology was validated both in a dynamic simulation environment where a custom plug-in for magnetic interaction was written, as well as on an experimental platform. The tethered capsule was demonstrated to follow desired trajectories in both position and orientation with accuracy that is acceptable for colonoscopy

Nonholonomic Closed-loop Velocity Control of a Soft-tethered Magnetic Capsule Endoscope

In this paper, we demonstrate velocity-level closedloop control of a tethered magnetic capsule endoscope that is actuated via serial manipulator with a permanent magnet at its end-effector. Closed-loop control (2 degrees-of-freedom in position, and 2 in orientation) is made possible with the use of a real-time magnetic localization algorithm that utilizes the actuating magnetic field and thus does not require additional hardware. Velocity control is implemented to create smooth motion that is clinically necessary for colorectal cancer diagnostics. Our control algorithm generates a spline that passes through a set of input points that roughly defines the shape of the desired trajectory. The velocity controller acts in the tangential direction to the path, while a secondary position controller enforces a nonholonomic constraint on capsule motion. A soft nonholonomic constraint is naturally imposed by the lumen while we enforce a strict constraint for both more accurate estimation of tether disturbance and hypothesized intuitiveness for a clinician’s teleoperation. An integrating disturbance force estimation control term is introduced to predict the disturbance of the tether. This paper presents the theoretical formulations and experimental validation of our methodology. Results show the system’s ability to achieve a repeatable velocity step response with low steady-state error as well as ability of the tethered capsule to maneuver around a bend

Enhanced real-time pose estimation for closed-loop robotic manipulation of magnetically actuated capsule endoscopes

Pose estimation methods for robotically guided magnetic actuation of capsule endoscopes have recently enabled trajectory following and automation of repetitive endoscopic maneuvers. However, these methods face significant challenges in their path to clinical adoption including the presence of regions of magnetic field singularity, where the accuracy of the system degrades, and the need for accurate initialization of the capsule's pose. In particular, the singularity problem exists for any pose estimation method that utilizes a single source of magnetic field if the method does not rely on the motion of the magnet to obtain multiple measurements from different vantage points. We analyze the workspace of such pose estimation methods with the use of the point-dipole magnetic field model and show that singular regions exist in areas where the capsule is nominally located during magnetic actuation. Since the dipole model can approximate most magnetic field sources, the problem discussed herein pertains to a wider set of pose estimation techniques. We then propose a novel hybrid approach employing static and time-varying magnetic field sources and show that this system has no regions of singularity. The proposed system was experimentally validated for accuracy, workspace size, update rate and performance in regions of magnetic singularity. The system performed as well or better than prior pose estimation methods without requiring accurate initialization and was robust to magnetic singularity. Experimental demonstration of closed-loop control of a tethered magnetic device utilizing the developed pose estimation technique is provided to ascertain its suitability for robotically guided capsule endoscopy. Hence, advances in closed-loop control and intelligent automation of magnetically actuated capsule endoscopes can be further pursued toward clinical realization by employing this pose estimation system

Autonomous Retroflexion of a Magnetic Flexible Endoscope

Retroflexion during colonoscopy is typically only practiced in the wider proximal and distal ends of the large intestine owing to the stiff nature of the colonoscope. This inability to examine the proximal side of the majority of colon folds contributes to today's suboptimal colorectal cancer detection rates. We have developed an algorithm for autonomous retroflexion of a flexible endoscope that is actuated magnetically from the tip. The magnetic wrench applied on the tip of the endoscope is optimized in real time with data from pose detection to compute motions of the actuating magnet. This is the first example of a completely autonomous maneuver by a magnetic endoscope for exploration of the gastrointestinal tract. The proposed approach was validated in plastic tubes of various diameters with a success rate of 98.8% for separation distances up to 50 mm. Additionally, a set of trials was conducted in an excised porcine colon observing a success rate of 100% with a mean time of 19.7 s. In terms of clinical safety, the maximum stress that is applied on the colon wall with our methodology is an order of magnitude below what would damage tissue

Hybrid 6-DoFs magnetic localization for robotic capsule endoscopes compatible with high-grade magnetic field navigation

This paper proposes a hybrid 6-DoFs localization system for endoscopic magnetic capsules, compatible with external high-grade permanent magnetic locomotion. The proposed localization system, which is able to provide an accurate estimation of the endoscopic capsule pose, finds application in the robotic endoscopy field to provide efficient closed-loop navigation of a magnetically-driven tethered capsule. It takes advantage of two optimization steps based on a triangulation approach, i.e. (1) mathematical approximations of the magnetic field, and (2) minimization of the magnetic field mean square deviation. The proposed localization system was tested in two different in-vitro scenarios for mimicking the clinical cases that a magnetic capsule would encounter during tele-operated magnetic navigation. The development phase was preceded by an in-depth work-space analysis to lay the groundwork for the localization design and implementation. Results of the hybrid 6-DoFs localization system show a significant accuracy in accordance with the state-of-the-art, i.e. < 5 mm and < 5° in position and orientation, but introducing benefits in expanding the work-space by increasing the number of electromagnets on the operating table as an independent solution with respect to the external magnetic locomotion source

New Robotic Technologies in Cancer Colon Screening

Colorectal cancer (CRC) is the 3rd most common cause of cancer death worldwide. Regular screening of the asymptomatic population can drastically reduce the mortality rate. CRC screening includes several proceedings although the gold standard remains optical colonoscopy (OC), which is unpleasant, causes pain and discomfort. New technologies exemplified by capsule endoscopy (CE) constitute alternative painless solutions and despite their limitations, e.g., passive locomotion and absence of on-board instrumentation, are being increasingly used for CRC screening. Research and development centres are investigating novel advanced robotic technologies for diagnostic and therapeutic use. These include wireless communication, active locomotion, sensors, diagnostic, and therapeutic instruments. This review describes the traditional OC procedure and the existing robotic technologies for CRC

Capsule endoscopy of the future: What's on the horizon?

Capsule endoscopes have evolved from passively moving diagnostic devices to actively moving systems with potential therapeutic capability. In this review, we will discuss the state of the art, define the current shortcomings of capsule endoscopy, and address research areas that aim to overcome said shortcomings. Developments in capsule mobility schemes are emphasized in this text, with magnetic actuation being the most promising endeavor. Research groups are working to integrate sensor data and fuse it with robotic control to outperform today's standard invasive procedures, but in a less intrusive manner. With recent advances in areas such as mobility, drug delivery, and therapeutics, we foresee a translation of interventional capsule technology from the bench-top to the clinical setting within the next 10 years

Development of A Soft Robotic Approach for An Intra-abdominal Wireless Laparoscopic Camera

In Single-Incision Laparoscopic Surgery (SILS), the Magnetic Anchoring and Guidance System (MAGS) arises as a promising technique to provide larger workspaces and field of vision for the laparoscopes, relief space for other instruments, and require fewer incisions. Inspired by MAGS, many concept designs related to fully insertable magnetically driven laparoscopes are developed and tested on the transabdominal operation. However, ignoring the tissue interaction and insertion procedure, most of the designs adopt rigid structures, which not only damage the patients\u27 tissue with excess stress concentration and sliding motion but also require complicated operation for the insertion. Meanwhile, lacking state tracking of the insertable camera including pose and contact force, the camera systems operate in open-loop control. This provides mediocre locomotion precision and limited robustness to uncertainties in the environment. This dissertation proposes, develops, and validates a soft robotic approach for an intra-abdominal wireless laparoscopic camera. Contributions presented in this work include (1) feasibility of a soft intra-abdominal laparoscopic camera with friendly tissue interaction and convenient insertion, (2) six degrees of freedom (DOF) real-time localization, (3) Closed-loop control for a robotic-assisted laparoscopic system and (4) untethering solution for wireless communication and high-quality video transmission. Embedding magnet pairs into the camera and external actuator, the camera can be steered and anchored along the abdominal wall through transabdominal magnetic coupling. To avoid the tissue rapture by the sliding motion and dry friction, a wheel structure is applied to achieve rolling motion. Borrowing the ideas from soft robotic research, the main body of the camera implements silicone material, which grants it the bendability to passively attach along the curved abdominal wall and the deformability for easier insertion. The six-DOF pose is estimated in real-time with internal multi-sensor fusion and Newton-Raphson iteration. Combining the pose tracking and force-torque sensor measurement, an interaction model between the deformable camera and tissue is established to evaluate the interaction force over the tissue surface. Moreover, the proposed laparoscopic system is integrated with a multi-DOF manipulator into a robotic-assisted surgical system, where a closed-loop control is realized based on a feedback controller and online optimization. Finally, the wireless control and video streaming are accomplished with Bluetooth Low Energy (BLE) and Analog Video (AV) transmission. Experimental assessments have been implemented to evaluate the performance of the laparoscopic system

Active Stabilization of Interventional Tasks Utilizing a Magnetically Manipulated Endoscope

Magnetically actuated robots have become increasingly popular in medical endoscopy over the past decade. Despite the significant improvements in autonomy and control methods, progress within the field of medical magnetic endoscopes has mainly been in the domain of enhanced navigation. Interventional tasks such as biopsy, polyp removal, and clip placement are a major procedural component of endoscopy. Little advancement has been done in this area due to the problem of adequately controlling and stabilizing magnetically actuated endoscopes for interventional tasks. In the present paper we discuss a novel model-based Linear Parameter Varying (LPV) control approach to provide stability during interventional maneuvers. This method linearizes the non-linear dynamic interaction between the external actuation system and the endoscope in a set of equilibria, associated to different distances between the magnetic source and the endoscope, and computes different controllers for each equilibrium. This approach provides the global stability of the overall system and robustness against external disturbances. The performance of the LPV approach is compared to an intelligent teleoperation control method (based on a Proportional Integral Derivative (PID) controller), on the Magnetic Flexible Endoscope (MFE) platform. Four biopsies in different regions of the colon and at two different system equilibria are performed. Both controllers are asked to stabilize the endoscope in the presence of external disturbances (i.e. the introduction of the biopsy forceps through the working channel of the endoscope). The experiments, performed in a benchtop colon simulator, show a maximum reduction of the mean orientation error of the endoscope of 45.8% with the LPV control compared to the PID controller

Explicit Model Predictive Control of a Magnetic Flexible Endoscope

In this letter, explicit model predictive control is applied in conjunction with nonlinear optimization to a magnetically actuated flexible endoscope for the first time. The approach is aimed at computing the motion of the external permanent magnet, given the desired forces and torques. The strategy described here takes advantage of the nonlinear nature of the magnetic actuation and explicitly considers the workspace boundaries, as well as the actuation constraints. Initially, a simplified dynamic model of the tethered capsule, based on the Euler-Lagrange equations is developed. Subsequently, the explicit model predictive control is described and a novel approach for the external magnet positioning, based on a single step, nonlinear optimisation routine, is proposed. Finally, the strategy is implemented on the experimental platform, where bench-top trials are performed on a realistic colon phantom, showing the effectiveness of the technique. The work presented here constitutes an initial exploration for model-based control techniques applied to magnetically manipulated payloads; the techniques described here may be applied to a wide range of devices, including flexible endoscopes and wireless capsules. To our knowledge, this is the first example of advanced closed-loop control of magnetic capsules