2D Reconstruction of Small Intestine's Interior Wall

Examining and interpreting of a large number of wireless endoscopic images from the gastrointestinal tract is a tiresome task for physicians. A practical solution is to automatically construct a two dimensional representation of the gastrointestinal tract for easy inspection. However, little has been done on wireless endoscopic image stitching, let alone systematic investigation. The proposed new wireless endoscopic image stitching method consists of two main steps to improve the accuracy and efficiency of image registration. First, the keypoints are extracted by Principle Component Analysis and Scale Invariant Feature Transform (PCA-SIFT) algorithm and refined with Maximum Likelihood Estimation SAmple Consensus (MLESAC) outlier removal to find the most reliable keypoints. Second, the optimal transformation parameters obtained from first step are fed to the Normalised Mutual Information (NMI) algorithm as an initial solution. With modified Marquardt-Levenberg search strategy in a multiscale framework, the NMI can find the optimal transformation parameters in the shortest time. The proposed methodology has been tested on two different datasets - one with real wireless endoscopic images and another with images obtained from Micro-Ball (a new wireless cubic endoscopy system with six image sensors). The results have demonstrated the accuracy and robustness of the proposed methodology both visually and quantitatively.Comment: Journal draf

Development of A Soft Robotic Approach for An Intra-abdominal Wireless Laparoscopic Camera

In Single-Incision Laparoscopic Surgery (SILS), the Magnetic Anchoring and Guidance System (MAGS) arises as a promising technique to provide larger workspaces and field of vision for the laparoscopes, relief space for other instruments, and require fewer incisions. Inspired by MAGS, many concept designs related to fully insertable magnetically driven laparoscopes are developed and tested on the transabdominal operation. However, ignoring the tissue interaction and insertion procedure, most of the designs adopt rigid structures, which not only damage the patients\u27 tissue with excess stress concentration and sliding motion but also require complicated operation for the insertion. Meanwhile, lacking state tracking of the insertable camera including pose and contact force, the camera systems operate in open-loop control. This provides mediocre locomotion precision and limited robustness to uncertainties in the environment. This dissertation proposes, develops, and validates a soft robotic approach for an intra-abdominal wireless laparoscopic camera. Contributions presented in this work include (1) feasibility of a soft intra-abdominal laparoscopic camera with friendly tissue interaction and convenient insertion, (2) six degrees of freedom (DOF) real-time localization, (3) Closed-loop control for a robotic-assisted laparoscopic system and (4) untethering solution for wireless communication and high-quality video transmission. Embedding magnet pairs into the camera and external actuator, the camera can be steered and anchored along the abdominal wall through transabdominal magnetic coupling. To avoid the tissue rapture by the sliding motion and dry friction, a wheel structure is applied to achieve rolling motion. Borrowing the ideas from soft robotic research, the main body of the camera implements silicone material, which grants it the bendability to passively attach along the curved abdominal wall and the deformability for easier insertion. The six-DOF pose is estimated in real-time with internal multi-sensor fusion and Newton-Raphson iteration. Combining the pose tracking and force-torque sensor measurement, an interaction model between the deformable camera and tissue is established to evaluate the interaction force over the tissue surface. Moreover, the proposed laparoscopic system is integrated with a multi-DOF manipulator into a robotic-assisted surgical system, where a closed-loop control is realized based on a feedback controller and online optimization. Finally, the wireless control and video streaming are accomplished with Bluetooth Low Energy (BLE) and Analog Video (AV) transmission. Experimental assessments have been implemented to evaluate the performance of the laparoscopic system

Doctor of Philosophy

dissertationClosed-loop control of wireless capsule endoscopes is an active area of research because it would drastically improve screening of the gastrointestinal tract. Traditional endoscopic procedures are unable to view the entire gastrointestinal tract and current commercial wireless capsule endoscopes are limited in their effectiveness due to their passive nature. This dissertation advances the field of active capsule endoscopy by developing methods to localize the full six-degree-of-freedom (6-DOF) pose of a screw-type magnetic capsule while it is being propelled through a lumen (such as the small intestines) using an external rotating magnetic dipole. The same external magnetic dipole is utilized for both propulsion and localization. Hardware was designed and constructed to enable testing of the magnetic localization and propulsion methods, including a robotic end-effector used as the external actuator magnet, and a prototype capsule embedded with Hall-effect sensors. Due to the use of a rotating magnetic field for propulsion, at any given time, the capsule can be in one of three regimes: synchronously rotating with the applied field, in "step-out" where it is free to move but the external field is rotating too quickly for the capsule to remain synchronously rotating, or completely stationary. We show that it is only necessary to distinguish whether or not the capsule is synchronously rotating (i.e., a single localization method can be used for a capsule in either the step-out or stationary regimes). Two magnetic localization methods are developed. The first uses nonlinear least squares to estimate the capsule's pose when it has no (or approximately no) net motion (e.g., to find the initial capsule pose or when it is stuck in an intestinal fold). The second method estimates the 6-DOF capsule pose as it synchronously rotates with the applied magnetic field using a square-root variant of the Unscented Kalman filter. A simple process model is adopted that restricts the capsule's movement to translation along and rotation about its principle axis. The capsule is actively propelled forward or backward, but it is not actively steered, rather, steering is provided by the lumen. The propulsion parameters that transform magnetic force and torque to the capsule's spatial velocity and angular velocity are estimated with an additional square-root Unscented Kalman filter to enable the capsule to navigate heterogeneous environments such as the small intestines. An optimized localization-propulsion system is described using the two localization algorithms and prior work in screw-type magnetic capsule propulsion with a single rotating dipole field. The capsule's regime is determined and the corresponding localization method is employed. Based on the capsule's estimated pose and the current estimates of its propulsion parameters, the actuator magnet's pose relative to the capsule is optimized to maximize the capsule's forward propulsion. Using this system, our prototype magnetic capsule successfully completed U-shaped and S-shaped trajectories in fresh bovine intestines with an average forward velocity of 5.5mm/s and 3.5 mm/s, respectively. At this rate it would take approximately 18-30 minutes to traverse the 6 meters of a typical human small intestine

sCAM: An Untethered Insertable Laparoscopic Surgical Camera Robot

Fully insertable robotic imaging devices represent a promising future of minimally invasive laparoscopic vision. Emerging research efforts in this field have resulted in several proof-of-concept prototypes. One common drawback of these designs derives from their clumsy tethering wires which not only cause operational interference but also reduce camera mobility. Meanwhile, these insertable laparoscopic cameras are manipulated without any pose information or haptic feedback, which results in open loop motion control and raises concerns about surgical safety caused by inappropriate use of force.This dissertation proposes, implements, and validates an untethered insertable laparoscopic surgical camera (sCAM) robot. Contributions presented in this work include: (1) feasibility of an untethered fully insertable laparoscopic surgical camera, (2) camera-tissue interaction characterization and force sensing, (3) pose estimation, visualization, and feedback with sCAM, and (4) robotic-assisted closed-loop laparoscopic camera control. Borrowing the principle of spherical motors, camera anchoring and actuation are achieved through transabdominal magnetic coupling in a stator-rotor manner. To avoid the tethering wires, laparoscopic vision and control communication are realized with dedicated wireless links based on onboard power. A non-invasive indirect approach is proposed to provide real-time camera-tissue interaction force measurement, which, assisted by camera-tissue interaction modeling, predicts stress distribution over the tissue surface. Meanwhile, the camera pose is remotely estimated and visualized using complementary filtering based on onboard motion sensing. Facilitated by the force measurement and pose estimation, robotic-assisted closed-loop control has been realized in a double-loop control scheme with shared autonomy between surgeons and the robotic controller.The sCAM has brought robotic laparoscopic imaging one step further toward less invasiveness and more dexterity. Initial ex vivo test results have verified functions of the implemented sCAM design and the proposed force measurement and pose estimation approaches, demonstrating the technical feasibility of a tetherless insertable laparoscopic camera. Robotic-assisted control has shown its potential to free surgeons from low-level intricate camera manipulation workload and improve precision and intuitiveness in laparoscopic imaging

A Magnetic Actuated Fully Insertable Robotic Camera System for Single Incision Laparoscopic Surgery

Minimally Invasive Surgery (MIS) is a common surgical procedure which makes tiny incisions in the patients anatomy, inserting surgical instruments and using laparoscopic cameras to guide the procedure. Compared with traditional open surgery, MIS allows surgeons to perform complex surgeries with reduced trauma to the muscles and soft tissues, less intraoperative hemorrhaging and postoperative pain, and faster recovery time. Surgeons rely heavily on laparoscopic cameras for hand-eye coordination and control during a procedure. However, the use of a standard laparoscopic camera, achieved by pushing long sticks into a dedicated small opening, involves multiple incisions for the surgical instruments. Recently, single incision laparoscopic surgery (SILS) and natural orifice translumenal endoscopic surgery (NOTES) have been introduced to reduce or even eliminate the number of incisions. However, the shared use of a single incision or a natural orifice for both surgical instruments and laparoscopic cameras further reduces dexterity in manipulating instruments and laparoscopic cameras with low efficient visual feedback. In this dissertation, an innovative actuation mechanism design is proposed for laparoscopic cameras that can be navigated, anchored and orientated wirelessly with a single rigid body to improve surgical procedures, especially for SILS. This design eliminates the need for an articulated design and the integrated motors to significantly reduce the size of the camera. The design features a unified mechanism for anchoring, navigating, and rotating a fully insertable camera by externally generated rotational magnetic field. The key component and innovation of the robotic camera is the magnetic driving unit, which is referred to as a rotor, driven externally by a specially designed magnetic stator. The rotor, with permanent magnets (PMs) embedded in a capsulated camera, can be magnetically coupled to a stator placed externally against or close to a dermal surface. The external stator, which consists of PMs and coils, generates 3D rotational magnetic field that thereby produces torque to rotate the rotor for desired camera orientation, and force to serve as an anchoring system that keeps the camera steady during a surgical procedure. Experimental assessments have been implemented to evaluate the performance of the camera system

Optimized state feedback regulation of 3DOF helicopter system via extremum seeking

In this paper, an optimized state feedback regulation of a 3 degree of freedom (DOF) helicopter is designed via extremum seeking (ES) technique. Multi-parameter ES is applied to optimize the tracking performance via tuning State Vector Feedback with Integration of the Control Error (SVFBICE). Discrete multivariable version of ES is developed to minimize a cost function that measures the performance of the controller. The cost function is a function of the error between the actual and desired axis positions. The controller parameters are updated online as the optimization takes place. This method significantly decreases the time in obtaining optimal controller parameters. Simulations were conducted for the online optimization under both fixed and varying operating conditions. The results demonstrate the usefulness of using ES for preserving the maximum attainable performance

The Design and Development of a Mobile Colonoscopy Robot

The conventional colonoscopy is a common procedure used to access the colon. Despite it being considered the Gold Standard procedure for colorectal cancer diagnosis and treatment, it has a number of major drawbacks, including high patient discomfort, infrequent but serious complications and high skill required to perform the procedure. There are a number of potential alternatives to the conventional colonoscopy, from augmenting the colonoscope to using Computed Tomography Colonography (CTC) - a completely non-invasive method. However, a truly effective, all-round alternative has yet to be found. This thesis explores the design and development of a novel solution: a fully mobile colonoscopy robot called “RollerBall”. Unlike current passive diagnostic capsules, such as PillCam, this device uses wheels at the end of adjustable arms to provide locomotion through the colon, while providing a stable platform for the use of diagnostic and therapeutic tools. The work begins by reviewing relevant literature to better understand the problem and potential solutions. RollerBall is then introduced and its design described in detail. A robust prototype was then successfully fabricated using a 3D printing technique and its performance assessed in a series of benchtop experiments. These showed that the mechanisms functioned as intended and encouraged the further development of the concept. Next, the fundamental requirement of gaining traction on the colon was shown to be possible using hexagonal shaped, macro-scale tread patterns. A friction coefficient ranging between 0.29 and 0.55 was achieved with little trauma to the tissue substrate. The electronics hardware and control were then developed and evaluated in a series of tests in silicone tubes. An open-loop strategy was first used to establish the control algorithm to map the user inputs to motor outputs (wheel speeds). These tests showed the efficacy of the locomotion technique and the control algorithm used, but they highlighted the need for autonomy. To address this, feedback was included to automate the adjusting of the arm angle and amount of force applied by the device; a forward facing camera was also used to automate the orientation control by tracking a user-defined target. Force and orientation control were then combined to show that semi-autonomous control was possible and as a result, it was concluded that clinical use may be feasible in future developments

Intelligent Circuits and Systems

ICICS-2020 is the third conference initiated by the School of Electronics and Electrical Engineering at Lovely Professional University that explored recent innovations of researchers working for the development of smart and green technologies in the fields of Energy, Electronics, Communications, Computers, and Control. ICICS provides innovators to identify new opportunities for the social and economic benefits of society.　 This conference bridges the gap between academics and R&D institutions, social visionaries, and experts from all strata of society to present their ongoing research activities and foster research relations between them. It provides opportunities for the exchange of new ideas, applications, and experiences in the field of smart technologies and finding global partners for future collaboration. The ICICS-2020 was conducted in two broad categories, Intelligent Circuits & Intelligent Systems and Emerging Technologies in Electrical Engineering

Optical Methods in Sensing and Imaging for Medical and Biological Applications

The recent advances in optical sources and detectors have opened up new opportunities for sensing and imaging techniques which can be successfully used in biomedical and healthcare applications. This book, entitled ‘Optical Methods in Sensing and Imaging for Medical and Biological Applications’, focuses on various aspects of the research and development related to these areas. The book will be a valuable source of information presenting the recent advances in optical methods and novel techniques, as well as their applications in the fields of biomedicine and healthcare, to anyone interested in this subject