A hybrid localization method for Wireless Capsule Endoscopy (WCE)

Wireless capsule endoscopy (WCE) is a well-established diagnostic tool for visualizing the Gastrointestinal (GI) tract. WCE provides a unique view of the GI system with minimum discomfort for patients. Doctors can determine the type and severity of abnormality by analyzing the taken images. Early diagnosis helps them act and treat the disease in its earlier stages. However, the location information is missing in the frames. Pictures labeled by their location assist doctors in prescribing suitable medicines. The disease progress can be monitored, and the best treatment can be advised for the patients. Furthermore, at the time of surgery, it indicates the correct position for operation. Several attempts have been performed to localize the WCE accurately. Methods such as Radio frequency (RF), magnetic, image processing, ultrasound, and radiative imaging techniques have been investigated. Each one has its strengths and weaknesses. RF-based and magnetic-based localization methods need an external reference point, such as a belt or box around the patient, which limits their activities and causes discomfort. Computing the location solely based on an external reference could not distinguish between GI motion from capsule motion. Hence, this relative motion causes errors in position estimation. The GI system can move inside the body, while the capsule is stationary inside the intestine. This proposal presents two pose fusion methods, Method 1 and Method 2, that compensate for the relative motion of the GI tract with respect to the body. Method 1 is based on the data fusion from the Inertial measurement unit (IMU) sensor and side wall cameras. The IMU sensor consists of 9 Degree-Of-Freedom (DOF), including a gyroscope, an accelerometer, and a magnetometer to monitor the capsule’s orientation and its heading vector (the heading vector is a three-dimensional vector pointing to the direction of the capsule's head). Four monochromic cameras are placed at the side of the capsule to measure the displacement. The proposed method computes the heading vector using IMU data. Then, the heading vector is fused with displacements to estimate the 3D trajectory. This method has high accuracy in the short term. Meanwhile, due to the accumulation of errors from side wall cameras, the estimated trajectory tends to drift over time. Method 2 was developed to resolve the drifting issue while keeping the same positioning error. The capsule is equipped with four side wall cameras and a magnet. Magnetic localization acquires the capsule’s global position using 9 three-axis Hall effect sensors. However, magnetic localization alone cannot distinguish between the capsule’s and GI tract’s motions. To overcome this issue and increase tracking accuracy, side wall cameras are utilized, which are responsible for measuring the capsule’s movement, not the involuntary motion of the GI system. A complete setup is designed to test the capsule and perform the experiments. The results show that Method 2 has an average position error of only 3.5 mm and can compensate for the GI tract’s relative movements. Furthermore, environmental parameters such as magnetic interference and the nonhomogeneous structure of the GI tract have little influence on our system compared to the available magnetic localization methods. The experiment showed that Method 2 is suitable for localizing the WCE inside the body

Edge cross-section profile for colonoscopic object detection

Colorectal cancer is the second leading cause of cancer-related deaths, claiming close to 50,000 lives annually in the United States alone. Colonoscopy is an important screening tool that has contributed to a significant decline in colorectal cancer-related deaths. During colonoscopy, a tiny video camera at the tip of the endoscope generates a video signal of the internal mucosa of the human colon. The video data is displayed on a monitor for real-time diagnosis by the endoscopist. Despite the success of colonoscopy in lowering cancer-related deaths, a significant miss rate for detection of both large polyps and cancers is estimated around 4-12%. As a result, in recent years, many computer-aided object detection techniques have been developed with the ultimate goal to assist the endoscopist in lowering the polyp miss rate. Automatic object detection in recorded video data during colonoscopy is challenging due to the noisy nature of endoscopic images caused by camera motion, strong light reflections, the wide angle lens that cannot be automatically focused, and the location and appearance variations of objects within the colon. The unique characteristics of colonoscopy video require new image/video analysis techniques. The dissertation presents our investigation on edge cross-section profile (ECSP), a local appearance model, for colonoscopic object detection. We propose several methods to derive new features on ECSP from its surrounding region pixels, its first-order derivative profile, and its second-order derivative profile. These ECSP features describe discriminative patterns for different types of objects in colonoscopy. The new algorithms and software using the ECSP features can effectively detect three representative types of objects and extract their corresponding semantic unit in terms of both accuracy and analysis time. The main contributions of dissertation are summarized as follows. The dissertation presents 1) a new ECSP calculation method and feature-based ECSP method that extracts features on ECSP for object detection, 2) edgeless ECSP method that calculates ECSP without using edges, 3) part-based multi-derivative ECSP algorithm that segments ECSP, its 1st - order and its 2nd - order derivative functions into parts and models each part using the method that is suitable to that part, 4) ECSP based algorithms for detecting three representative types of colonoscopic objects including appendiceal orifices, endoscopes during retroflexion operations, and polyps and extracting videos or segmented shots containing these objects as semantic units, and 5) a software package that implements these techniques and provides meaningful visual feedback of the detected results to the endoscopist. Ideally, we would like the software to provide feedback to the endoscopist before the next video frame becomes available and to process video data at the rate in which the data are captured (typically at about 30 frames per second (fps)). This real-time requirement is difficult to achieve using today\u27s affordable off-the-shelf workstations. We aim for achieving near real-time performance where the analysis and feedback complete at the rate of at least 1 fps. The dissertation has the following broad impacts. Firstly, the performance study shows that our proposed ECSP based techniques are promising both in terms of the detection rate and execution time for detecting the appearance of the three aforementioned types of objects in colonoscopy video. Our ECSP based techniques can be extended to both detect other types of colonoscopic objects such as diverticula, lumen and vessel, and analyze other endoscopy procedures, such as laparoscopy, upper gastrointestinal endoscopy, wireless capsule endoscopy and EGD. Secondly, to our best knowledge, our polyp detection system is the only computer-aided system that can warn the endoscopist the appearance of polyps in near real time. Our retroflexion detection system is also the first computer-aided system that can detect retroflexion in near real-time. Retroflexion is a maneuver used by the endoscopist to inspect the colon area that is hard to reach. The use of our system in future clinical trials may contribute to the decline in the polyp miss rate during live colonoscopy. Our system may be used as a training platform for novice endoscopists. Lastly, the automatic documentation of detected semantic units of colonoscopic objects can be helpful to discover unknown patterns of colorectal cancers or new diseases and used as educational resources for endoscopic research

Wireless capsule endoscope for targeted drug delivery

The diagnosis and treatment of pathologies of the gastrointestinal (GI) tract are performed routinely by gastroenterologists using endoscopes and colonoscopes, however the small intestinal tract is beyond the reach of these conventional systems. Attempts have been made to access the small intestines with wireless capsule endoscopes (WCE). These pill-sized cameras take pictures of the intestinal wall and then relay them back for evaluation. This practice enables the detection and diagnosis of pathologies of the GI tract such as Crohn's disease, small intestinal tumours such as lymphoma and small intestinal cancer. The problems with these systems are that they have limited diagnostic capabilities and they do not offer the ability to perform therapy to the affected areas leaving only the options of administering large quantities of drugs or surgical intervention. To address the issue of administering therapy in the small intestinal tract this thesis presents an active swallowable microrobotic platform which has novel functionality enabling the microrobot to treat pathologies through a targeted drug delivery system. This thesis first reviews the state-of-the-art in WCE through the evaluation of current and past literature. A review of current practises such as flexible sigmoidoscopy, virtual colonoscopy and wireless capsule endoscopy are presented. The following sections review the state-of-the-art in methods of resisting peristalsis, drug targeting systems and drug delivery. A review of actuators is presented, in the context of WCE, with a view to evaluate their acceptability in adding functionality to current WCEs. The thesis presents a novel biologically-inspired holding mechanism which overcomes the issue of resisting natural peristalsis in the GI tract. An analysis of the two components of peristaltic force, circumferential and longitudinal peristaltic contractions, are presented to ensure correct functionality of the holding mechanism. A detailed analysis of the motorised method employed to deploy the expanding mechanism is described and a 5:1 scale prototype is presented which characterises the gearbox and validates the holding mechanism. The functionality of WCE is further extended by the inclusion of a novel targeting mechanism capable of delivering a metered dose of medication to a target site of interest in the GI tract. A solution to the problem of positioning a needle within a 360 degree envelope, operating the needle and safely retracting the needle in the GI tract is discussed. A comprehensive analysis of the mechanism to manoeuvre the needle is presented and validation of the mechanism is demonstrated through the evaluation of scale prototypes. Finally a drug delivery system is presented which can expel a 1 ml dose of medication, stored onboard the capsule, into the subcutaneous tissue of the GI tract wall. An analysis of the force required to expel the medication in a set period of time is presented and the design and analysis of a variable pitch conical compression spring which will be used to deliver the medication is discussed. A thermo mechanical trigger mechanism is presented which will be employed to release the compressed conical spring. Experimental results using 1:1 scale prototype parts validate the performance of the mechanisms.Open Acces

Towards tactile sensing active capsule endoscopy

Examination of the gastrointestinal(GI) tract has traditionally been performed using tethered endoscopy tools with limited reach and more recently with passive untethered capsule endoscopy with limited capability. Inspection of small intestines is only possible using the latter capsule endoscopy with on board camera system. Limited to visual means it cannot detect features beneath the lumen wall if they have not affected the lumen structure or colour. This work presents an improved capsule endoscopy system with locomotion for active exploration of the small intestines and tactile sensing to detect deformation of the capsule outer surface when it follows the intestinal wall. In laboratory conditions this system is capable of identifying sub-lumen features such as submucosal tumours.Through an extensive literary review the current state of GI tract inspection in particular using remote operated miniature robotics, was investigated, concluding no solution currently exists that utilises tactile sensing with a capsule endoscopy. In order to achieve such a platform, further investigation was made in to tactile sensing technologies, methods of locomotion through the gut, and methods to support an increased power requirement for additional electronics and actuation. A set of detailed criteria were compiled for a soft formed sensor and flexible bodied locomotion system. The sensing system is built on the biomimetic tactile sensing device, Tactip, \cite{Chorley2008, Chorley2010, Winstone2012, Winstone2013} which has been redesigned to fit the form of a capsule endoscopy. These modifications have required a $360^{o}$ cylindrical sensing surface with $360^{o}$ panoramic optical system. Multi-material 3D printing has been used to build an almost complete sensor assembly with a combination of hard and soft materials, presenting a soft compliant tactile sensing system that mimics the tactile sensing methods of the human finger. The cylindrical Tactip has been validated using artificial submucosal tumours in laboratory conditions. The first experiment has explored the new form factor and measured the device's ability to detect surface deformation when travelling through a pipe like structure with varying lump obstructions. Sensor data was analysed and used to reconstruct the test environment as a 3D rendered structure. A second tactile sensing experiment has explored the use of classifier algorithms to successfully discriminate between three tumour characteristics; shape, size and material hardness. Locomotion of the capsule endoscopy has explored further bio-inspiration from earthworm's peristaltic locomotion, which share operating environment similarities. A soft bodied peristaltic worm robot has been developed that uses a tuned planetary gearbox mechanism to displace tendons that contract each worm segment. Methods have been identified to optimise the gearbox parameter to a pipe like structure of a given diameter. The locomotion system has been tested within a laboratory constructed pipe environment, showing that using only one actuator, three independent worm segments can be controlled. This configuration achieves comparable locomotion capabilities to that of an identical robot with an actuator dedicated to each individual worm segment. This system can be miniaturised more easily due to reduced parts and number of actuators, and so is more suitable for capsule endoscopy. Finally, these two developments have been integrated to demonstrate successful simultaneous locomotion and sensing to detect an artificial submucosal tumour embedded within the test environment. The addition of both tactile sensing and locomotion have created a need for additional power beyond what is available from current battery technology. Early stage work has reviewed wireless power transfer (WPT) as a potential solution to this problem. Methods for optimisation and miniaturisation to implement WPT on a capsule endoscopy have been identified with a laboratory built system that validates the methods found. Future work would see this combined with a miniaturised development of the robot presented. This thesis has developed a novel method for sub-lumen examination. With further efforts to miniaturise the robot it could provide a comfortable and non-invasive procedure to GI tract inspection reducing the need for surgical procedures and accessibility for earlier stage of examination. Furthermore, these developments have applicability in other domains such as veterinary medicine, industrial pipe inspection and exploration of hazardous environments

Low-Cost Technologies for Flexible Endoscopy: Design, Control and Autonomy for a Water-Jet Actuated Soft Continuum Endoscope

Despite the outstanding diagnostic performance brought by new technologies in medicine, cancer remains a significant burden worldwide. In addition to prevention strategies, the ability to detect malignancy early is crucial in enabling effective treatment and dramatically increasing the survival rate of patients. In the case of gastric cancer, diagnosis is generally performed using Flexible Endoscopy (or Endoscope) (FE). The FE has been proven to be a powerful, reliable and cost-effective tool in the fight against gastric cancer. However, its effectiveness strongly depends on the skills of trained Gastro Enterologists (GE) who perform the procedures. Moreover, accessibility and availability of such tools are often limited to people residing in major cities, while remote and rural areas remain poorly served by their health systems. The advent of robotics in medicine offers a new solution to these problems. When possible, automating diagnostic procedures or surgical tasks has the potential to deliver reliable, repeatable and cost-effective alternatives to standard human-in-the-loop procedures. Embedding autonomous capabilities into a machine, optimally designed to execute a specific task, could enable the device to automatically adapt to different conditions and non-skilled personnel to perform the procedure by supervising the actions of the robotic platform. In these scenarios, safety represents a major concern and in the majority of the cases, a safe interaction between the robot and the tissues can be guaranteed by building compliant robots made of soft materials. However, if the possibility of using compliant devices offers a number of advantages to the final user or patient, it defines a series of technical challenges that have to be addressed to deliver a stable and reliable control of the platform. Finally, by adopting low-cost designs, single-use solutions can be realised to address the issue and complication of sterilisation. This dissertation discusses the research effort targeted at the development of a water-jet actuated low-cost, disposable gastroscopy platform to offer a safe, cost-effective, fault-free alternative to standard FE

Dual-Continuum Design Approach for Intuitive and Low-Cost Upper Gastrointestinal Endoscopy

Objective: This paper introduces a methodology to design intuitive, low-cost, and portable devices for visual inspection of the upper gastrointestinal tract. Methods: The proposed approach mechanically couples a multi-backbone continuum structure, as the user interface, and a parallel bellows actuator, as the endoscopic tip. Analytical modeling techniques derived from continuum robotics were adopted to describe the endoscopic tip motion from user input, accounting for variations in component size and pneumatic compressibility. The modeling framework was used to improve intuitiveness of user-to-task mapping. This was assessed against a 1:1 target, while ease-of-use was validated using landmark identification tasks performed in a stomach simulator by one expert and ten non-expert users; benchmarked against conventional flexible endoscopy. Pre-clinical validation consisted of comparative trials in in-vivo porcine and human cadaver models. Results: Target mapping was achieved with an average error of $5^\circ$ in bending angle. Simulated endoscopies were performed by an expert user successfully, within a time comparable to conventional endoscopy ( $<$ 1 min difference). Non-experts using the proposed device achieved visualization of the stomach in a shorter time (9 s faster on average) than with a conventional endoscope. The estimated cost is $<$ 10 USD and $<$ 30 USD for disposable and reusable parts, respectively. Significance and Conclusions: Flexible endoscopes are complex and expensive devices, actuated via non-intuitive cable-driven mechanisms. They frequently break, requiring costly repair, and necessitate a dedicated reprocessing facility to prevent cross contamination. The proposed solution is portable, inexpensive, and easy to use, thus lending itself to disposable use by personnel without formal training in flexible endoscopy

Activity profiling for minimally invasive surgery

Imperial Users onl

Bio-inspired robotic control in underactuation: principles for energy efficacy, dynamic compliance interactions and adaptability.

Biological systems achieve energy efficient and adaptive behaviours through extensive autologous and exogenous compliant interactions. Active dynamic compliances are created and enhanced from musculoskeletal system (joint-space) to external environment (task-space) amongst the underactuated motions. Underactuated systems with viscoelastic property are similar to these biological systems, in that their self-organisation and overall tasks must be achieved by coordinating the subsystems and dynamically interacting with the environment. One important question to raise is: How can we design control systems to achieve efficient locomotion, while adapt to dynamic conditions as the living systems do? In this thesis, a trajectory planning algorithm is developed for underactuated microrobotic systems with bio-inspired self-propulsion and viscoelastic property to achieve synchronized motion in an energy efficient, adaptive and analysable manner. The geometry of the state space of the systems is explicitly utilized, such that a synchronization of the generalized coordinates is achieved in terms of geometric relations along the desired motion trajectory. As a result, the internal dynamics complexity is sufficiently reduced, the dynamic couplings are explicitly characterised, and then the underactuated dynamics are projected onto a hyper-manifold. Following such a reduction and characterization, we arrive at mappings of system compliance and integrable second-order dynamics with the passive degrees of freedom. As such, the issue of trajectory planning is converted into convenient nonlinear geometric analysis and optimal trajectory parameterization. Solutions of the reduced dynamics and the geometric relations can be obtained through an optimal motion trajectory generator. Theoretical background of the proposed approach is presented with rigorous analysis and developed in detail for a particular example. Experimental studies are conducted to verify the effectiveness of the proposed method. Towards compliance interactions with the environment, accurate modelling or prediction of nonlinear friction forces is a nontrivial whilst challenging task. Frictional instabilities are typically required to be eliminated or compensated through efficiently designed controllers. In this work, a prediction and analysis framework is designed for the self-propelled vibro-driven system, whose locomotion greatly relies on the dynamic interactions with the nonlinear frictions. This thesis proposes a combined physics-based and analytical-based approach, in a manner that non-reversible characteristic for static friction, presliding as well as pure sliding regimes are revealed, and the frictional limit boundaries are identified. Nonlinear dynamic analysis and simulation results demonstrate good captions of experimentally observed frictional characteristics, quenching of friction-induced vibrations and satisfaction of energy requirements. The thesis also performs elaborative studies on trajectory tracking. Control schemes are designed and extended for a class of underactuated systems with concrete considerations on uncertainties and disturbances. They include a collocated partial feedback control scheme, and an adaptive variable structure control scheme with an elaborately designed auxiliary control variable. Generically, adaptive control schemes using neural networks are designed to ensure trajectory tracking. Theoretical background of these methods is presented with rigorous analysis and developed in detail for particular examples. The schemes promote the utilization of linear filters in the control input to improve the system robustness. Asymptotic stability and convergence of time-varying reference trajectories for the system dynamics are shown by means of Lyapunov synthesis