Surgical Subtask Automation for Intraluminal Procedures using Deep Reinforcement Learning

Intraluminal procedures have opened up a new sub-field of minimally invasive surgery that use flexible instruments to navigate through complex luminal structures of the body, resulting in reduced invasiveness and improved patient benefits. One of the major challenges in this field is the accurate and precise control of the instrument inside the human body. Robotics has emerged as a promising solution to this problem. However, to achieve successful robotic intraluminal interventions, the control of the instrument needs to be automated to a large extent. The thesis first examines the state-of-the-art in intraluminal surgical robotics and identifies the key challenges in this field, which include the need for safe and effective tool manipulation, and the ability to adapt to unexpected changes in the luminal environment. To address these challenges, the thesis proposes several levels of autonomy that enable the robotic system to perform individual subtasks autonomously, while still allowing the surgeon to retain overall control of the procedure. The approach facilitates the development of specialized algorithms such as Deep Reinforcement Learning (DRL) for subtasks like navigation and tissue manipulation to produce robust surgical gestures. Additionally, the thesis proposes a safety framework that provides formal guarantees to prevent risky actions. The presented approaches are evaluated through a series of experiments using simulation and robotic platforms. The experiments demonstrate that subtask automation can improve the accuracy and efficiency of tool positioning and tissue manipulation, while also reducing the cognitive load on the surgeon. The results of this research have the potential to improve the reliability and safety of intraluminal surgical interventions, ultimately leading to better outcomes for patients and surgeons

Image-guidance in endoscopic pituitary surgery: an in-silico study of errors involved in tracker-based techniques

Background: Endoscopic endonasal surgery is an established minimally invasive technique for resecting pituitary adenomas. However, understanding orientation and identifying critical neurovascular structures in this anatomically dense region can be challenging. In clinical practice, commercial navigation systems use a tracked pointer for guidance. Augmented Reality (AR) is an emerging technology used for surgical guidance. It can be tracker based or vision based, but neither is widely used in pituitary surgery. Methods: This pre-clinical study aims to assess the accuracy of tracker-based navigation systems, including those that allow for AR. Two setups were used to conduct simulations: (1) the standard pointer setup, tracked by an infrared camera; and (2) the endoscope setup that allows for AR, using reflective markers on the end of the endoscope, tracked by infrared cameras. The error sources were estimated by calculating the Euclidean distance between a point’s true location and the point’s location after passing it through the noisy system. A phantom study was then conducted to verify the in-silico simulation results and show a working example of image-based navigation errors in current methodologies. Results: The errors of the tracked pointer and tracked endoscope simulations were 1.7 and 2.5 mm respectively. The phantom study showed errors of 2.14 and 3.21 mm for the tracked pointer and tracked endoscope setups respectively. Discussion: In pituitary surgery, precise neighboring structure identification is crucial for success. However, our simulations reveal that the errors of tracked approaches were too large to meet the fine error margins required for pituitary surgery. In order to achieve the required accuracy, we would need much more accurate tracking, better calibration and improved registration techniques

Endoscopy

Endoscopy is a fast moving field, and new techniques are continuously emerging. In recent decades, endoscopy has evolved and branched out from a diagnostic modality to enhanced video and computer assisting imaging with impressive interventional capabilities. The modern endoscopy has seen advances not only in types of endoscopes available, but also in types of interventions amenable to the endoscopic approach. To date, there are a lot more developments that are being trialed. Modern endoscopic equipment provides physicians with the benefit of many technical advances. Endoscopy is an effective and safe procedure even in special populations including pediatric patients and renal transplant patients. It serves as the tool for diagnosis and therapeutic interventions of many organs including gastrointestinal tract, head and neck, urinary tract and others

Advanced Endoscopic Navigation:Surgical Big Data,Methodology,and Applications

随着科学技术的飞速发展，健康与环境问题日益成为人类面临的最重大问题之一。信息科学、计算机技术、电子工程与生物医学工程等学科的综合应用交叉前沿课题，研究现代工程技术方法，探索肿瘤癌症等疾病早期诊断、治疗和康复手段。本论文综述了计算机辅助微创外科手术导航、多模态医疗大数据、方法论及其临床应用：从引入微创外科手术导航概念出发，介绍了医疗大数据的术前与术中多模态医学成像方法、阐述了先进微创外科手术导航的核心流程包括计算解剖模型、术中实时导航方案、三维可视化方法及交互式软件技术，归纳了各类微创外科手术方法的临床应用。同时，重点讨论了全球各种手术导航技术在临床应用中的优缺点，分析了目前手术导航领域内的最新技术方法。在此基础上，提出了微创外科手术方法正向数字化、个性化、精准化、诊疗一体化、机器人化以及高度智能化的发展趋势。【Abstract】Interventional endoscopy (e.g., bronchoscopy, colonoscopy, laparoscopy, cystoscopy) is a widely performed procedure that involves either diagnosis of suspicious lesions or guidance for minimally invasive surgery in a variety of organs within the body cavity. Endoscopy may also be used to guide the introduction of certain items (e.g., stents) into the body. Endoscopic navigation systems seek to integrate big data with multimodal information (e.g., computed tomography, magnetic resonance images, endoscopic video sequences, ultrasound images, external trackers) relative to the patient's anatomy, control the movement of medical endoscopes and surgical tools, and guide the surgeon's actions during endoscopic interventions. Nevertheless, it remains challenging to realize the next generation of context-aware navigated endoscopy. This review presents a broad survey of various aspects of endoscopic navigation, particularly with respect to the development of endoscopic navigation techniques. First, we investigate big data with multimodal information involved in endoscopic navigation. Next, we focus on numerous methodologies used for endoscopic navigation. We then review different endoscopic procedures in clinical applications. Finally, we discuss novel techniques and promising directions for the development of endoscopic navigation.X.L. acknowledges funding from the Fundamental Research Funds for the Central Universities. T.M.P. acknowledges funding from the Canadian Foundation for Innovation, the Canadian Institutes for Health Research, the National Sciences and Engineering Research Council of Canada, and a grant from Intuitive Surgical Inc

Design and Modeling of Multi-Arm Continuum Robots

Continuum robots are snake-like systems able to deliver optimal therapies to pathologies deep inside the human cavity by following 3D complex paths. They show promise when anatomical pathways need to be traversed thanks to their enhanced flexibility and dexterity and show advantages when deployed in the field of single-port surgery. This PhD thesis concerns the development and modelling of multi-arm and hybrid continuum robots for medical interventions. The flexibility and steerability of the robot’s end-effector are achieved through concentric tube technology and push/pull technology. Medical robotic prototypes have been designed as proof of concepts and testbeds of the proposed theoretical works.System design considers the limitations and constraints that occur in the surgical procedures for which the systems were proposed for. Specifically, two surgical applications are considered. Our first prototype was designed to deliver multiple tools to the eye cavity for deep orbital interventions focusing on a currently invasive intervention named Optic Nerve Sheath Fenestration (ONSF). This thesis presents the end-to-end design, engineering and modelling of the prototype. The developed prototype is the first suggested system to tackle the challenges (limited workspace, need for enhanced flexibility and dexterity, danger for harming tissue with rigid instruments, extensive manipulation of the eye) arising in ONSF. It was designed taking into account the clinical requirements and constraints while theoretical works employing the Cosserat rod theory predict the shape of the continuum end-effector. Experimental runs including ex vivo experimental evaluations, mock-up surgical scenarios and tests with and without loading conditions prove the concept of accessing the eye cavity. Moreover, a continuum robot for thoracic interventions employing push/pull technology was designed and manufactured. The developed system can reach deep seated pathologies in the lungs and access regions in the bronchial tree that are inaccessible with rigid and straight instruments either robotically or manually actuated. A geometrically exact model of the robot that considers both the geometry of the robot and mechanical properties of the backbones is presented. It can predict the shape of the bronchoscope without the constant curvature assumption. The proposed model can also predict the robot shape and micro-scale movements accurately in contrast to the classic geometric model which provides an accurate description of the robot’s differential kinematics for large scale movements

Multimodal Optical Spectroscopy and Imaging for improving Cancer Detection in the Head and Neck at Endoscopy

Ph.DDOCTOR OF PHILOSOPH

Spatial frequency domain imaging towards improved detection of gastrointestinal cancers

Early detection and treatment of gastrointestinal cancers has been shown to drastically improve patients survival rates. However, wide population based screening for gastrointestinal cancers is not feasible due to its high cost, risk of potential complications, and time consuming nature. This thesis forms the proposal for the development of a cost-effective, minimally invasive device to return quantitative tissue information for gastrointestinal cancer detection in-vivo using spatial frequency domain imaging (SFDI). SFDI is a non-invasive imaging technique which can return close to real time maps of absorption and reduced scattering coefficients by projecting a 2D sinusoidal pattern onto a sample of interest. First a low-cost, conventional bench top system was constructed to characterise tissue mimicking phantoms. Phantoms were fabricated with specific absorption and reduced scattering coefficients, mimicking the variation in optical properties typically seen in healthy, cancerous, and pre-cancerous oesophageal tissue. The system shows accurate retrieval of absorption and reduced scattering coefficients of 19% and 11% error respectively. However, this bench top system consists of a bulky projector and is therefore not feasible for in-vivo imaging. For SFDI systems to be feasible for in-vivo imaging, they are required to be miniaturised. Many conditions must be considered when doing this such as various illumination conditions, lighting conditions and system geometries. Therefore to aid in the miniaturisation of the bench top system, an SFDI system was simulated in the open-source ray tracing software Blender, where the capability to simulate these conditions is possible. A material of tunable absorption and scattering properties was characterised such that the specific absorption and reduced scattering coefficients of the material were known. The simulated system shows capability in detecting optical properties of typical gastrointestinal conditions in an up-close, planar geometry, as well in a non-planar geometry of a tube simulating a lumen. Optical property imaging in the non-planar, tubular geometry was done with the use of a novel illumination pattern, developed for this work. Finally, using the knowledge gained from the simulation model, the bench top system was miniaturised to a 3 mm diameter prototype. The novel use of a fiber array producing the necessary interfering fringe patterns replaced the bulky projector. The system showed capability to image phantoms simulating typical gastrointestinal conditions at two wavelengths (515 and 660 nm), measuring absorption and reduced scattering coefficients with 15% and 6% accuracy in comparison to the bench top system for the fabricated phantoms. It is proposed that this system may be used for cost-effective, minimally invasive, quantitative imaging of the gastrointestinal tract in-vivo, providing enhanced contrast for difficult to detect cancers

Laryngeal somatosensory deficits in Parkinson’s disease: implications for speech respiratory and phonatory control

Parkinson’s disease (PD) is often associated with substantial impairment of speech respiratory and phonatory control. However, the degree to which these impairments are related to abnormal laryngeal sensory function is unknown. This study examined whether individuals with PD exhibited abnormal and more asymmetric laryngeal somatosensory function compared with healthy controls, and whether these deficits were associated with disease and voice severity. Nineteen PD participants were tested and compared with 18 healthy controls. Testing included endoscopic assessment of laryngeal somatosensory function, with aerodynamic and acoustic assessment of respiratory and phonatory control, and clinical ratings of voice and disease severity. PD participants exhibited significantly abnormal and asymmetric laryngeal somatosensory function compared with healthy controls. Sensory deficits were significantly associated with timing of phonatory onset, voice intensity, respiratory driving pressure, laryngeal resistance, lung volume expended per syllable, disease severity, and voice severity. These results suggest that respiratory and phonatory control are influenced by laryngeal somatosensory function, that speech-related deficits in PD are related to abnormal laryngeal somatosensory function, and that this function may degrade as a function of disease severity. Thus, PD may represent a model of airway sensorimotor disintegration, highlighting the important role of the basal ganglia and related neural networks in the integration of laryngeal sensory input for speech-related motor control

Modeling, Analysis, Force Sensing and Control of Continuum Robots for Minimally Invasive Surgery

This dissertation describes design, modeling and application of continuum robotics for surgical applications, specifically parallel continuum robots (PCRs) and concentric tube manipulators (CTMs). The introduction of robotics into surgical applications has allowed for a greater degree of precision, less invasive access to more remote surgical sites, and user-intuitive interfaces with enhanced vision systems. The most recent developments have been in the space of continuum robots, whose exible structure create an inherent safety factor when in contact with fragile tissues. The design challenges that exist involve balancing size and strength of the manipulators, controlling the manipulators over long transmission pathways, and incorporating force sensing and feedback from the manipulators to the user. Contributions presented in this work include: (1) prototyping, design, force sensing, and force control investigations of PCRs, and (2) prototyping of a concentric tube manipulator for use in a standard colonoscope. A general kinetostatic model is presented for PCRs along with identification of multiple physical constraints encountered in design and construction. Design considerations and manipulator capabilities are examined in the form of matrix metrics and ellipsoid representations. Finally, force sensing and control are explored and experimental results are provided showing the accuracy of force estimates based on actuation force measurements and control capabilities. An overview of the design requirements, manipulator construction, analysis and experimental results are provided for a CTM used as a tool manipulator in a traditional colonoscope. Currently, tools used in colonoscopic procedures are straight and exit the front of the scope with 1 DOF of operation (jaws of a grasper, tightening of a loop, etc.). This research shows that with a CTM deployed, the dexterity of these tools can be increased dramatically, increasing accuracy of tool operation, ease of use and safety of the overall procedure. The prototype investigated in this work allows for multiple tools to be used during a single procedure. Experimental results show the feasibility and advantages of the newly-designed manipulators