Towards automated visual flexible endoscope navigation

Background:\ud The design of flexible endoscopes has not changed significantly in the past 50 years. A trend is observed towards a wider application of flexible endoscopes with an increasing role in complex intraluminal therapeutic procedures. The nonintuitive and nonergonomical steering mechanism now forms a barrier in the extension of flexible endoscope applications. Automating the navigation of endoscopes could be a solution for this problem. This paper summarizes the current state of the art in image-based navigation algorithms. The objectives are to find the most promising navigation system(s) to date and to indicate fields for further research.\ud Methods:\ud A systematic literature search was performed using three general search terms in two medical–technological literature databases. Papers were included according to the inclusion criteria. A total of 135 papers were analyzed. Ultimately, 26 were included.\ud Results:\ud Navigation often is based on visual information, which means steering the endoscope using the images that the endoscope produces. Two main techniques are described: lumen centralization and visual odometry. Although the research results are promising, no successful, commercially available automated flexible endoscopy system exists to date.\ud Conclusions:\ud Automated systems that employ conventional flexible endoscopes show the most promising prospects in terms of cost and applicability. To produce such a system, the research focus should lie on finding low-cost mechatronics and technologically robust steering algorithms. Additional functionality and increased efficiency can be obtained through software development. The first priority is to find real-time, robust steering algorithms. These algorithms need to handle bubbles, motion blur, and other image artifacts without disrupting the steering process

Open-source virtual bronchoscopy for image guided navigation

This thesis describes the development of an open-source system for virtual bronchoscopy used in combination with electromagnetic instrument tracking. The end application is virtual navigation of the lung for biopsy of early stage cancer nodules. The open-source platform 3D Slicer was used for creating freely available algorithms for virtual bronchscopy. Firstly, the development of an open-source semi-automatic algorithm for prediction of solitary pulmonary nodule malignancy is presented. This approach may help the physician decide whether to proceed with biopsy of the nodule. The user-selected nodule is segmented in order to extract radiological characteristics (i.e., size, location, edge smoothness, calcification presence, cavity wall thickness) which are combined with patient information to calculate likelihood of malignancy. The overall accuracy of the algorithm is shown to be high compared to independent experts' assessment of malignancy. The algorithm is also compared with two different predictors, and our approach is shown to provide the best overall prediction accuracy. The development of an airway segmentation algorithm which extracts the airway tree from surrounding structures on chest Computed Tomography (CT) images is then described. This represents the first fundamental step toward the creation of a virtual bronchoscopy system. Clinical and ex-vivo images are used to evaluate performance of the algorithm. Different CT scan parameters are investigated and parameters for successful airway segmentation are optimized. Slice thickness is the most affecting parameter, while variation of reconstruction kernel and radiation dose is shown to be less critical. Airway segmentation is used to create a 3D rendered model of the airway tree for virtual navigation. Finally, the first open-source virtual bronchoscopy system was combined with electromagnetic tracking of the bronchoscope for the development of a GPS-like system for navigating within the lungs. Tools for pre-procedural planning and for helping with navigation are provided. Registration between the lungs of the patient and the virtually reconstructed airway tree is achieved using a landmark-based approach. In an attempt to reduce difficulties with registration errors, we also implemented a landmark-free registration method based on a balanced airway survey. In-vitro and in-vivo testing showed good accuracy for this registration approach. The centreline of the 3D airway model is extracted and used to compensate for possible registration errors. Tools are provided to select a target for biopsy on the patient CT image, and pathways from the trachea towards the selected targets are automatically created. The pathways guide the physician during navigation, while distance to target information is updated in real-time and presented to the user. During navigation, video from the bronchoscope is streamed and presented to the physician next to the 3D rendered image. The electromagnetic tracking is implemented with 5 DOF sensing that does not provide roll rotation information. An intensity-based image registration approach is implemented to rotate the virtual image according to the bronchoscope's rotations. The virtual bronchoscopy system is shown to be easy to use and accurate in replicating the clinical setting, as demonstrated in the pre-clinical environment of a breathing lung method. Animal studies were performed to evaluate the overall system performance

Pre-clinical validation of virtual bronchoscopy using 3D Slicer

Lung cancer still represents the leading cause of cancer-related death, and the long-term survival rate remains low. Computed tomography (CT) is currently the most common imaging modality for lung diseases recognition. The purpose of this work was to develop a simple and easily accessible virtual bronchoscopy system to be coupled with a customized electromagnetic (EM) tracking system for navigation in the lung and which requires as little user interaction as possible, while maintaining high usability. The proposed method has been implemented as an extension to the open-source platform, 3D Slicer. It creates a virtual reconstruction of the airways starting from CT images for virtual navigation. It provides tools for pre-procedural planning and virtual navigation, and it has been optimized for use in combination with a of freedom EM tracking sensor. Performance of the algorithm has been evaluated in ex vivo and in vivo testing. During ex vivo testing, nine volunteer physicians tested the implemented algorithm to navigate three separate targets placed inside a breathing pig lung model. In general, the system proved easy to use and accurate in replicating the clinical setting and seemed to help choose the correct path without any previous experience or image analysis. Two separate animal studies confirmed technical feasibility and usability of the system. This work describes an easily accessible virtual bronchoscopy system for navigation in the lung. The system provides the user with a complete set of tools that facilitate navigation towards user-selected regions of interest. Results from ex vivo and in vivo studies showed that the system opens the way for potential future work with virtual navigation for safe and reliable airway disease diagnosis

Minimally Invasive Lung Tissue Differentiation Using Electrical Impedance Spectroscopy : A Comparison of the 3- and 4-Electrode Methods

Altres ajuts: Secretariat d'Universitats i Recerca, Generalitat de Catalunya; the European Social Fund.Multiple imaging techniques are used for the diagnosis of lung diseases. The choice of a technique depends on the suspected diagnosis. Computed tomography (CT) of the thorax and positron emission tomography (PET) are imaging techniques used for the detection, characterization, staging and follow-up of lung cancer, and these techniques use ionizing radiation and are radiologist-dependent. Electrical impedance spectroscopy (EIS) performed through a bronchoscopic process could serve as a minimally invasive non-ionizing method complementary to CT and PET to characterize lung tissue. The aim of this study was to analyse the feasibility and ability of minimally invasive EIS bioimpedance measures to differentiate among healthy lung, bronchial and neoplastic lung tissues through bronchoscopy using the 3- and 4-electrode methods. Tissue differentiation was performed in 13 patients using the 4-electrode method (13 healthy lung, 12 bronchial and 3 neoplastic lung tissues) and the 3-electrode method (9 healthy lung, 10 bronchial and 2 neoplastic lung tissues). One-way analysis of variance (ANOVA) showed a statistically significant difference (P < 0.001) between bronchial and healthy lung tissues for both the 3- and 4-electrode methods. The 3-electrode method seemed to differentiate cancer types through changes in the cellular structures of the tissues by both the reactance (Xc) and the resistance (R). Minimally invasive measurements obtained using the 3-electrode method seem to be most suitable for differentiating between healthy and bronchial lung tissues. In the future, EIS using the 3-electrode method could be a method complementary to PET/CT and biopsy in lung pathology diagnosis

Minimally invasive lung tissue differentiation using electrical impedance spectroscopy: a comparison of the 3- and 4-electrode methods

Multiple imaging techniques are used for the diagnosis of lung diseases. The choice of a technique depends on the suspected diagnosis. Computed tomography (CT) of the thorax and positron emission tomography (PET) are imaging techniques used for the detection, characterization, staging and follow-up of lung cancer, and these techniques use ionizing radiation and are radiologist-dependent. Electrical impedance spectroscopy (EIS) performed through a bronchoscopic process could serve as a minimally invasive non-ionizing method complementary to CT and PET to characterize lung tissue. The aim of this study was to analyse the feasibility and ability of minimally invasive EIS bioimpedance measures to differentiate among healthy lung, bronchial and neoplastic lung tissues through bronchoscopy using the 3- and 4-electrode methods. Tissue differentiation was performed in 13 patients using the 4-electrode method (13 healthy lung, 12 bronchial and 3 neoplastic lung tissues) and the 3-electrode method (9 healthy lung, 10 bronchial and 2 neoplastic lung tissues). One-way analysis of variance (ANOVA) showed a statistically significant difference (P < 0.001) between bronchial and healthy lung tissues for both the 3- and 4-electrode methods. The 3-electrode method seemed to differentiate cancer types through changes in the cellular structures of the tissues by both the reactance (Xc) and the resistance (R). Minimally invasive measurements obtained using the 3-electrode method seem to be most suitable for differentiating between healthy and bronchial lung tissues. In the future, EIS using the 3-electrode method could be a method complementary to PET/CT and biopsy in lung pathology diagnosis.Peer ReviewedPostprint (author's final draft

Development of an Alveolar Transbronchial Catheter for Concurrent Fiber Optics-Based Imaging and Fluid Delivery

Optical molecular imaging is an emerging field and high resolution optical imaging of the distal lung parenchyma has been made possible with the advent of clinically approved fiber based imaging modalities. However, currently, there is no single method of allowing the simultaneous imaging and delivery of targeted molecular imaging agents. The objective of this research is to create a catheterized device capable of fulfilling this need. We describe the rationale, development, and validation in ex vivo ovine lung to near clinical readiness of a triple lumen bronchoscopy catheter that allows concurrent imaging and fluid delivery, with the aim of clinical use to deliver multiple fluorescent compounds to image alveolar pathology. Using this device, we were able to produce high-quality images of bacterial infiltrates in ex-vivo ovine lung within 60 seconds of instilling a single microdose of (<100 mcgs) imaging agent. This has many advantages for future clinical usage over the current state of the art.status: publishe

Design and Modeling of Multi-Arm Continuum Robots

Continuum robots are snake-like systems able to deliver optimal therapies to pathologies deep inside the human cavity by following 3D complex paths. They show promise when anatomical pathways need to be traversed thanks to their enhanced flexibility and dexterity and show advantages when deployed in the field of single-port surgery. This PhD thesis concerns the development and modelling of multi-arm and hybrid continuum robots for medical interventions. The flexibility and steerability of the robot’s end-effector are achieved through concentric tube technology and push/pull technology. Medical robotic prototypes have been designed as proof of concepts and testbeds of the proposed theoretical works.System design considers the limitations and constraints that occur in the surgical procedures for which the systems were proposed for. Specifically, two surgical applications are considered. Our first prototype was designed to deliver multiple tools to the eye cavity for deep orbital interventions focusing on a currently invasive intervention named Optic Nerve Sheath Fenestration (ONSF). This thesis presents the end-to-end design, engineering and modelling of the prototype. The developed prototype is the first suggested system to tackle the challenges (limited workspace, need for enhanced flexibility and dexterity, danger for harming tissue with rigid instruments, extensive manipulation of the eye) arising in ONSF. It was designed taking into account the clinical requirements and constraints while theoretical works employing the Cosserat rod theory predict the shape of the continuum end-effector. Experimental runs including ex vivo experimental evaluations, mock-up surgical scenarios and tests with and without loading conditions prove the concept of accessing the eye cavity. Moreover, a continuum robot for thoracic interventions employing push/pull technology was designed and manufactured. The developed system can reach deep seated pathologies in the lungs and access regions in the bronchial tree that are inaccessible with rigid and straight instruments either robotically or manually actuated. A geometrically exact model of the robot that considers both the geometry of the robot and mechanical properties of the backbones is presented. It can predict the shape of the bronchoscope without the constant curvature assumption. The proposed model can also predict the robot shape and micro-scale movements accurately in contrast to the classic geometric model which provides an accurate description of the robot’s differential kinematics for large scale movements

Surgical Subtask Automation for Intraluminal Procedures using Deep Reinforcement Learning

Intraluminal procedures have opened up a new sub-field of minimally invasive surgery that use flexible instruments to navigate through complex luminal structures of the body, resulting in reduced invasiveness and improved patient benefits. One of the major challenges in this field is the accurate and precise control of the instrument inside the human body. Robotics has emerged as a promising solution to this problem. However, to achieve successful robotic intraluminal interventions, the control of the instrument needs to be automated to a large extent. The thesis first examines the state-of-the-art in intraluminal surgical robotics and identifies the key challenges in this field, which include the need for safe and effective tool manipulation, and the ability to adapt to unexpected changes in the luminal environment. To address these challenges, the thesis proposes several levels of autonomy that enable the robotic system to perform individual subtasks autonomously, while still allowing the surgeon to retain overall control of the procedure. The approach facilitates the development of specialized algorithms such as Deep Reinforcement Learning (DRL) for subtasks like navigation and tissue manipulation to produce robust surgical gestures. Additionally, the thesis proposes a safety framework that provides formal guarantees to prevent risky actions. The presented approaches are evaluated through a series of experiments using simulation and robotic platforms. The experiments demonstrate that subtask automation can improve the accuracy and efficiency of tool positioning and tissue manipulation, while also reducing the cognitive load on the surgeon. The results of this research have the potential to improve the reliability and safety of intraluminal surgical interventions, ultimately leading to better outcomes for patients and surgeons