Medical image computing and computer-aided medical interventions applied to soft tissues. Work in progress in urology

Until recently, Computer-Aided Medical Interventions (CAMI) and Medical Robotics have focused on rigid and non deformable anatomical structures. Nowadays, special attention is paid to soft tissues, raising complex issues due to their mobility and deformation. Mini-invasive digestive surgery was probably one of the first fields where soft tissues were handled through the development of simulators, tracking of anatomical structures and specific assistance robots. However, other clinical domains, for instance urology, are concerned. Indeed, laparoscopic surgery, new tumour destruction techniques (e.g. HIFU, radiofrequency, or cryoablation), increasingly early detection of cancer, and use of interventional and diagnostic imaging modalities, recently opened new challenges to the urologist and scientists involved in CAMI. This resulted in the last five years in a very significant increase of research and developments of computer-aided urology systems. In this paper, we propose a description of the main problems related to computer-aided diagnostic and therapy of soft tissues and give a survey of the different types of assistance offered to the urologist: robotization, image fusion, surgical navigation. Both research projects and operational industrial systems are discussed

Robotic System Development for Precision MRI-Guided Needle-Based Interventions

This dissertation describes the development of a methodology for implementing robotic systems for interventional procedures under intraoperative Magnetic Resonance Imaging (MRI) guidance. MRI is an ideal imaging modality for surgical guidance of diagnostic and therapeutic procedures, thanks to its ability to perform high resolution, real-time, and high soft tissue contrast imaging without ionizing radiation. However, the strong magnetic field and sensitivity to radio frequency signals, as well as tightly confined scanner bore render great challenges to developing robotic systems within MRI environment. Discussed are potential solutions to address engineering topics related to development of MRI-compatible electro-mechanical systems and modeling of steerable needle interventions. A robotic framework is developed based on a modular design approach, supporting varying MRI-guided interventional procedures, with stereotactic neurosurgery and prostate cancer therapy as two driving exemplary applications. A piezoelectrically actuated electro-mechanical system is designed to provide precise needle placement in the bore of the scanner under interactive MRI-guidance, while overcoming the challenges inherent to MRI-guided procedures. This work presents the development of the robotic system in the aspects of requirements definition, clinical work flow development, mechanism optimization, control system design and experimental evaluation. A steerable needle is beneficial for interventional procedures with its capability to produce curved path, avoiding anatomical obstacles or compensating for needle placement errors. Two kinds of steerable needles are discussed, i.e. asymmetric-tip needle and concentric-tube cannula. A novel Gaussian-based ContinUous Rotation and Variable-curvature (CURV) model is proposed to steer asymmetric-tip needle, which enables variable curvature of the needle trajectory with independent control of needle rotation and insertion. While concentric-tube cannula is suitable for clinical applications where a curved trajectory is needed without relying on tissue interaction force. This dissertation addresses fundamental challenges in developing and deploying MRI-compatible robotic systems, and enables the technologies for MRI-guided needle-based interventions. This study applied and evaluated these techniques to a system for prostate biopsy that is currently in clinical trials, developed a neurosurgery robot prototype for interstitial thermal therapy of brain cancer under MRI guidance, and demonstrated needle steering using both asymmetric tip and pre-bent concentric-tube cannula approaches on a testbed

Towards Closed-loop, Robot Assisted Percutaneous Interventions under MRI Guidance

Image guided therapy procedures under MRI guidance has been a focused research area over past decade. Also, over the last decade, various MRI guided robotic devices have been developed and used clinically for percutaneous interventions, such as prostate biopsy, brachytherapy, and tissue ablation. Though MRI provides better soft tissue contrast compared to Computed Tomography and Ultrasound, it poses various challenges like constrained space, less ergonomic patient access and limited material choices due to its high magnetic field. Even after, advancements in MRI compatible actuation methods and robotic devices using them, most MRI guided interventions are still open-loop in nature and relies on preoperative or intraoperative images. In this thesis, an intraoperative MRI guided robotic system for prostate biopsy comprising of an MRI compatible 4-DOF robotic manipulator, robot controller and control application with Clinical User Interface (CUI) and surgical planning applications (3DSlicer and RadVision) is presented. This system utilizes intraoperative images acquired after each full or partial needle insertion for needle tip localization. Presented system was approved by Institutional Review Board at Brigham and Women\u27s Hospital(BWH) and has been used in 30 patient trials. Successful translation of such a system utilizing intraoperative MR images motivated towards the development of a system architecture for close-loop, real-time MRI guided percutaneous interventions. Robot assisted, close-loop intervention could help in accurate positioning and localization of the therapy delivery instrument, improve physician and patient comfort and allow real-time therapy monitoring. Also, utilizing real-time MR images could allow correction of surgical instrument trajectory and controlled therapy delivery. Two of the applications validating the presented architecture; closed-loop needle steering and MRI guided brain tumor ablation are demonstrated under real-time MRI guidance

Optimized non-invasive MRI protocols for characterizing tissue microstructures: applications in humans to prostate cancer and fetal brain development

This PhD project was aimed to optimize MRI protocols for pelvis imaging, in particular for the diagnosis of prostate cancer (PCa) and for the fetal brain development. Different non-invasive MRI techniques were employed to investigate biological tissues, with the purpose to obtain information on microstructures and potentially metabolism. Prostate cancer is the second most common malignancy and the fifth leading cause of death in men worldwide. Due to the high incidence of PCa and the limitations of current diagnostic methods, the primary goal of this work was to develop an MRI protocol able to improve the sensitivity of the diagnostic. The investigation of prostate cancer started with ex-vivo experiments conducted on specimens of human prostate gland, obtained after radical prostatectomy, with the 9.4T scanner at the NMR and Medical Physics Laboratory of CNR-ISC (Sapienza). Diffusion Tensor Imaging (DTI) and Diffusion Kurtosis Imaging (DKI) were performed at high-resolution (70x70 micrometers in plane) to evaluate diffusion metrics in the different prostate compartment and directly compare measurements with the histopathology results. This study proceeded with in-vivo experiments with a 3T clinical MR scanner (Philips Achieva at Policlinico Tor Vergata) on subjects with diagnosed PCa. DTI was performed with the purpose to assess its diagnostic ability in individuating and classifying PCa with different ranges of diffusion weightings, i.e. b-values. Our results showing that the diagnostic accuracy of DTI is improved with high diffusion weightings motivated our interest in performing DKI, a technique that captures water diffusion features when high b-values are employed, providing additional information on tissue microstructures, inaccessible to DTI technique. The second part of this PhD project was conducted at the Center for Magnetic Resonance Research (CMRR) in Minneapolis and was funded by the European Union's Horizon 2020 research and innovation program under the Marie Curie grant agreement No 691110 (MICROBRADAM). The study was dedicated to perform prostate cancer imaging with new contrast mechanisms, based on T1rho and T2rho relaxation times. T1rho and T2rho characterize the relaxation of the nuclear magnetization in the rotating frame and they are sensitive to molecular dynamics occurring at frequencies in the range of kHz, characteristic of several in-vivo processes, enabling the access to important information on tissue microenvironment. T1rho and T2rho imaging is limited by the intensive energy deposited by the acquisition sequence, which it is usually overcome by increasing the acquisition time, preventing the possibility of diagnostic applications. Therefore the aim of this work was to develop a new approach to perform imaging in the rotating frame with a three-dimensional acquisition method, recently developed at the CMRR, in order to address the aforementioned shortcomings. Given the incidence of PCa, this research has international interest and potentially contributes to improve not only the sensitivity of PCa diagnostic but also the knowledge of the tissue micro-changing caused by the tumor development. A part of this project was dedicated to Diffusion MRI application in woman pelvis to image fetuses during gestation. The aim of this work was to develop a fast and reliable protocol for fetal imaging to minimize mother-fetal motion artifact and perfusion effects. The protocol designed for acquisition and post-processing was employed to successfully study fetal brain development during the second and third trimester of gestation, in normal cases and in fetuses affected by ventriculomegaly disease. These preliminary data can contribute to delineate a reference standard to assess the normal progress of sulcation and myelination as well as the normative biometry of the fetal brain, improving the knowledge of brain maturation. Globally, the impact of this research lies in having demonstrated that the sensitivity of DMRI for microstructural changes in body tissue caused by cancer, brain disease or normal condition like brain maturation can be fruitfully utilized in combination with artifact correction methods. Moreover, new strategy of image reconstruction, such as 3D gradient echo, can be successfully employed to perform abdominal imaging, enriching the investigation of in-vivo systems with information on tissue microenvironment and metabolism

Medical robots for MRI guided diagnosis and therapy

Magnetic Resonance Imaging (MRI) provides the capability of imaging tissue with fine resolution and superior soft tissue contrast, when compared with conventional ultrasound and CT imaging, which makes it an important tool for clinicians to perform more accurate diagnosis and image guided therapy. Medical robotic devices combining the high resolution anatomical images with real-time navigation, are ideal for precise and repeatable interventions. Despite these advantages, the MR environment imposes constraints on mechatronic devices operating within it. This thesis presents a study on the design and development of robotic systems for particular MR interventions, in which the issue of testing the MR compatibility of mechatronic components, actuation control, kinematics and workspace analysis, and mechanical and electrical design of the robot have been investigated. Two types of robotic systems have therefore been developed and evaluated along the above aspects. (i) A device for MR guided transrectal prostate biopsy: The system was designed from components which are proven to be MR compatible, actuated by pneumatic motors and ultrasonic motors, and tracked by optical position sensors and ducial markers. Clinical trials have been performed with the device on three patients, and the results reported have demonstrated its capability to perform needle positioning under MR guidance, with a procedure time of around 40mins and with no compromised image quality, which achieved our system speci cations. (ii) Limb positioning devices to facilitate the magic angle effect for diagnosis of tendinous injuries: Two systems were designed particularly for lower and upper limb positioning, which are actuated and tracked by the similar methods as the first device. A group of volunteers were recruited to conduct tests to verify the functionality of the systems. The results demonstrate the clear enhancement of the image quality with an increase in signal intensity up to 24 times in the tendon tissue caused by the magic angle effect, showing the feasibility of the proposed devices to be applied in clinical diagnosis

Medical SLAM in an autonomous robotic system

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-operative morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilities by observing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted instruments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This thesis addresses the ambitious goal of achieving surgical autonomy, through the study of the anatomical environment by Initially studying the technology present and what is needed to analyze the scene: vision sensors. A novel endoscope for autonomous surgical task execution is presented in the first part of this thesis. Which combines a standard stereo camera with a depth sensor. This solution introduces several key advantages, such as the possibility of reconstructing the 3D at a greater distance than traditional endoscopes. Then the problem of hand-eye calibration is tackled, which unites the vision system and the robot in a single reference system. Increasing the accuracy in the surgical work plan. In the second part of the thesis the problem of the 3D reconstruction and the algorithms currently in use were addressed. In MIS, simultaneous localization and mapping (SLAM) can be used to localize the pose of the endoscopic camera and build ta 3D model of the tissue surface. Another key element for MIS is to have real-time knowledge of the pose of surgical tools with respect to the surgical camera and underlying anatomy. Starting from the ORB-SLAM algorithm we have modified the architecture to make it usable in an anatomical environment by adding the registration of the pre-operative information of the intervention to the map obtained from the SLAM. Once it has been proven that the slam algorithm is usable in an anatomical environment, it has been improved by adding semantic segmentation to be able to distinguish dynamic features from static ones. All the results in this thesis are validated on training setups, which mimics some of the challenges of real surgery and on setups that simulate the human body within Autonomous Robotic Surgery (ARS) and Smart Autonomous Robotic Assistant Surgeon (SARAS) projects

Medical SLAM in an autonomous robotic system

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-operative morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilities by observing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted instruments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This thesis addresses the ambitious goal of achieving surgical autonomy, through the study of the anatomical environment by Initially studying the technology present and what is needed to analyze the scene: vision sensors. A novel endoscope for autonomous surgical task execution is presented in the first part of this thesis. Which combines a standard stereo camera with a depth sensor. This solution introduces several key advantages, such as the possibility of reconstructing the 3D at a greater distance than traditional endoscopes. Then the problem of hand-eye calibration is tackled, which unites the vision system and the robot in a single reference system. Increasing the accuracy in the surgical work plan. In the second part of the thesis the problem of the 3D reconstruction and the algorithms currently in use were addressed. In MIS, simultaneous localization and mapping (SLAM) can be used to localize the pose of the endoscopic camera and build ta 3D model of the tissue surface. Another key element for MIS is to have real-time knowledge of the pose of surgical tools with respect to the surgical camera and underlying anatomy. Starting from the ORB-SLAM algorithm we have modified the architecture to make it usable in an anatomical environment by adding the registration of the pre-operative information of the intervention to the map obtained from the SLAM. Once it has been proven that the slam algorithm is usable in an anatomical environment, it has been improved by adding semantic segmentation to be able to distinguish dynamic features from static ones. All the results in this thesis are validated on training setups, which mimics some of the challenges of real surgery and on setups that simulate the human body within Autonomous Robotic Surgery (ARS) and Smart Autonomous Robotic Assistant Surgeon (SARAS) projects

OPTICAL NAVIGATION TECHNIQUES FOR MINIMALLY INVASIVE ROBOTIC SURGERIES

Minimally invasive surgery (MIS) involves small incisions in a patient's body, leading to reduced medical risk and shorter hospital stays compared to open surgeries. For these reasons, MIS has experienced increased demand across different types of surgery. MIS sometimes utilizes robotic instruments to complement human surgical manipulation to achieve higher precision than can be obtained with traditional surgeries. Modern surgical robots perform within a master-slave paradigm, in which a robotic slave replicates the control gestures emanating from a master tool manipulated by a human surgeon. Presently, certain human errors due to hand tremors or unintended acts are moderately compensated at the tool manipulation console. However, errors due to robotic vision and display to the surgeon are not equivalently addressed. Current vision capabilities within the master-slave robotic paradigm are supported by perceptual vision through a limited binocular view, which considerably impacts the hand-eye coordination of the surgeon and provides no quantitative geometric localization for robot targeting. These limitations lead to unexpected surgical outcomes, and longer operating times compared to open surgery. To improve vision capabilities within an endoscopic setting, we designed and built several image guided robotic systems, which obtained sub-millimeter accuracy. With this improved accuracy, we developed a corresponding surgical planning method for robotic automation. As a demonstration, we prototyped an autonomous electro-surgical robot that employed quantitative 3D structural reconstruction with near infrared registering and tissue classification methods to localize optimal targeting and suturing points for minimally invasive surgery. Results from validation of the cooperative control and registration between the vision system in a series of in vivo and in vitro experiments are presented and the potential enhancement to autonomous robotic minimally invasive surgery by utilizing our technique will be discussed

Multiparametric Magnetic Resonance Imaging Artificial Intelligence Pipeline for Oropharyngeal Cancer Radiotherapy Treatment Guidance

Oropharyngeal cancer (OPC) is a widespread disease and one of the few domestic cancers that is rising in incidence. Radiographic images are crucial for assessment of OPC and aid in radiotherapy (RT) treatment. However, RT planning with conventional imaging approaches requires operator-dependent tumor segmentation, which is the primary source of treatment error. Further, OPC expresses differential tumor/node mid-RT response (rapid response) rates, resulting in significant differences between planned and delivered RT dose. Finally, clinical outcomes for OPC patients can also be variable, which warrants the investigation of prognostic models. Multiparametric MRI (mpMRI) techniques that incorporate simultaneous anatomical and functional information coupled to artificial intelligence (AI) approaches could improve clinical decision support for OPC by providing immediately actionable clinical rationale for adaptive RT planning. If tumors could be reproducibly segmented, rapid response could be classified, and prognosis could be reliably determined, overall patient outcomes would be optimized to improve the therapeutic index as a function of more risk-adapted RT volumes. Consequently, there is an unmet need for automated and reproducible imaging which can simultaneously segment tumors and provide predictive value for actionable RT adaptation. This dissertation primarily seeks to explore and optimize image processing, tumor segmentation, and patient outcomes in OPC through a combination of advanced imaging techniques and AI algorithms. In the first specific aim of this dissertation, we develop and evaluate mpMRI pre-processing techniques for use in downstream segmentation, response prediction, and outcome prediction pipelines. Various MRI intensity standardization and registration approaches were systematically compared and benchmarked. Moreover, synthetic image algorithms were developed to decrease MRI scan time in an effort to optimize our AI pipelines. We demonstrated that proper intensity standardization and image registration can improve mpMRI quality for use in AI algorithms, and developed a novel method to decrease mpMRI acquisition time. Subsequently, in the second specific aim of this dissertation, we investigated underlying questions regarding the implementation of RT-related auto-segmentation. Firstly, we quantified interobserver variability for an unprecedented large number of observers for various radiotherapy structures in several disease sites (with a particular emphasis on OPC) using a novel crowdsourcing platform. We then trained an AI algorithm on a series of extant matched mpMRI datasets to segment OPC primary tumors. Moreover, we validated and compared our best model\u27s performance to clinical expert observers. We demonstrated that AI-based mpMRI OPC tumor auto-segmentation offers decreased variability and comparable accuracy to clinical experts, and certain mpMRI input channel combinations could further improve performance. Finally, in the third specific aim of this dissertation, we predicted OPC primary tumor mid-therapy (rapid) treatment response and prognostic outcomes. Using co-registered pre-therapy and mid-therapy primary tumor manual segmentations of OPC patients, we generated and characterized treatment sensitive and treatment resistant pre-RT sub-volumes. These sub-volumes were used to train an AI algorithm to predict individual voxel-wise treatment resistance. Additionally, we developed an AI algorithm to predict OPC patient progression free survival using pre-therapy imaging from an international data science competition (ranking 1st place), and then translated these approaches to mpMRI data. We demonstrated AI models could be used to predict rapid response and prognostic outcomes using pre-therapy imaging, which could help guide treatment adaptation, though further work is needed. In summary, the completion of these aims facilitates the development of an image-guided fully automated OPC clinical decision support tool. The resultant deliverables from this project will positively impact patients by enabling optimized therapeutic interventions in OPC. Future work should consider investigating additional imaging timepoints, imaging modalities, uncertainty quantification, perceptual and ethical considerations, and prospective studies for eventual clinical implementation. A dynamic version of this dissertation is publicly available and assigned a digital object identifier through Figshare (doi: 10.6084/m9.figshare.22141871)