Medical image computing and computer-aided medical interventions applied to soft tissues. Work in progress in urology

Until recently, Computer-Aided Medical Interventions (CAMI) and Medical Robotics have focused on rigid and non deformable anatomical structures. Nowadays, special attention is paid to soft tissues, raising complex issues due to their mobility and deformation. Mini-invasive digestive surgery was probably one of the first fields where soft tissues were handled through the development of simulators, tracking of anatomical structures and specific assistance robots. However, other clinical domains, for instance urology, are concerned. Indeed, laparoscopic surgery, new tumour destruction techniques (e.g. HIFU, radiofrequency, or cryoablation), increasingly early detection of cancer, and use of interventional and diagnostic imaging modalities, recently opened new challenges to the urologist and scientists involved in CAMI. This resulted in the last five years in a very significant increase of research and developments of computer-aided urology systems. In this paper, we propose a description of the main problems related to computer-aided diagnostic and therapy of soft tissues and give a survey of the different types of assistance offered to the urologist: robotization, image fusion, surgical navigation. Both research projects and operational industrial systems are discussed

Robot Autonomy for Surgery

Autonomous surgery involves having surgical tasks performed by a robot operating under its own will, with partial or no human involvement. There are several important advantages of automation in surgery, which include increasing precision of care due to sub-millimeter robot control, real-time utilization of biosignals for interventional care, improvements to surgical efficiency and execution, and computer-aided guidance under various medical imaging and sensing modalities. While these methods may displace some tasks of surgical teams and individual surgeons, they also present new capabilities in interventions that are too difficult or go beyond the skills of a human. In this chapter, we provide an overview of robot autonomy in commercial use and in research, and present some of the challenges faced in developing autonomous surgical robots

Anatomical variability, multi-modal coordinate systems, and precision targeting in the marmoset brain

Localising accurate brain regions needs careful evaluation in each experimental species due to their individual variability. However, the function and connectivity of brain areas is commonly studied using a single-subject cranial landmark-based stereotactic atlas in animal neuroscience. Here, we address this issue in a small primate, the common marmoset, which is increasingly widely used in systems neuroscience. We developed a non-invasive multi-modal neuroimaging-based targeting pipeline, which accounts for intersubject anatomical variability in cranial and cortical landmarks in marmosets. This methodology allowed creation of multi-modal templates (MarmosetRIKEN20) including head CT and brain MR images, embedded in coordinate systems of anterior and posterior commissures (AC-PC) and CIFTI grayordinates. We found that the horizontal plane of the stereotactic coordinate was significantly rotated in pitch relative to the AC-PC coordinate system (10 degrees, frontal downwards), and had a significant bias and uncertainty due to positioning procedures. We also found that many common cranial and brain landmarks (e.g., bregma, intraparietal sulcus) vary in location across subjects and are substantial relative to average marmoset cortical area dimensions. Combining the neuroimaging-based targeting pipeline with robot-guided surgery enabled proof-of-concept targeting of deep brain structures with an accuracy of 0.2 mm. Altogether, our findings demonstrate substantial intersubject variability in marmoset brain and cranial landmarks, implying that subject-specific neuroimaging-based localization is needed for precision targeting in marmosets. The population-based templates and atlases in grayordinates, created for the first time in marmoset monkeys, should help bridging between macroscale and microscale analyses

Evaluating Human Performance for Image-Guided Surgical Tasks

The following work focuses on the objective evaluation of human performance for two different interventional tasks; targeted prostate biopsy tasks using a tracked biopsy device, and external ventricular drain placement tasks using a mobile-based augmented reality device for visualization and guidance. In both tasks, a human performance methodology was utilized which respects the trade-off between speed and accuracy for users conducting a series of targeting tasks using each device. This work outlines the development and application of performance evaluation methods using these devices, as well as details regarding the implementation of the mobile AR application. It was determined that the Fitts’ Law methodology can be applied for evaluation of tasks performed in each surgical scenario, and was sensitive to differentiate performance across a range which spanned experienced and novice users. This methodology is valuable for future development of training modules for these and other medical devices, and can provide details about the underlying characteristics of the devices, and how they can be optimized with respect to human performance

On uncertainty propagation in image-guided renal navigation: Exploring uncertainty reduction techniques through simulation and in vitro phantom evaluation

Image-guided interventions (IGIs) entail the use of imaging to augment or replace direct vision during therapeutic interventions, with the overall goal is to provide effective treatment in a less invasive manner, as an alternative to traditional open surgery, while reducing patient trauma and shortening the recovery time post-procedure. IGIs rely on pre-operative images, surgical tracking and localization systems, and intra-operative images to provide correct views of the surgical scene. Pre-operative images are used to generate patient-specific anatomical models that are then registered to the patient using the surgical tracking system, and often complemented with real-time, intra-operative images. IGI systems are subject to uncertainty from several sources, including surgical instrument tracking / localization uncertainty, model-to-patient registration uncertainty, user-induced navigation uncertainty, as well as the uncertainty associated with the calibration of various surgical instruments and intra-operative imaging devices (i.e., laparoscopic camera) instrumented with surgical tracking sensors. All these uncertainties impact the overall targeting accuracy, which represents the error associated with the navigation of a surgical instrument to a specific target to be treated under image guidance provided by the IGI system. Therefore, understanding the overall uncertainty of an IGI system is paramount to the overall outcome of the intervention, as procedure success entails achieving certain accuracy tolerances specific to individual procedures. This work has focused on studying the navigation uncertainty, along with techniques to reduce uncertainty, for an IGI platform dedicated to image-guided renal interventions. We constructed life-size replica patient-specific kidney models from pre-operative images using 3D printing and tissue emulating materials and conducted experiments to characterize the uncertainty of both optical and electromagnetic surgical tracking systems, the uncertainty associated with the virtual model-to-physical phantom registration, as well as the uncertainty associated with live augmented reality (AR) views of the surgical scene achieved by enhancing the pre-procedural model and tracked surgical instrument views with live video views acquires using a camera tracked in real time. To better understand the effects of the tracked instrument calibration, registration fiducial configuration, and tracked camera calibration on the overall navigation uncertainty, we conducted Monte Carlo simulations that enabled us to identify optimal configurations that were subsequently validated experimentally using patient-specific phantoms in the laboratory. To mitigate the inherent accuracy limitations associated with the pre-procedural model-to-patient registration and their effect on the overall navigation, we also demonstrated the use of tracked video imaging to update the registration, enabling us to restore targeting accuracy to within its acceptable range. Lastly, we conducted several validation experiments using patient-specific kidney emulating phantoms using post-procedure CT imaging as reference ground truth to assess the accuracy of AR-guided navigation in the context of in vitro renal interventions. This work helped find answers to key questions about uncertainty propagation in image-guided renal interventions and led to the development of key techniques and tools to help reduce optimize the overall navigation / targeting uncertainty

New Mechatronic Systems for the Diagnosis and Treatment of Cancer

Both two dimensional (2D) and three dimensional (3D) imaging modalities are useful tools for viewing the internal anatomy. Three dimensional imaging techniques are required for accurate targeting of needles. This improves the efficiency and control over the intervention as the high temporal resolution of medical images can be used to validate the location of needle and target in real time. Relying on imaging alone, however, means the intervention is still operator dependent because of the difficulty of controlling the location of the needle within the image. The objective of this thesis is to improve the accuracy and repeatability of needle-based interventions over conventional techniques: both manual and automated techniques. This includes increasing the accuracy and repeatability of these procedures in order to minimize the invasiveness of the procedure. In this thesis, I propose that by combining the remote center of motion concept using spherical linkage components into a passive or semi-automated device, the physician will have a useful tracking and guidance system at their disposal in a package, which is less threatening than a robot to both the patient and physician. This design concept offers both the manipulative transparency of a freehand system, and tremor reduction through scaling currently offered in automated systems. In addressing each objective of this thesis, a number of novel mechanical designs incorporating an remote center of motion architecture with varying degrees of freedom have been presented. Each of these designs can be deployed in a variety of imaging modalities and clinical applications, ranging from preclinical to human interventions, with an accuracy of control in the millimeter to sub-millimeter range

Liver Segmentation and its Application to Hepatic Interventions

The thesis addresses the development of an intuitive and accurate liver segmentation approach, its integration into software prototypes for the planning of liver interventions, and research on liver regeneration. The developed liver segmentation approach is based on a combination of the live wire paradigm and shape-based interpolation. Extended with two correction modes and integrated into a user-friendly workflow, the method has been applied to more than 5000 data sets. The combination of the liver segmentation with image analysis of hepatic vessels and tumors allows for the computation of anatomical and functional remnant liver volumes. In several projects with clinical partners world-wide, the benefit of the computer-assisted planning was shown. New insights about the postoperative liver function and regeneration could be gained, and most recent investigations into the analysis of MRI data provide the option to further improve hepatic intervention planning

Complexity Reduction in Image-Based Breast Cancer Care

The diversity of malignancies of the breast requires personalized diagnostic and therapeutic decision making in a complex situation. This thesis contributes in three clinical areas: (1) For clinical diagnostic image evaluation, computer-aided detection and diagnosis of mass and non-mass lesions in breast MRI is developed. 4D texture features characterize mass lesions. For non-mass lesions, a combined detection/characterisation method utilizes the bilateral symmetry of the breast s contrast agent uptake. (2) To improve clinical workflows, a breast MRI reading paradigm is proposed, exemplified by a breast MRI reading workstation prototype. Instead of mouse and keyboard, it is operated using multi-touch gestures. The concept is extended to mammography screening, introducing efficient navigation aids. (3) Contributions to finite element modeling of breast tissue deformations tackle two clinical problems: surgery planning and the prediction of the breast deformation in a MRI biopsy device

Intra-Operative Needle Tracking Using Optical Shape Sensing Technology

RÉSUMÉ Contexte : Les métastases hépatiques colorectales sont la principale cause de décès liée au cancer du foie dans le monde. Au cours de la dernière décennie, il a été démontré que l’ablation par radiofréquence (RFA, pour radiofrequency ablation) est une méthode de traitement percutané très efficace contre ce type de métastases. Cela dit, un positionnement précis de l’embout de l’aiguille utilisé en RFA est essentiel afin de se départir adéquatement de la totalité des cellules cancéreuses. Une technologie prometteuse pour obtenir la forme et la position de l’aiguille en temps réel est basée sur l’utilisation de réseaux de Bragg (FBG, pour fiber Bragg grating) à titre de senseur de contrainte. En effet, ce type de senseurs a une vitesse d’acquisition allant jusqu’à 20 kHz, ce qui est suffisamment rapide pour permettre des applications de guidage en temps réel. Méthode : Les travaux présentés au sein de ce mémoire décrivent le développement d’une technologie, compatible aux systèmes d’imageries par résonance magnétique (IRM), permettant d’effectuer le suivi de la forme de l’aiguille utilisée en RFA. Premièrement, trois fibres contenant une série de réseaux de Bragg ont été collées dans une géométrie spécifique et intégrées à l’intérieur d’une aiguille 20G-150 mm. Ensuite, un algorithme de reconstruction de forme tridimensionnelle a été développé, basé sur les mesures de translation spectrales des FBGs acquises en temps réel durant le guidage de l’aiguille. La position du bout de l’aiguille ainsi que la forme tridimensionnelle complète de celle-ci ont été représentées et comparées à la position de la zone ciblée à la suite d’une simple méthode de calibration. Finalement, nous avons validé notre système de navigation en effectuant une série d’expériences in vitro. La précision du système de reconstruction tridimensionnelle de la forme et de l’orientation de l’aiguille a été évaluée en utilisant deux caméras positionnées perpendiculairement de manière à connaitre la position de l’aiguille dans le système d’axes du laboratoire. L’évaluation de la précision au bout de l’aiguille a quant à elle été faite en utilisant des fantômes précisément conçus à cet effet. Finalement, des interventions guidées en IRM ont été testées et comparées au système de navigation électromagnétique NDI Aurora (EMTS, pour Electromagnétic tracking system) par le biais du FRE (fiducial registration error) et du TRE (target registration error). Résultats: Lors de nos premières expériences in vitro, la précision obtenue quant à la position du bout de l’aiguille était de 0,96 mm pour une déflexion allant jusqu’à ±10,68 mm. À titre comparatif, le système d’Aurora a une précision de 0.84 mm dans des circonstances similaires. Les résultats obtenus lors de nos seconds tests ont démontré que l’erreur entre la position réelle du bout de l’aiguille et la position fournie par notre système de reconstruction de forme est de 1,04 mm, alors qu’elle est de 0,82 mm pour le EMTS d’Aurora. Pour ce qui est de notre dispositif, cette erreur est proportionnelle à l’amplitude de déflexion de l’aiguille, contrairement à l’EMTS pour qui l’erreur demeure relativement constante. La dernière expérience a été effectuée à l’aide d’un fantôme en gélatine, pour laquelle nous avons obtenu un TRE de 1,19 mm pour notre système basé sur les FBG et de 1.06 mm pour le système de navigation par senseurs électromagnétiques (EMTS). Les résultats démontrent que l’évaluation du FRE est similaire pour les deux approches. De plus, l’information fournie par les caméras permet d’estimer la précision de notre dispositif en tout point le long de l’aiguille. Conclusion : En analysant et en interprétant les résultats obtenus lors de nos expériences in vitro, nous pouvons conclure que la précision de notre système de navigation basé sur les FBG est bien adaptée pour l’évaluation de la position du bout et la forme de l’aiguille lors d’interventions RFA des tumeurs du foie. La précision de notre système de navigation est fortement comparable avec celle du système basé sur des senseurs électromagnétiques commercialisé par Aurora. L’erreur obtenue par notre système est attribuable à un mauvais alignement des réseaux de Bragg par rapport au plan associé à la région sensorielle et aussi à la différence entre le diamètre des fibres et celui de la paroi interne de l’aiguille.----------ABSTRACT Background: Colorectal liver metastasis is the leading cause of liver cancer death in the world. In the past decade, radiofrequency ablation (RFA) has proven to be an effective percutaneous treatment modality for the treatment of metastatic hepatic cancer. Accurate needle tip placement is essential for RFA of liver tumors. A promising technology to obtain the real-time information of the shape of the needle is by using fiber Bragg grating (FBG) sensors at high frequencies (up to 20 kHz). Methods: In this thesis work, we developed an MR-compatible needle tracking technology designed for RFA procedures in liver cancer. At first, three fibers each containing a series of FBGs were glued together and integrated inside a 20G-150 mm needle. Then a three-dimensional needle shape reconstruction algorithm was developed, based on the FBG measurements collected in real-time during needle guidance. The tip position and shape of the reconstructed 3D needle model were represented with respect to the target defined in the image space by performing a fiducial-based registration. Finally, we validated our FBG-based needle navigation by doing a series of in-vitro experiments. The shape of the 3D reconstructed needle was compared to measurements obtained from camera images. In addition, the needle tip accuracy was assessed on the ground-truth phantoms. Finally, MRI guided intervention was tested and compared to an NDI Aurora EM tracking system (EMTS) in terms of fiducial registration error (FRE) and target registration error (TRE). Results: In our first in-vitro experiment, the tip tracking accuracy of our FBG tracking system was of 0.96 mm for the maximum tip deflection of up to ±10.68 mm, while the tip tracking accuracy of the Aurora system for the similar test was 0.84 mm. Results obtained from the second in-vitro experiment demonstrated tip tracking accuracy of 1.04 mm and 0.82 mm for our FBG tracking system and Aurora EMTS, respectively for the maximum tip deflection of up to ±16.83 mm. The tip tracking error in the developed FBG-based system reduced linearly with decreasing tip deflection, while the error was similar but randomly varying for the EMTS. The last experiment was done with a gel phantom, yielding a TRE of 1.19 mm and 1.06 mm for the FBG and EM tracking, respectively. Results showed that across all experiments, the computed FRE of both tracking systems was similar. Moreover, actual shape information obtained from the camera images ensured the shape accuracy of our FBG-based needle shape model. Conclusion: By analyzing and interpreting the results obtained from the in-vitro experiments, we conclude that the accuracy of our FBG-based tracking system is suitable for needle tip detection in RFA of liver tumors. The accuracy of our tracking system is nearly comparable to that of the Aurora EMTS. The error given by our tracking system is attributed to the misalignment of the FBG sensors in a single axial plane and also to the gap between the needle's inner wall and the fibers inside