Towards development of automatic path planning system in image-guided neurosurgery

With the advent of advanced computer technology, many computer-aided systems have evolved to assist in medical related work including treatment, diagnosis, and even surgery. In modern neurosurgery, Magnetic Resonance Image guided stereotactic surgery exactly complies with this trend. It is a minimally invasive operation being much safer than the traditional open-skull surgery, and offers higher precision and more effective operating procedures compared to conventional craniotomy. However, such operations still face significant challenges of planning the optimal neurosurgical path in order to reach the ideal position without damage to important internal structures. This research aims to address this major challenge. The work begins with an investigation of the problem of distortion induced by MR images. It then goes on to build a template of the Circle of Wills brain vessels, realized from a collection of Magnetic Resonance Angiography images, which is needed to maintain operating standards when, as in many cases, Magnetic Resonance Angiography images are not available for patients. Demographic data of brain tumours are also studied to obtain further understanding of diseased human brains through the development of an effect classifier. The developed system allows the internal brain structure to be ‘seen’ clearly before the surgery, giving surgeons a clear picture and thereby makes a significant contribution to the eventual development of a fully automatic path planning system

Towards the development of safe, collaborative robotic freehand ultrasound

The use of robotics in medicine is of growing importance for modern health services, as robotic systems have the capacity to improve upon human tasks, thereby enhancing the treatment ability of a healthcare provider. In the medical sector, ultrasound imaging is an inexpensive approach without the high radiation emissions often associated with other modalities, especially when compared to MRI and CT imaging respectively. Over the past two decades, considerable effort has been invested into freehand ultrasound robotics research and development. However, this research has focused on the feasibility of the application, not the robotic fundamentals, such as motion control, calibration, and contextual awareness. Instead, much of the work is concentrated on custom designed robots, ultrasound image generation and visual servoing, or teleoperation. Research based on these topics often suffer from important limitations that impede their use in an adaptable, scalable, and real-world manner. Particularly, while custom robots may be designed for a specific application, commercial collaborative robots are a more robust and economical solution. Otherwise, various robotic ultrasound studies have shown the feasibility of using basic force control, but rarely explore controller tuning in the context of patient safety and deformable skin in an unstructured environment. Moreover, many studies evaluate novel visual servoing approaches, but do not consider the practicality of relying on external measurement devices for motion control. These studies neglect the importance of robot accuracy and calibration, which allow a system to safely navigate its environment while reducing the imaging errors associated with positioning. Hence, while the feasibility of robotic ultrasound has been the focal point in previous studies, there is a lack of attention to what occurs between system design and image output. This thesis addresses limitations of the current literature through three distinct contributions. Given the force-controlled nature of an ultrasound robot, the first contribution presents a closed-loop calibration approach using impedance control and low-cost equipment. Accuracy is a fundamental requirement for high-quality ultrasound image generation and targeting. This is especially true when following a specified path along a patient or synthesizing 2D slices into a 3D ultrasound image. However, even though most industrial robots are inherently precise, they are not necessarily accurate. While robot calibration itself has been extensively studied, many of the approaches rely on expensive and highly delicate equipment. Experimental testing showed that this method is comparable in quality to traditional calibration using a laser tracker. As demonstrated through an experimental study and validated with a laser tracker, the absolute accuracy of a collaborative robot was improved to a maximum error of 0.990mm, representing a 58.4% improvement when compared to the nominal model. The second contribution explores collisions and contact events, as they are a natural by-product of applications involving physical human-robot interaction (pHRI) in unstructured environments. Robot-assisted medical ultrasound is an example of a task where simply stopping the robot upon contact detection may not be an appropriate reaction strategy. Thus, the robot should have an awareness of body contact location to properly plan force-controlled trajectories along the human body using the imaging probe. This is especially true for remote ultrasound systems where safety and manipulability are important elements to consider when operating a remote medical system through a communication network. A framework is proposed for robot contact classification using the built-in sensor data of a collaborative robot. Unlike previous studies, this classification does not discern between intended vs. unintended contact scenarios, but rather classifies what was involved in the contact event. The classifier can discern different ISO/TS 15066:2016 specific body areas along a human-model leg with 89.37% accuracy. Altogether, this contact distinction framework allows for more complex reaction strategies and tailored robot behaviour during pHRI. Lastly, given that the success of an ultrasound task depends on the capability of the robot system to handle pHRI, pure motion control is insufficient. Force control techniques are necessary to achieve effective and adaptable behaviour of a robotic system in the unstructured ultrasound environment while also ensuring safe pHRI. While force control does not require explicit knowledge of the environment, to achieve an acceptable dynamic behaviour, the control parameters must be tuned. The third contribution proposes a simple and effective online tuning framework for force-based robotic freehand ultrasound motion control. Within the context of medical ultrasound, different human body locations have a different stiffness and will require unique tunings. Through real-world experiments with a collaborative robot, the framework tuned motion control for optimal and safe trajectories along a human leg phantom. The optimization process was able to successfully reduce the mean absolute error (MAE) of the motion contact force to 0.537N through the evolution of eight motion control parameters. Furthermore, contextual awareness through motion classification can offer a framework for pHRI optimization and safety through predictive motion behaviour with a future goal of autonomous pHRI. As such, a classification pipeline, trained using the tuning process motion data, was able to reliably classify the future force tracking quality of a motion session with an accuracy of 91.82 %

Optimization and validation of a new 3D-US imaging robot to detect, localize and quantify lower limb arterial stenoses

L’athérosclérose est une maladie qui cause, par l’accumulation de plaques lipidiques, le durcissement de la paroi des artères et le rétrécissement de la lumière. Ces lésions sont généralement localisées sur les segments artériels coronariens, carotidiens, aortiques, rénaux, digestifs et périphériques. En ce qui concerne l’atteinte périphérique, celle des membres inférieurs est particulièrement fréquente. En effet, la sévérité de ces lésions artérielles est souvent évaluée par le degré d’une sténose (réduction >50 % du diamètre de la lumière) en angiographie, imagerie par résonnance magnétique (IRM), tomodensitométrie ou échographie. Cependant, pour planifier une intervention chirurgicale, une représentation géométrique artérielle 3D est notamment préférable. Les méthodes d’imagerie par coupe (IRM et tomodensitométrie) sont très performantes pour générer une imagerie tridimensionnelle de bonne qualité mais leurs utilisations sont dispendieuses et invasives pour les patients. L’échographie 3D peut constituer une avenue très prometteuse en imagerie pour la localisation et la quantification des sténoses. Cette modalité d’imagerie offre des avantages distincts tels la commodité, des coûts peu élevés pour un diagnostic non invasif (sans irradiation ni agent de contraste néphrotoxique) et aussi l’option d’analyse en Doppler pour quantifier le flux sanguin. Étant donné que les robots médicaux ont déjà été utilisés avec succès en chirurgie et en orthopédie, notre équipe a conçu un nouveau système robotique d’échographie 3D pour détecter et quantifier les sténoses des membres inférieurs. Avec cette nouvelle technologie, un radiologue fait l’apprentissage manuel au robot d’un balayage échographique du vaisseau concerné. Par la suite, le robot répète à très haute précision la trajectoire apprise, contrôle simultanément le processus d’acquisition d’images échographiques à un pas d’échantillonnage constant et conserve de façon sécuritaire la force appliquée par la sonde sur la peau du patient. Par conséquent, la reconstruction d’une géométrie artérielle 3D des membres inférieurs à partir de ce système pourrait permettre une localisation et une quantification des sténoses à très grande fiabilité. L’objectif de ce projet de recherche consistait donc à valider et optimiser ce système robotisé d’imagerie échographique 3D. La fiabilité d’une géométrie reconstruite en 3D à partir d’un système référentiel robotique dépend beaucoup de la précision du positionnement et de la procédure de calibration. De ce fait, la précision pour le positionnement du bras robotique fut évaluée à travers son espace de travail avec un fantôme spécialement conçu pour simuler la configuration des artères des membres inférieurs (article 1 - chapitre 3). De plus, un fantôme de fils croisés en forme de Z a été conçu pour assurer une calibration précise du système robotique (article 2 - chapitre 4). Ces méthodes optimales ont été utilisées pour valider le système pour l’application clinique et trouver la transformation qui convertit les coordonnées de l’image échographique 2D dans le référentiel cartésien du bras robotisé. À partir de ces résultats, tout objet balayé par le système robotique peut être caractérisé pour une reconstruction 3D adéquate. Des fantômes vasculaires compatibles avec plusieurs modalités d’imagerie ont été utilisés pour simuler différentes représentations artérielles des membres inférieurs (article 2 - chapitre 4, article 3 - chapitre 5). La validation des géométries reconstruites a été effectuée à l`aide d`analyses comparatives. La précision pour localiser et quantifier les sténoses avec ce système robotisé d’imagerie échographique 3D a aussi été déterminée. Ces évaluations ont été réalisées in vivo pour percevoir le potentiel de l’utilisation d’un tel système en clinique (article 3- chapitre 5).Atherosclerosis is a disease caused by the accumulation of lipid deposits inducing the remodeling and hardening of the vessel wall, which leads to a progressive narrowing of arteries. These lesions are generally located on the coronary, carotid, aortic, renal, digestive and peripheral arteries. With regards to peripheral vessels, lower limb arteries are frequently affected. The severity of arterial lesions are evaluated by the stenosis degree (reduction > 50.0 % of the lumen diameter) using angiography, magnetic resonance angiography (MRA), computed tomography (CT) and ultrasound (US). However, to plan a surgical therapeutic intervention, a 3D arterial geometric representation is notably preferable. Imaging methods such as MRA and CT are very efficient to generate a three-dimensional imaging of good quality even though their use is expensive and invasive for patients. 3D-ultrasound can be perceived as a promising avenue in imaging for the location and the quantification of stenoses. This non invasive, non allergic (i.e, nephrotoxic contrast agent) and non-radioactive imaging modality offers distinct advantages in convenience, low cost and also multiple diagnostic options to quantify blood flow in Doppler. Since medical robots already have been used with success in surgery and orthopedics, our team has conceived a new medical 3D-US robotic imaging system to localize and quantify arterial stenoses in lower limb vessels. With this new technology, a clinician manually teaches the robotic arm the scanning path. Then, the robotic arm repeats with high precision the taught trajectory and controls simultaneously the ultrasound image acquisition process at even sampling and preserves safely the force applied by the US probe. Consequently, the reconstruction of a lower limb arterial geometry in 3D with this system could allow the location and quantification of stenoses with high accuracy. The objective of this research project consisted in validating and optimizing this 3D-ultrasound imaging robotic system. The reliability of a 3D reconstructed geometry obtained with 2D-US images captured with a robotic system depends considerably on the positioning accuracy and the calibration procedure. Thus, the positioning accuracy of the robotic arm was evaluated in the workspace with a lower limb-mimicking phantom design (article 1 - chapter 3). In addition, a Z-phantom was designed to assure a precise calibration of the robotic system. These optimal methods were used to validate the system for the clinical application and to find the transformation which converts image coordinates of a 2D-ultrasound image into the robotic arm referential. From these results, all objects scanned by the robotic system can be adequately reconstructed in 3D. Multimodal imaging vascular phantoms of lower limb arteries were used to evaluate the accuracy of the 3D representations (article 2 - chapter 4, article 3 - chapter 5). The validation of the reconstructed geometry with this system was performed by comparing surface points with the manufacturing vascular phantom file surface points. The accuracy to localize and quantify stenoses with the 3D-ultrasound robotic imaging system was also determined. These same evaluations were analyzed in vivo to perceive the feasibility of the study

CO-ROBOTIC ULTRASOUND IMAGING: A COOPERATIVE FORCE CONTROL APPROACH

Ultrasound (US) imaging remains one of the most commonly used imaging modalities in medical practice due to its low cost and safety. However, 63-91% of ultrasonographers develop musculoskeletal disorders due to the effort required to perform imaging tasks. Robotic ultrasound (RUS), the application of robotic systems to assist ultrasonographers in ultrasound scanning procedures, has been proposed in literature and recently deployed in clinical settings using limited degree-of-freedom (DOF) systems. An example of this includes breast-scanning systems, which allow one-DOF translation of a large ultrasound array in order to capture patients’ breast scans and minimize sonographer effort while preserving a desired clinical outcome. Recently, the robotic industry has evolved to provide light-weight, compact, accurate, and cost-effective manipulators. We leverage this new reality in able to provide ultrasonographers with a full 6-DOF system that provides force assistance to facilitate US image acquisition. Admittance robot control allows for smooth human-machine interaction in a desired task. In the case of RUS, force control is capable of assisting sonographers in facilitating and even improving the imaging results of typical procedures. We propose a new system setup for collaborative force control in US applications. This setup consists of the 6-DOF UR5 industrial robot, and a 6-axes force sensor attached to the robot tooltip, which in turn has an US probe attached to it through a custom-designed probe attachment mechanism. Additionally, an independent one-axis load cell is placed inside this attachment device and used to measure the contact force between the probe and the patient’s anatomy in real time and independent of any other forces. As the sonographer guides the US probe, the robot collaborates with the hand motions, following the path of the user. When imaging, the robot can offer assistance to the sonographer by augmenting the forces applied by him or her, thereby lessening the physical effort required as well as the resulting strain. Additional benefits include force and velocity limiting for patient safety and robot motion constraints for particular imaging tasks. Initial results of a conducted user study show the feasibility of implementing the presented robot-assisted system in a clinical setting

Intraoperative Fourier domain optical coherence tomography for microsurgery guidance and assessment

In this dissertation, advanced high-speed Fourier domain optical coherence tomography (FD-OCT)systems were investigated and developed. Several real-time, high resolution functional Spectral-domain OCT (SD-OCT) systems capable of imaging and sensing blood flow and motion were designed and developed. The system were designed particularly for microsurgery guidance and assessment. The systems were tested for their ability to assessing microvascular anastomosis and vulnerable plaque development. An all fiber-optic common-path optical coherence tomography (CP-OCT) system capable of measuring high-resolution optical distances, was built and integrated into di fferent imaging modalities. First, a novel non-contact accurate in-vitro intra-ocular lens power measurement method was proposed and validated based on CP-OCT. Second, CP-OCT was integrated with a ber bundle based confocal microscope to achieve motion-compensated imaging. Distance between the probe and imaged target was monitored by the CP-OCT system in real-time.The distance signal from the CP-OCT system was routed to a high speed, high resolution linear motor to compensate for the axial motion of the sample in a closed-loop control. Finally a motion-compensated hand-held common-path Fourier domain optical coherence tomography probe was developed for image-guided intervention. Both phantom and ex vivo models were used to test and evaluate the probe. As the data acquisition speed of current OCT systems continue to increase, the means to process the data in real-time are in critically needed. Previous graphics processing unit accelerated OCT signal processing methods have shown their potential to achieve real-time imaging. In this dissertation, algorithms to perform real-time reference A-line subtraction and saturation artifact removal were proposed, realized and integrated into previously developed FD-OCT system CPU-GPU heterogeneous structure. Fourier domain phase resolved Doppler OCT (PRDOCT) system capable of real-time simultaneous structure and flow imaging based on dual GPUs was also developed and implemented. Finally, systematic experiments were conducted to validate the system for surgical applications. FD-OCT system was used to detect atherosclerotic plaque and drug effi ciency test in mouse model. Application of PRDOCT for both suture and cu ff based microvascular anastomosis guidance and assessment was extensively stuided in rodent model