740 research outputs found

    Neurosurgical Ultrasound Pose Estimation Using Image-Based Registration and Sensor Fusion - A Feasibility Study

    Get PDF
    Modern neurosurgical procedures often rely on computer-assisted real-time guidance using multiple medical imaging modalities. State-of-the-art commercial products enable the fusion of pre-operative with intra-operative images (e.g., magnetic resonance [MR] with ultrasound [US] images), as well as the on-screen visualization of procedures in progress. In so doing, US images can be employed as a template to which pre-operative images can be registered, to correct for anatomical changes, to provide live-image feedback, and consequently to improve confidence when making resection margin decisions near eloquent regions during tumour surgery. In spite of the potential for tracked ultrasound to improve many neurosurgical procedures, it is not widely used. State-of-the-art systems are handicapped by optical tracking’s need for consistent line-of-sight, keeping tracked rigid bodies clean and rigidly fixed, and requiring a calibration workflow. The goal of this work is to improve the value offered by co-registered ultrasound images without the workflow drawbacks of conventional systems. The novel work in this thesis includes: the exploration and development of a GPU-enabled 2D-3D multi-modal registration algorithm based on the existing LC2 metric; and the use of this registration algorithm in the context of a sensor and image-fusion algorithm. The work presented here is a motivating step in a vision towards a heterogeneous tracking framework for image-guided interventions where the knowledge from intraoperative imaging, pre-operative imaging, and (potentially disjoint) wireless sensors in the surgical field are seamlessly integrated for the benefit of the surgeon. The technology described in this thesis, inspired by advances in robot localization demonstrate how inaccurate pose data from disjoint sources can produce a localization system greater than the sum of its parts

    Fast catheter segmentation and tracking based on x-ray fluoroscopic and echocardiographic modalities for catheter-based cardiac minimally invasive interventions

    Get PDF
    X-ray fluoroscopy and echocardiography imaging (ultrasound, US) are two imaging modalities that are widely used in cardiac catheterization. For these modalities, a fast, accurate and stable algorithm for the detection and tracking of catheters is required to allow clinicians to observe the catheter location in real-time. Currently X-ray fluoroscopy is routinely used as the standard modality in catheter ablation interventions. However, it lacks the ability to visualize soft tissue and uses harmful radiation. US does not have these limitations but often contains acoustic artifacts and has a small field of view. These make the detection and tracking of the catheter in US very challenging. The first contribution in this thesis is a framework which combines Kalman filter and discrete optimization for multiple catheter segmentation and tracking in X-ray images. Kalman filter is used to identify the whole catheter from a single point detected on the catheter in the first frame of a sequence of x-ray images. An energy-based formulation is developed that can be used to track the catheters in the following frames. We also propose a discrete optimization for minimizing the energy function in each frame of the X-ray image sequence. Our approach is robust to tangential motion of the catheter and combines the tubular and salient feature measurements into a single robust and efficient framework. The second contribution is an algorithm for catheter extraction in 3D ultrasound images based on (a) the registration between the X-ray and ultrasound images and (b) the segmentation of the catheter in X-ray images. The search space for the catheter extraction in the ultrasound images is constrained to lie on or close to a curved surface in the ultrasound volume. The curved surface corresponds to the back-projection of the extracted catheter from the X-ray image to the ultrasound volume. Blob-like features are detected in the US images and organized in a graphical model. The extracted catheter is modelled as the optimal path in this graphical model. Both contributions allow the use of ultrasound imaging for the improved visualization of soft tissue. However, X-ray imaging is still required for each ultrasound frame and the amount of X-ray exposure has not been reduced. The final contribution in this thesis is a system that can track the catheter in ultrasound volumes automatically without the need for X-ray imaging during the tracking. Instead X-ray imaging is only required for the system initialization and for recovery from tracking failures. This allows a significant reduction in the amount of X-ray exposure for patient and clinicians.Open Acces

    A surgical system for automatic registration, stiffness mapping and dynamic image overlay

    Full text link
    In this paper we develop a surgical system using the da Vinci research kit (dVRK) that is capable of autonomously searching for tumors and dynamically displaying the tumor location using augmented reality. Such a system has the potential to quickly reveal the location and shape of tumors and visually overlay that information to reduce the cognitive overload of the surgeon. We believe that our approach is one of the first to incorporate state-of-the-art methods in registration, force sensing and tumor localization into a unified surgical system. First, the preoperative model is registered to the intra-operative scene using a Bingham distribution-based filtering approach. An active level set estimation is then used to find the location and the shape of the tumors. We use a recently developed miniature force sensor to perform the palpation. The estimated stiffness map is then dynamically overlaid onto the registered preoperative model of the organ. We demonstrate the efficacy of our system by performing experiments on phantom prostate models with embedded stiff inclusions.Comment: International Symposium on Medical Robotics (ISMR 2018

    Mechanical Characterization of the Vessel Wall by Data Assimilation of Intravascular Ultrasound Studies

    Get PDF
    Atherosclerotic plaque rupture and erosion are the most important mechanisms underlying the sudden plaque growth, responsible for acute coronary syndromes and even fatal cardiac events. Advances in the understanding of the culprit plaque structure and composition are already reported in the literature, however, there is still much work to be done toward in-vivo plaque visualization and mechanical characterization to assess plaque stability, patient risk, diagnosis and treatment prognosis. In this work, a methodology for the mechanical characterization of the vessel wall plaque and tissues is proposed based on the combination of intravascular ultrasound (IVUS) imaging processing, data assimilation and continuum mechanics models within a high performance computing (HPC) environment. Initially, the IVUS study is gated to obtain volumes of image sequences corresponding to the vessel of interest at different cardiac phases. These sequences are registered against the sequence of the end-diastolic phase to remove transversal and longitudinal rigid motions prescribed by the moving environment due to the heartbeat. Then, optical flow between the image sequences is computed to obtain the displacement fields of the vessel (each associated to a certain pressure level). The obtained displacement fields are regarded as observations within a data assimilation paradigm, which aims to estimate the material parameters of the tissues within the vessel wall. Specifically, a reduced order unscented Kalman filter is employed, endowed with a forward operator which amounts to address the solution of a hyperelastic solid mechanics model in the finite strain regime taking into account the axially stretched state of the vessel, as well as the effect of internal and external forces acting on the arterial wall. Due to the computational burden, a HPC approach is mandatory. Hence, the data assimilation and computational solid mechanics computations are parallelized at three levels: (i) a Kalman filter level; (ii) a cardiac phase level; and (iii) a mesh partitioning level. To illustrate the capabilities of this novel methodology toward the in-vivo analysis of patient-specific vessel constituents, mechanical material parameters are estimated using in-silico and in-vivo data retrieved from IVUS studies. Limitations and potentials of this approach are exposed and discussed.Fil: Maso Talou, Gonzalo Daniel. Laboratorio Nacional de Computacao Cientifica; BrasilFil: Blanco, Pablo Javier. Laboratorio Nacional de Computacao Cientifica; BrasilFil: Ares, Gonzalo Damián. Universidad Nacional de Mar del Plata. Facultad de Ingeniería. Departamento de Mecanica. Grupo de Ingeniería Asistida Por Computador; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Mar del Plata; ArgentinaFil: Guedes Bezerra, Cristiano. Heart Institute (Incor); BrasilFil: Lemos, Pedro A.. Heart Institute (Incor); BrasilFil: Feijóo, Raúl Antonino. Laboratorio Nacional de Computacao Cientifica; Brasi

    Exploiting Temporal Image Information in Minimally Invasive Surgery

    Get PDF
    Minimally invasive procedures rely on medical imaging instead of the surgeons direct vision. While preoperative images can be used for surgical planning and navigation, once the surgeon arrives at the target site real-time intraoperative imaging is needed. However, acquiring and interpreting these images can be challenging and much of the rich temporal information present in these images is not visible. The goal of this thesis is to improve image guidance for minimally invasive surgery in two main areas. First, by showing how high-quality ultrasound video can be obtained by integrating an ultrasound transducer directly into delivery devices for beating heart valve surgery. Secondly, by extracting hidden temporal information through video processing methods to help the surgeon localize important anatomical structures. Prototypes of delivery tools, with integrated ultrasound imaging, were developed for both transcatheter aortic valve implantation and mitral valve repair. These tools provided an on-site view that shows the tool-tissue interactions during valve repair. Additionally, augmented reality environments were used to add more anatomical context that aids in navigation and in interpreting the on-site video. Other procedures can be improved by extracting hidden temporal information from the intraoperative video. In ultrasound guided epidural injections, dural pulsation provides a cue in finding a clear trajectory to the epidural space. By processing the video using extended Kalman filtering, subtle pulsations were automatically detected and visualized in real-time. A statistical framework for analyzing periodicity was developed based on dynamic linear modelling. In addition to detecting dural pulsation in lumbar spine ultrasound, this approach was used to image tissue perfusion in natural video and generate ventilation maps from free-breathing magnetic resonance imaging. A second statistical method, based on spectral analysis of pixel intensity values, allowed blood flow to be detected directly from high-frequency B-mode ultrasound video. Finally, pulsatile cues in endoscopic video were enhanced through Eulerian video magnification to help localize critical vasculature. This approach shows particular promise in identifying the basilar artery in endoscopic third ventriculostomy and the prostatic artery in nerve-sparing prostatectomy. A real-time implementation was developed which processed full-resolution stereoscopic video on the da Vinci Surgical System

    Development of a cognitive robotic system for simple surgical tasks

    Get PDF
    The introduction of robotic surgery within the operating rooms has significantly improved the quality of many surgical procedures. Recently, the research on medical robotic systems focused on increasing the level of autonomy in order to give them the possibility to carry out simple surgical actions autonomously. This paper reports on the development of technologies for introducing automation within the surgical workflow. The results have been obtained during the ongoing FP7 European funded project Intelligent Surgical Robotics (I-SUR). The main goal of the project is to demonstrate that autonomous robotic surgical systems can carry out simple surgical tasks effectively and without major intervention by surgeons. To fulfil this goal, we have developed innovative solutions (both in terms of technologies and algorithms) for the following aspects: fabrication of soft organ models starting from CT images, surgical planning and execution of movement of robot arms in contact with a deformable environment, designing a surgical interface minimizing the cognitive load of the surgeon supervising the actions, intra-operative sensing and reasoning to detect normal transitions and unexpected events. All these technologies have been integrated using a component-based software architecture to control a novel robot designed to perform the surgical actions under study. In this work we provide an overview of our system and report on preliminary results of the automatic execution of needle insertion for the cryoablation of kidney tumours

    Over time RF fitting for Jitter Free 3D Vertebra Reconstruction from Video Fluoroscopy

    Get PDF
    Over the past decades, there has been an increasing interest in spine kinematics. Various approaches have been proposed on how to observe and analyse spine kinematics from a computer vision perspective. Amongst all, emphasis has been given to both the shape of the individual vertebrae as well as the overall spine curvature as a means of providing accurate and valid spinal condition diagnosis. Traditional invasive methods cannot accurately delineate the intersegmental motion of the spine vertebrae. On the contrary, capturing and measuring spinal motion via the non-invasive fluoroscopy has been a popular technique choice because of its low incurred patient radiation exposure nature. In general, image-based and other reconstruction methods target individual frames and focus on static spine instances. However, even the ones analysing sequences yield in unstable and jittery animations of the reconstructed spine. In this report, we address this issue using a novel approach to robustly reconstruct and rigidly derive a shape with no inter-frame variations. This is to produce animations that are jitter free across our sequence based on fluoroscopy video. Our main contributions are 1) retaining the shape of the solid vertebrae across the frame range, 2) helping towards a more accurate image segmentation even when there's a limited training set. We show our pipeline's success by reconstructing and comparing 3D animations of the lumbar spine from a corresponding fluoroscopic video

    A Low-Cost Camera-based Transducer Tracking System for Freehand Three-Dimensional Ultrasound Imaging

    Get PDF
    Freehand three-dimensional ultrasound (3D US) imaging is commonly used for clinical diagnosis and therapy monitoring. In this technique, accurate tracking of the US transducer is a crucial requirement to develop high-quality 3D US volumes. However, current methods for transducer tracking are generally expensive and inconvenient. This thesis presents a low-cost camera-based system for tracking the US transducer with six degrees of freedom (DoF). In this system, two orthogonal cameras with non-overlapped views are mounted on the US transducer. During US scanning, the two cameras are employed to track artificial features attached to the skin of the patient. A 3D surface map is constructed based on the tracked features and the 3D poses of each camera with respect to the skin are extracted separately. The estimated poses of the two cameras are spatially combined to provide accurate and robust pose estimation of the US transducer. In particular, the fusion of the estimated poses by the two cameras is performed using Kalman filtering based technique, which is a popular optimization technique in motion guidance and tracking. The camera-based tracking of the US transducer has been applied to synthesize freehand 3D US volumes. The performance of the proposed system is evaluated by performing in-vitro 3D US imaging experiments and quantifying the synthesized US volumes. The results demonstrate that two points in the 3D US volume separated by a distance of 10 mm can be reconstructed with an average error of 0.35 mm and a 3D volume of a cylinder can be estimated within an error of 3.8%

    Combinatorial optimisation for arterial image segmentation.

    Get PDF
    Cardiovascular disease is one of the leading causes of the mortality in the western world. Many imaging modalities have been used to diagnose cardiovascular diseases. However, each has different forms of noise and artifacts that make the medical image analysis field important and challenging. This thesis is concerned with developing fully automatic segmentation methods for cross-sectional coronary arterial imaging in particular, intra-vascular ultrasound and optical coherence tomography, by incorporating prior and tracking information without any user intervention, to effectively overcome various image artifacts and occlusions. Combinatorial optimisation methods are proposed to solve the segmentation problem in polynomial time. A node-weighted directed graph is constructed so that the vessel border delineation is considered as computing a minimum closed set. A set of complementary edge and texture features is extracted. Single and double interface segmentation methods are introduced. Novel optimisation of the boundary energy function is proposed based on a supervised classification method. Shape prior model is incorporated into the segmentation framework based on global and local information through the energy function design and graph construction. A combination of cross-sectional segmentation and longitudinal tracking is proposed using the Kalman filter and the hidden Markov model. The border is parameterised using the radial basis functions. The Kalman filter is used to adapt the inter-frame constraints between every two consecutive frames to obtain coherent temporal segmentation. An HMM-based border tracking method is also proposed in which the emission probability is derived from both the classification-based cost function and the shape prior model. The optimal sequence of the hidden states is computed using the Viterbi algorithm. Both qualitative and quantitative results on thousands of images show superior performance of the proposed methods compared to a number of state-of-the-art segmentation methods

    Identification et caractérisation des conditions aux limites pour des simulations biomécaniques patient-spécifiques

    Get PDF
    The purpose of the work is to find a way to estimate the boundary conditions of the liver. They play an essential role in forming the predictive capacity of the biomechanical model, but are presented mainly by ligaments, vessels, and surrounding organs, the properties of which are "patient specific" and cannot be measured reliably. We propose to present the boundary conditions as nonlinear springs and estimate their parameters. Firstly, we create a generalized initial approximation using the constitutive law available in the literature and a statistical atlas, obtained from a set of models with segmented ligaments. Then, we correct the approximation based on the nonlinear Kalman filtering approach, which assimilates data obtained from a modality during surgical intervention. To assess the approach, we performed experiments for both synthetic and real data. The results show a certain improvement in simulation accuracy for the cases with estimated boundaries.L'objectif de ce travail est trouvé un moyen d'estimer les conditions aux limites du foie. Elles jouent un rôle essentiel dans la capacité de prédiction du modèle biomécanique, mais sont principalement présentées par les ligaments, les vaisseaux et les organes environnants, dont les propriétés sont "spécifiques au patient" et ne peuvent être mesurées fidèlement. Nous proposons de présenter ces conditions comme des ressorts non linéaires et d'estimer ses paramètres. D’abord, nous créons une approximation initiale en utilisant la loi constitutive disponible dans la littérature et un atlas statistique obtenu à partir des modèles avec des ligaments segmentés. Après, nous la corrigeons basée sur le filtrage de Kalman non linéaire, qui assimile les données acquises d'une modalité pendant la chirurgie. Pour évaluation, nous avons réalisé des expériences avec des données synthétiques et réelles. Les résultats montrent une amélioration de la précision pour les cas avec des limites estimées
    • …
    corecore