Augmented Reality-based Feedback for Technician-in-the-loop C-arm Repositioning

Interventional C-arm imaging is crucial to percutaneous orthopedic procedures as it enables the surgeon to monitor the progress of surgery on the anatomy level. Minimally invasive interventions require repeated acquisition of X-ray images from different anatomical views to verify tool placement. Achieving and reproducing these views often comes at the cost of increased surgical time and radiation dose to both patient and staff. This work proposes a marker-free "technician-in-the-loop" Augmented Reality (AR) solution for C-arm repositioning. The X-ray technician operating the C-arm interventionally is equipped with a head-mounted display capable of recording desired C-arm poses in 3D via an integrated infrared sensor. For C-arm repositioning to a particular target view, the recorded C-arm pose is restored as a virtual object and visualized in an AR environment, serving as a perceptual reference for the technician. We conduct experiments in a setting simulating orthopedic trauma surgery. Our proof-of-principle findings indicate that the proposed system can decrease the 2.76 X-ray images required per desired view down to zero, suggesting substantial reductions of radiation dose during C-arm repositioning. The proposed AR solution is a first step towards facilitating communication between the surgeon and the surgical staff, improving the quality of surgical image acquisition, and enabling context-aware guidance for surgery rooms of the future. The concept of technician-in-the-loop design will become relevant to various interventions considering the expected advancements of sensing and wearable computing in the near future

Quantifying the Impact of Chronic Ischemic Injury on Clinical Outcomes in Acute Stroke With Machine Learning.

Acute stroke is often superimposed on chronic damage from previous cerebrovascular events. This background will inevitably modulate the impact of acute injury on clinical outcomes to an extent that will depend on the precise anatomical pattern of damage. Previous attempts to quantify such modulation have employed only reductive models that ignore anatomical detail. The combination of automated image processing, large-scale data, and machine learning now enables us to quantify the impact of this with high-dimensional multivariate models sensitive to individual variations in the detailed anatomical pattern. We introduce and validate a new automated chronic lesion segmentation routine for use with non-contrast CT brain scans, combining non-parametric outlier-detection score, Zeta, with an unsupervised 3-dimensional maximum-flow, minimum-cut algorithm. The routine was then applied to a dataset of 1,704 stroke patient scans, obtained at their presentation to a hyper-acute stroke unit (St George's Hospital, London, UK), and used to train a support vector machine (SVM) model to predict between low (0-2) and high (3-6) pre-admission and discharge modified Rankin Scale (mRS) scores, quantifying performance by the area under the receiver operating curve (AUROC). In this single center retrospective observational study, our SVM models were able to differentiate between low (0-2) and high (3-6) pre-admission and discharge mRS scores with an AUROC of 0.77 (95% confidence interval of 0.74-0.79), and 0.76 (0.74-0.78), respectively. The chronic lesion segmentation routine achieved a mean (standard deviation) sensitivity, specificity and Dice similarity coefficient of 0.746 (0.069), 0.999 (0.001), and 0.717 (0.091), respectively. We have demonstrated that machine learning models capable of capturing the high-dimensional features of chronic injuries are able to stratify patients-at the time of presentation-by pre-admission and discharge mRS scores. Our fully automated chronic stroke lesion segmentation routine simplifies this process, and utilizes routinely collected CT head scans, thereby facilitating future large-scale studies to develop supportive clinical decision tools

Mesh-Derived Image Partition for 3D-2D Registration in Image-Guided Interventions

RÉSUMÉ Les interventions guidées par images effectuées sous modalité 2D bénéficient de la superposition d'images 3D prises en stage préopératoire. La technologie nécessaire pour cette superposition est le recalage 3D-2D, qui consiste à trouver la position et l'orientation de l'image préopératoire 3D par rapport aux images intraopératoires 2D. Une intégration adéquate d'un algorithme de recalage à un processus chirurgical a le potentiel d'avoir un impact positif sur l'issue de la chirurgie et la durée de l'intervention. Cependant, beaucoup de chirurgies sont effectuées sans l'assistance du recalage, car aucune des solutions actuelles n’est applicable dans leur contexte clinique spécifique. Pour remédier à cette situation, cette thèse porte sur la recherche de solutions pratiques applicables à des interventions guidées par images spécifiques. La première chirurgie étudiée est l'ablation par cathéter pour fibrillation atriale/auriculaire (AC pour FA) effectuée sous fluoroscopie rayons X, une procédure électrophysiologique traitant l'arythmie cardiaque. Dans cette chirurgie, une image volumétrique (soit résonance magnétique (RM) ou tomodensitométrie (TDM)) est prise avant l'opération pour définir l'anatomie de l'atrium gauche (AG) et des veines pulmonaires (VP)s. Un maillage segmenté de ce volume est ensuite utilisé pour offrir un support visuel intraopératoire lors du placement du cathéter d'ablation via sa superposition aux images fluoroscopiques. Cependant, les solutions de recalage actuelles sont trop lentes et requièrent des interventions manuelles, ce qui est problématique quand un recalage intraopératoire est nécessaire pour permettre de pallier aux mouvements du patient. Aussi, les solutions automatiques actuelles qui recalent les images 3D et 2D directement, sans passer par l'identification manuelle de points ficudiaux, ne sont pas assez précises pour être cliniquement utilisables. De plus, les solutions qui n'utilisent pas la cartographie électromagnétique ne fonctionnent pas avec les modalités RM/fluoroscopie rayons X. Ceci est un problème, car nous visons les interventions de AC pour FA qui utilisent la modalité RM sans la cartographie électromagnétique. Il y a deux défis principaux pour arriver à une solution utile cliniquement. Premièrement, résoudre le difficile problème du recalage RM/fluoroscopie complexifié dans le cas de AC pour FA à cause de la correspondance partielle entre les modalités au niveau des VPs. Deuxièmement, de faire ce recalage assez rapidement pour permettre une mise à jour intraopératoire en temps réel dans les cas où le patient bouge pendant l'opération. Afin de remédier à cette situation, nous introduisons une nouvelle méthode de recalage basée sur la partition d'image dérivée d'un maillage (recalage PIDM). Cette méthode utilise les projections d'un maillage segmenté de la modalité 3D pour inférer une segmentation des images fluoroscopiques 2D. Ceci est beaucoup plus rapide que de faire des projections volumétriques et, puisque le maillage peut être segmenté d'une image RM ou TDM sans distinction, la même procédure est valide pour les deux modalités. La justesse du recalage est évaluée par des mesures de similarité qui comparent les propriétés statistiques des zones segmentées et incorporent l'information de profondeur des maillages afin de tenir compte de la correspondance partielle au niveau des VPs. Nous validons l'algorithme de recalage PIDM sur des interventions chirurgicales de AC pour FA provenant de 7 patients différents. Quatre mesures de similarité basées sur le principe de la partition à partir du maillage sont introduites et mises à l'épreuve sur 1400 cas biplans chacune. La précision, la portée et la robustesse de la solution sont évaluées en calculant la distribution de l'erreur (distance de projection) en fonction de la justesse de la pose initiale pour chacun des 5600 recalages. La précision est également évaluée de manière visuelle, en superposant les résultats du recalage et les valeurs-vérité sur les images fluoroscopiques. Pour donner une juste appréciation de la performance attendue de notre algorithme, les exemples visuels sont tirés de cas représentant l'erreur moyenne ainsi que d'un écart-type au-dessus et en dessous. Afin d'évaluer l'extension du recalage PIDM à d'autres types de chirurgies, celui-ci est appliqué à des cas de sclérothérapie de malformation veineuse (SdMV). Ce type de chirurgie est particulièrement délicat à recaler car la malformation peut être présente sur toutes les parties du corps, ce qui offre peu de prévisibilité sur les propriétés des images médicales à recaler d'un patient à l'autre. De plus, cette chirurgie est effectuée en imagerie monoplan et les données ne sont pas accompagnées de méta-information permettant la calibration géométrique du système. Nous démontrons que le recalage PIDM est applicable aux cas de SdMV, mais doit être modifié pour être applicable à la grande variété de parties du corps où les malformations veineuses peuvent être présentes. Le protocole développé pour les chirurgies de AC pour FA peut être utilisé dans les cas où une embolisation ou une démarcation intérieure/extérieure d'une partie du corps est proéminente, mais il est nécessaire d'intégrer l'information de gradients dans les mesures de similarité pour recaler les organes où les os sont prédominants.----------ABSTRACT Image-guided interventions conducted under a 2D modality benefit from the overlay of relevant 3D information from the preoperative stage. The enabling technology for this overlay is 3D-2D registration: the process of finding the spatial pose of a 3D preoperative image in relation to 2D intraoperative images. The successful integration of a registration solution to a surgery has the potential for significant positive impact in terms of likelihood of treatment success and intervention duration. However, many surgeries are routinely done without the assistance of registration because no current solution is practical in their clinical context. In order to remedy these issues, we focus on producing practical, targeted registration solutions to assist image-guided interventions. The first surgery we address is catheter ablation for atrial fibrillation (CA for AF), an electrophysiology procedure to treat heart arrhythmia conducted under X-ray fluoroscopy. In this surgery, a 3D image, either magnetic resonance (MR) or computed tomography (CT), is taken preoperatively to define the anatomy of the left atrium (LA) and pulmonary veins (PV)s. A mesh, segmented from the 3D image, is subsequently used to help positioning the ablation catheter via its overlay on the intraoperative fluoroscopic images. Current clinical registration solutions for CA for AF are slow and often require extensive manual manipulations such as the identification of fiducial points, which is problematic when intraoperative updates of the 3D image's pose are required because of patient movement. The automatic solutions are currently not precise enough to be used clinically. Also, the solutions which do not involve electroanatomic mapping are not suitable for MR/fluoroscopy registration. This is problematic since we target CA for AF interventions where the 3D modality is MR and electroanatomic mapping is not used. There are two principal challenges to overcome in order to provide a clinically useful registration algorithm. First, solving the notoriously hard MR to X-ray fluoroscopy registration problem which is further complicated in cases of CA for AF because of the partial match between modalities at the level of the PVs. Second, solving the registration quickly enough to allow for intraoperative updates required due to the patient's movement. We introduce a new registration methodology based on mesh-derived image partition (MDIP) which uses projections of a mesh segmented from the 3D image in order to infer a segmentation of the 2D X-ray fluoroscopy images. This is orders of magnitude faster than producing volumetric projections and, since the mesh can be segmented from either MR or CT, the same procedure is valid for both modalities. The fitness of the registration is evaluated by custom-built similarity measures that compare the statistical properties of the segmented zones and incorporates mask-depth information to account for the partial match at the level of the PVs. We validate the MDIP algorithm on 7 cases of patients undergoing CA for AF surgery. Four MDIP-based similarity measures are introduced; each one is validated on 1400 biplane registrations. The precision, range, speed and robustness of the solution is assessed by calculating the distribution of projection distance error in function of the correctness of the initial pose for all 5600 biplane registrations. The precision is also evaluated visually by overlaying the ground-truths with results from the registration algorithm. To give a fair appraisal of the expected behavior, the examples are taken from cases exemplifying the average error measured as well as one standard deviation above and under. The registration algorithm is also applied to cases of sclerotherapy for venous malformation (SfVM) in order to assess its portability to other type of surgeries. SfVM are especially challenging because the malformation can be present on any body part, which offers little predictability on the properties of the medical images from one patient to another. Our dataset is sampled from monoplane surgeries and did not come with metadata allowing a geometrical calibration of the system. We demonstrate that MDIP-based registration is applicable to cases of monoplane SfVM, but that modifications are required in order to account for the wide variety of body parts where VMs are common. The protocol developed for CA for AF surgeries can be used for embolizations or when the interior/exterior border of the organ is prominent, but gradient information has to be taken into account by the similarity measures in order to properly register cases where bones are predominant

간 조영술을 위한 혈관 모델 기반의 국부 적응 2D-3D 정합 알고리즘 기법 연구

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2017. 2. 신영길.Two-dimensional–three-dimensional (2D–3D) registration between intra-operative 2D digital subtraction angiography (DSA) and pre-operative 3D computed tomography angiography (CTA) can be used for roadmapping purposes. However, through the projection of 3D vessels, incorrect intersections and overlaps between vessels are produced because of the complex vascular structure, which make it difficult to obtain the correct solution of 2D–3D registration. To overcome these problems, we propose a registration method that selects a suitable part of a 3D vascular structure for a given DSA image and finds the optimized solution to the partial 3D structure. The proposed algorithm can reduce the registration errors because it restricts the range of the 3D vascular structure for the registration by using only the relevant 3D vessels with the given DSA. To search for the appropriate 3D partial structure, we first construct a tree model of the 3D vascular structure and divide it into several subtrees in accordance with the connectivity. Then, the best matched subtree with the given DSA image is selected using the results from the coarse registration between each subtree and the vessels in the DSA image. Finally, a fine registration is conducted to minimize the difference between the selected subtree and the vessels of the DSA image. In experimental results obtained using 10 clinical datasets, the average distance errors in the case of the proposed method were 2.34 ± 1.94 mm. The proposed algorithm converges faster and produces more correct results than the conventional method in evaluations on patient datasets.Chapter 1 Introduction 1 1.1 Background 1 1.2 Problem statement 6 1.3 Main contributions 8 1.4 Contents organization 10 Chapter 2 Related Works 12 2.1 Overview 12 2.1.1 Definitions 14 2.1.2 Intensity-based and feature-based registration 17 2.2 Neurovascular applications 19 2.3 Liver applications 22 2.4 Cardiac applications 27 2.4.1 Rigid registration 27 2.4.2 Non-rigid registration 31 Chapter 3 3D Vascular Structure Model 33 3.1 Vessel segmentation 34 3.1.1 Overview 34 3.1.2 Vesselness filter 36 3.1.3 Vessel segmentation 39 3.2 Skeleton extraction 40 3.2.1 Overview 40 3.2.2 Skeleton extraction based on fast marching method 41 3.3 Graph construction 45 3.4 Generation of subtree structures from 3D tree model 46 Chapter 4 Locally Adaptive Registration 52 4.1 2D centerline extraction 53 4.1.1 Extraction from a single DSA image 54 4.1.2 Extraction from angiographic image sequence 55 4.2 Coarse registration for the detection of the best matched subtree 58 4.3 Fine registration with selected 3D subtree 61 Chapter 5 Experimental Results 63 5.1 Materials 63 5.2 Phantom study 65 5.3 Performance evaluation 69 5.3.1 Evaluation for a single DSA image 69 5.3.2 Evaluation for angiographic image sequence 75 5.4 Comparison with other methods 77 5.5 Parameter study 87 Chapter 6 Conclusion 90 Bibliography 92 초록 109Docto