3D shape instantiation for intra-operative navigation from a single 2D projection

Unlike traditional open surgery where surgeons can see the operation area clearly, in robot-assisted Minimally Invasive Surgery (MIS), a surgeon’s view of the region of interest is usually limited. Currently, 2D images from fluoroscopy, Magnetic Resonance Imaging (MRI), endoscopy or ultrasound are used for intra-operative guidance as real-time 3D volumetric acquisition is not always possible due to the acquisition speed or exposure constraints. 3D reconstruction, however, is key to navigation in complex in vivo geometries and can help resolve this issue. Novel 3D shape instantiation schemes are developed in this thesis, which can reconstruct the high-resolution 3D shape of a target from limited 2D views, especially a single 2D projection or slice. To achieve a complete and automatic 3D shape instantiation pipeline, segmentation schemes based on deep learning are also investigated. These include normalization schemes for training U-Nets and network architecture design of Atrous Convolutional Neural Networks (ACNNs). For U-Net normalization, four popular normalization methods are reviewed, then Instance-Layer Normalization (ILN) is proposed. It uses a sigmoid function to linearly weight the feature map after instance normalization and layer normalization, and cascades group normalization after the weighted feature map. Detailed validation results potentially demonstrate the practical advantages of the proposed ILN for effective and robust segmentation of different anatomies. For network architecture design in training Deep Convolutional Neural Networks (DCNNs), the newly proposed ACNN is compared to traditional U-Net where max-pooling and deconvolutional layers are essential. Only convolutional layers are used in the proposed ACNN with different atrous rates and it has been shown that the method is able to provide a fully-covered receptive field with a minimum number of atrous convolutional layers. ACNN enhances the robustness and generalizability of the analysis scheme by cascading multiple atrous blocks. Validation results have shown the proposed method achieves comparable results to the U-Net in terms of medical image segmentation, whilst reducing the trainable parameters, thus improving the convergence and real-time instantiation speed. For 3D shape instantiation of soft and deforming organs during MIS, Sparse Principle Component Analysis (SPCA) has been used to analyse a 3D Statistical Shape Model (SSM) and to determine the most informative scan plane. Synchronized 2D images are then scanned at the most informative scan plane and are expressed in a 2D SSM. Kernel Partial Least Square Regression (KPLSR) has been applied to learn the relationship between the 2D and 3D SSM. It has been shown that the KPLSR-learned model developed in this thesis is able to predict the intra-operative 3D target shape from a single 2D projection or slice, thus permitting real-time 3D navigation. Validation results have shown the intrinsic accuracy achieved and the potential clinical value of the technique. The proposed 3D shape instantiation scheme is further applied to intra-operative stent graft deployment for the robot-assisted treatment of aortic aneurysms. Mathematical modelling is first used to simulate the stent graft characteristics. This is then followed by the Robust Perspective-n-Point (RPnP) method to instantiate the 3D pose of fiducial markers of the graft. Here, Equally-weighted Focal U-Net is proposed with a cross-entropy and an additional focal loss function. Detailed validation has been performed on patient-specific stent grafts with an accuracy between 1-3mm. Finally, the relative merits and potential pitfalls of all the methods developed in this thesis are discussed, followed by potential future research directions and additional challenges that need to be tackled.Open Acces

An easy-to-use 2D-3D registration process

This report describes the work carried out during the project representing the final part of the Professional Doctorate in Engineering (PDEng) degree pro-gram in Software Technology provided by the Eindhoven University of Technology and Stan Ackermans Institute. The project name is "An Easy-to-use 2D-3D Registration Process". This project consists of the design and development of an easy-to-use inter-face for image registration for the VesselNavigator prototype developed in Philips Healthcare. The report is addressed to a technical audience that has a general knowledge about software design and medical imaging. Readers that are interested in the clinical background and the goals of the project should refer to Chapters 2-5. Details about the software architecture design and implementation are provid-ed in Chapters 6-8. For the result of the project and the software process used readers must address Chapters 9 and 10

3D Reconstruction of Interventional Material from Very Few X-Ray Projections for Interventional Image Guidance

Today, minimally invasive endovascular interventions are usually guided by 2D fluoroscopy, i.e. a live 2D X-ray image. However, 3D fluoroscopy, i.e. a live 3D image reconstructed from a stream of 2D X-ray images, could improve spatial awareness. 3D fluoroscopy is, however, not used today, since no appropriate 3D reconstruction algorithm is known. Existing algorithms for the real-time reconstruction of interventional material (guidewires, stents, catheters, etc.) are either only capable of reconstructing a single guidewire or catheter, or use too many X-ray images and therefore too much dose per 3D reconstruction. The goal of this thesis was to reconstruct complex arrangements of interventional material from as few X-ray images as possible. To this end, a previously proposed algorithm for the reconstruction of interventional material from four X-ray images was adapted. Five key improvements allowed to reduce the number of X-ray images per 3D reconstruction from four to two: a) use of temporal information in a rotating imaging setup, b) separate reconstruction of different types of interventional material enabled by the computation of semantic interventional material extraction images, c) compensation of stent motion by spatial transformer networks, d) per-projection backprojection and e) binarization of the guidewire extraction images. While previously only single curves could be reconstructed from two newly acquired X-ray images, the proposed pipeline can reconstruct stents and even stent-guidewire combinations. Submillimeter reconstruction accuracy was demonstrated on measured X-ray images of interventional material inside an anthropomorphic phantom with simulated respiratory motion. Measurements of the dose area product rate of the proposed 3D reconstruction pipeline indicate a dose burden roughly similar to that of 2D fluoroscopy

Augmented Image-Guidance for Transcatheter Aortic Valve Implantation

The introduction of transcatheter aortic valve implantation (TAVI), an innovative stent-based technique for delivery of a bioprosthetic valve, has resulted in a paradigm shift in treatment options for elderly patients with aortic stenosis. While there have been major advancements in valve design and access routes, TAVI still relies largely on single-plane fluoroscopy for intraoperative navigation and guidance, which provides only gross imaging of anatomical structures. Inadequate imaging leading to suboptimal valve positioning contributes to many of the early complications experienced by TAVI patients, including valve embolism, coronary ostia obstruction, paravalvular leak, heart block, and secondary nephrotoxicity from contrast use. A potential method of providing improved image-guidance for TAVI is to combine the information derived from intra-operative fluoroscopy and TEE with pre-operative CT data. This would allow the 3D anatomy of the aortic root to be visualized along with real-time information about valve and prosthesis motion. The combined information can be visualized as a `merged\u27 image where the different imaging modalities are overlaid upon each other, or as an `augmented\u27 image, where the location of key target features identified on one image are displayed on a different imaging modality. This research develops image registration techniques to bring fluoroscopy, TEE, and CT models into a common coordinate frame with an image processing workflow that is compatible with the TAVI procedure. The techniques are designed to be fast enough to allow for real-time image fusion and visualization during the procedure, with an intra-procedural set-up requiring only a few minutes. TEE to fluoroscopy registration was achieved using a single-perspective TEE probe pose estimation technique. The alignment of CT and TEE images was achieved using custom-designed algorithms to extract aortic root contours from XPlane TEE images, and matching the shape of these contours to a CT-derived surface model. Registration accuracy was assessed on porcine and human images by identifying targets (such as guidewires or coronary ostia) on the different imaging modalities and measuring the correspondence of these targets after registration. The merged images demonstrated good visual alignment of aortic root structures, and quantitative assessment measured an accuracy of less than 1.5mm error for TEE-fluoroscopy registration and less than 6mm error for CT-TEE registration. These results suggest that the image processing techniques presented have potential for development into a clinical tool to guide TAVI. Such a tool could potentially reduce TAVI complications, reducing morbidity and mortality and allowing for a safer procedure

Contrast‐Enhanced Renal MRA

The rapid growth of magnetic resonance imaging systems with enhanced gradient systems together with improved pulse sequences has improved the ability to image blood vessels with a spatial and temporal resolution similar to conventional X‐ray angiography. With patients who cannot undergo X‐ray angiography because they are contraindicated for iodinated contrast agents (having a creatinine level > 2.0), MRA (magnetic resonance angiography) has proven to be the modality of choice. Since the first demonstration of such contrast‐enhanced studies in the abdominal aorta, there have been continual improvements in methods due to improved hardware/software capabilities. This unit presents the MR protocols to image vascular morphology using contrast‐enhanced 3‐D‐MRA techniques. The pulse sequences described herein are based on the authors’ experience with a Siemens 1.5 T Vision and 1.5 T Sonata scanners, but are expected to be equally applicable to machines from other manufacturers.Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/145369/1/cpmia2801.pd

A Robotic Platform for Endovascular Aneurysm Repair

International audienceAn EndoVascular Aneurysm Repair (EVAR) isa procedure used to fix an aneurysm of the aorta. In thisprocedure, a guide is inserted by the femoral artery. This guidegoes through to the height of the aneurysm and then a catheterfollows the guide. Next, a stent graft is deployed in order torepair the aortic aneurysm. The objectives of our work is todevelop a low-cost robotic system and implement a programthat helps the trajectory planning during an endovascularoperation. More precisely, this program can predict if the aortawill break or not depending on the guide used. Such a roboticplatform could serve as a teaching instrument by creating anenvironment for young surgeons in which they will be able topractice their skills to perform an EVAR. This paper describesthe different components of this platform and provides someexperimental results

3D Imaging for Planning of Minimally Invasive Surgical Procedures

Novel minimally invasive surgeries are used for treating cardiovascular diseases and are performed under 2D fluoroscopic guidance with a C-arm system. 3D multidetector row computed tomography (MDCT) images are routinely used for preprocedural planning and postprocedural follow-up. For preprocedural planning, the ability to integrate the MDCT with fluoroscopic images for intraprocedural guidance is of clinical interest. Registration may be facilitated by rotating the C-arm to acquire 3D C-arm CT images. This dissertation describes the development of optimal scan and contrast parameters for C-arm CT in 6 swine. A 5-s ungated C-arm CT acquisition during rapid ventricular pacing with aortic root injection using minimal contrast (36 mL), producing high attenuation (1226), few artifacts (2.0), and measurements similar to those from MDCT (p\u3e0.05) was determined optimal. 3D MDCT and C-arm CT images were registered to overlay the aortic structures from MDCT onto fluoroscopic images for guidance in placing the prosthesis. This work also describes the development of a methodology to develop power equation (R2\u3e0.998) for estimating dose with C-arm CT based on applied tube voltage. Application in 10 patients yielded 5.48┬▒177 2.02 mGy indicating minimal radiation burden. For postprocedural follow-up, combinations of non-contrast, arterial, venous single energy CT (SECT) scans are used to monitor patients at multiple time intervals resulting in high cumulative radiation dose. Employing a single dual-energy CT (DECT) scan to replace two SECT scans can reduce dose. This work focuses on evaluating the feasibility of DECT imaging in the arterial phase. The replacement of non-contrast and arterial SECT acquisitions with one arterial DECT acquisition in 30 patients allowed generation of virtual non-contrast (VNC) images with 31 dose savings. Aortic luminal attenuation in VNC (32┬▒177 2 HU) was similar to true non-contrast images (35┬▒177 4 HU) indicating presence of unattenuated blood. To improve discrimination between c