3D shape instantiation for intra-operative navigation from a single 2D projection

Unlike traditional open surgery where surgeons can see the operation area clearly, in robot-assisted Minimally Invasive Surgery (MIS), a surgeon’s view of the region of interest is usually limited. Currently, 2D images from fluoroscopy, Magnetic Resonance Imaging (MRI), endoscopy or ultrasound are used for intra-operative guidance as real-time 3D volumetric acquisition is not always possible due to the acquisition speed or exposure constraints. 3D reconstruction, however, is key to navigation in complex in vivo geometries and can help resolve this issue. Novel 3D shape instantiation schemes are developed in this thesis, which can reconstruct the high-resolution 3D shape of a target from limited 2D views, especially a single 2D projection or slice. To achieve a complete and automatic 3D shape instantiation pipeline, segmentation schemes based on deep learning are also investigated. These include normalization schemes for training U-Nets and network architecture design of Atrous Convolutional Neural Networks (ACNNs). For U-Net normalization, four popular normalization methods are reviewed, then Instance-Layer Normalization (ILN) is proposed. It uses a sigmoid function to linearly weight the feature map after instance normalization and layer normalization, and cascades group normalization after the weighted feature map. Detailed validation results potentially demonstrate the practical advantages of the proposed ILN for effective and robust segmentation of different anatomies. For network architecture design in training Deep Convolutional Neural Networks (DCNNs), the newly proposed ACNN is compared to traditional U-Net where max-pooling and deconvolutional layers are essential. Only convolutional layers are used in the proposed ACNN with different atrous rates and it has been shown that the method is able to provide a fully-covered receptive field with a minimum number of atrous convolutional layers. ACNN enhances the robustness and generalizability of the analysis scheme by cascading multiple atrous blocks. Validation results have shown the proposed method achieves comparable results to the U-Net in terms of medical image segmentation, whilst reducing the trainable parameters, thus improving the convergence and real-time instantiation speed. For 3D shape instantiation of soft and deforming organs during MIS, Sparse Principle Component Analysis (SPCA) has been used to analyse a 3D Statistical Shape Model (SSM) and to determine the most informative scan plane. Synchronized 2D images are then scanned at the most informative scan plane and are expressed in a 2D SSM. Kernel Partial Least Square Regression (KPLSR) has been applied to learn the relationship between the 2D and 3D SSM. It has been shown that the KPLSR-learned model developed in this thesis is able to predict the intra-operative 3D target shape from a single 2D projection or slice, thus permitting real-time 3D navigation. Validation results have shown the intrinsic accuracy achieved and the potential clinical value of the technique. The proposed 3D shape instantiation scheme is further applied to intra-operative stent graft deployment for the robot-assisted treatment of aortic aneurysms. Mathematical modelling is first used to simulate the stent graft characteristics. This is then followed by the Robust Perspective-n-Point (RPnP) method to instantiate the 3D pose of fiducial markers of the graft. Here, Equally-weighted Focal U-Net is proposed with a cross-entropy and an additional focal loss function. Detailed validation has been performed on patient-specific stent grafts with an accuracy between 1-3mm. Finally, the relative merits and potential pitfalls of all the methods developed in this thesis are discussed, followed by potential future research directions and additional challenges that need to be tackled.Open Acces

Constrained Statistical Modelling of Knee Flexion from Multi-Pose Magnetic Resonance Imaging

© 1982-2012 IEEE.Reconstruction of the anterior cruciate ligament (ACL) through arthroscopy is one of the most common procedures in orthopaedics. It requires accurate alignment and drilling of the tibial and femoral tunnels through which the ligament graft is attached. Although commercial computer-Assisted navigation systems exist to guide the placement of these tunnels, most of them are limited to a fixed pose without due consideration of dynamic factors involved in different knee flexion angles. This paper presents a new model for intraoperative guidance of arthroscopic ACL reconstruction with reduced error particularly in the ligament attachment area. The method uses 3D preoperative data at different flexion angles to build a subject-specific statistical model of knee pose. To circumvent the problem of limited training samples and ensure physically meaningful pose instantiation, homogeneous transformations between different poses and local-deformation finite element modelling are used to enlarge the training set. Subsequently, an anatomical geodesic flexion analysis is performed to extract the subject-specific flexion characteristics. The advantages of the method were also tested by detailed comparison to standard Principal Component Analysis (PCA), nonlinear PCA without training set enlargement, and other state-of-The-Art articulated joint modelling methods. The method yielded sub-millimetre accuracy, demonstrating its potential clinical value

Computer-Assisted Electroanatomical Guidance for Cardiac Electrophysiology Procedures

Cardiac arrhythmias are serious life-threatening episodes affecting both the aging population and younger patients with pre-existing heart conditions. One of the most effective therapeutic procedures is the minimally-invasive catheter-driven endovascular electrophysiology study, whereby electrical potentials and activation patterns in the affected cardiac chambers are measured and subsequent ablation of arrhythmogenic tissue is performed. Despite emerging technologies such as electroanatomical mapping and remote intraoperative navigation systems for improved catheter manipulation and stability, successful ablation of arrhythmias is still highly-dependent on the operator’s skills and experience. This thesis proposes a framework towards standardisation in the electroanatomical mapping and ablation planning by merging knowledge transfer from previous cases and patient-specific data. In particular, contributions towards four different procedural aspects were made: optimal electroanatomical mapping, arrhythmia path computation, catheter tip stability analysis, and ablation simulation and optimisation. In order to improve the intraoperative electroanatomical map, anatomical areas of high mapping interest were proposed, as learned from previous electrophysiology studies. Subsequently, the arrhythmic wave propagation on the endocardial surface and potential ablation points were computed. The ablation planning is further enhanced, firstly by the analysis of the catheter tip stability and the probability of slippage at sparse locations on the endocardium and, secondly, by the simulation of the ablation result from the computation of convolutional matrices which model mathematically the ablation process. The methods proposed by this thesis were validated on data from patients with complex congenital heart disease, who present unusual cardiac anatomy and consequently atypical arrhythmias. The proposed methods also build a generic framework for computer guidance of electrophysiology, with results showing complementary information that can be easily integrated into the clinical workflow.Open Acces

Appearance Modelling and Reconstruction for Navigation in Minimally Invasive Surgery

Minimally invasive surgery is playing an increasingly important role for patient care. Whilst its direct patient benefit in terms of reduced trauma, improved recovery and shortened hospitalisation has been well established, there is a sustained need for improved training of the existing procedures and the development of new smart instruments to tackle the issue of visualisation, ergonomic control, haptic and tactile feedback. For endoscopic intervention, the small field of view in the presence of a complex anatomy can easily introduce disorientation to the operator as the tortuous access pathway is not always easy to predict and control with standard endoscopes. Effective training through simulation devices, based on either virtual reality or mixed-reality simulators, can help to improve the spatial awareness, consistency and safety of these procedures. This thesis examines the use of endoscopic videos for both simulation and navigation purposes. More specifically, it addresses the challenging problem of how to build high-fidelity subject-specific simulation environments for improved training and skills assessment. Issues related to mesh parameterisation and texture blending are investigated. With the maturity of computer vision in terms of both 3D shape reconstruction and localisation and mapping, vision-based techniques have enjoyed significant interest in recent years for surgical navigation. The thesis also tackles the problem of how to use vision-based techniques for providing a detailed 3D map and dynamically expanded field of view to improve spatial awareness and avoid operator disorientation. The key advantage of this approach is that it does not require additional hardware, and thus introduces minimal interference to the existing surgical workflow. The derived 3D map can be effectively integrated with pre-operative data, allowing both global and local 3D navigation by taking into account tissue structural and appearance changes. Both simulation and laboratory-based experiments are conducted throughout this research to assess the practical value of the method proposed

Context-aware learning for robot-assisted endovascular catheterization

Endovascular intervention has become a mainstream treatment of cardiovascular diseases. However, multiple challenges remain such as unwanted radiation exposures, limited two-dimensional image guidance, insufficient force perception and haptic cues. Fast evolving robot-assisted platforms improve the stability and accuracy of instrument manipulation. The master-slave system also removes radiation to the operator. However, the integration of robotic systems into the current surgical workflow is still debatable since repetitive, easy tasks have little value to be executed by the robotic teleoperation. Current systems offer very low autonomy, potential autonomous features could bring more benefits such as reduced cognitive workloads and human error, safer and more consistent instrument manipulation, ability to incorporate various medical imaging and sensing modalities. This research proposes frameworks for automated catheterisation with different machine learning-based algorithms, includes Learning-from-Demonstration, Reinforcement Learning, and Imitation Learning. Those frameworks focused on integrating context for tasks in the process of skill learning, hence achieving better adaptation to different situations and safer tool-tissue interactions. Furthermore, the autonomous feature was applied to next-generation, MR-safe robotic catheterisation platform. The results provide important insights into improving catheter navigation in the form of autonomous task planning, self-optimization with clinical relevant factors, and motivate the design of intelligent, intuitive, and collaborative robots under non-ionizing image modalities.Open Acces

Surgical GPS Proof of Concept for Scoliosis Surgery

Scoliotic deformities may be addressed with either anterior or posterior approaches for scoliosis correction procedures. While typically quite invasive, the impact of these operations may be reduced through the use of computer-assisted surgery. A combination of physician-designated anatomical landmarks and surgical ontologies allows for real-time intraoperative guidance during computer-assisted surgical interventions. Predetermined landmarks are labeled on an identical patient model, which seeks to encompass vertebrae, intervertebral disks, ligaments, and other soft tissues. The inclusion of this anatomy permits the consideration of hypothetical forces that are previously not well characterized in a patient-specific manner. Updated ontologies then suggest procedural directions throughout the surgical corridor, observing the positioning of both the physician and the anatomical landmarks of interest at the present moment. Merging patient-specific models, physician-designated landmarks, and ontologies to produce real-time recommendations magnifies the successful outcome of scoliosis correction through enhanced pre-surgical planning, reduced invasiveness, and shorted recovery time