On the Real-Time Performance, Robustness and Accuracy of Medical Image Non-Rigid Registration

Three critical issues about medical image non-rigid registration are performance, robustness and accuracy. A registration method, which is capable of responding timely with an accurate alignment, robust against the variation of the image intensity and the missing data, is desirable for its clinical use. This work addresses all three of these issues. Unacceptable execution time of Non-rigid registration (NRR) often presents a major obstacle to its routine clinical use. We present a hybrid data partitioning method to parallelize a NRR method on a cooperative architecture, which enables us to get closer to the goal: accelerating using architecture rather than designing a parallel algorithm from scratch. to further accelerate the performance for the GPU part, a GPU optimization tool is provided to automatically optimize GPU execution configuration.;Missing data and variation of the intensity are two severe challenges for the robustness of the registration method. A novel point-based NRR method is presented to resolve mapping function (deformation field) with the point correspondence missing. The novelty of this method lies in incorporating a finite element biomechanical model into an Expectation and Maximization (EM) framework to resolve the correspondence and mapping function simultaneously. This method is extended to deal with the deformation induced by tumor resection, which imposes another challenge, i.e. incomplete intra-operative MRI. The registration is formulated as a three variable (Correspondence, Deformation Field, and Resection Region) functional minimization problem and resolved by a Nested Expectation and Maximization framework. The experimental results show the effectiveness of this method in correcting the deformation in the vicinity of the tumor. to deal with the variation of the intensity, two different methods are developed depending on the specific application. For the mono-modality registration on delayed enhanced cardiac MRI and cine MRI, a hybrid registration method is designed by unifying both intensity- and feature point-based metrics into one cost function. The experiment on the moving propagation of suspicious myocardial infarction shows effectiveness of this hybrid method. For the multi-modality registration on MRI and CT, a Mutual Information (MI)-based NRR is developed by modeling the underlying deformation as a Free-Form Deformation (FFD). MI is sensitive to the variation of the intensity due to equidistant bins. We overcome this disadvantage by designing a Top-to-Down K-means clustering method to naturally group similar intensities into one bin. The experiment shows this method can increase the accuracy of the MI-based registration.;In image registration, a finite element biomechanical model is usually employed to simulate the underlying movement of the soft tissue. We develop a multi-tissue mesh generation method to build a heterogeneous biomechanical model to realistically simulate the underlying movement of the brain. We focus on the following four critical mesh properties: tissue-dependent resolution, fidelity to tissue boundaries, smoothness of mesh surfaces, and element quality. Each mesh property can be controlled on a tissue level. The experiments on comparing the homogeneous model with the heterogeneous model demonstrate the effectiveness of the heterogeneous model in improving the registration accuracy

Complementary Situational Awareness for an Intelligent Telerobotic Surgical Assistant System

Robotic surgical systems have contributed greatly to the advancement of Minimally Invasive Surgeries (MIS). More specifically, telesurgical robots have provided enhanced dexterity to surgeons performing MIS procedures. However, current robotic teleoperated systems have only limited situational awareness of the patient anatomy and surgical environment that would typically be available to a surgeon in an open surgery. Although the endoscopic view enhances the visualization of the anatomy, perceptual understanding of the environment and anatomy is still lacking due to the absence of sensory feedback. In this work, these limitations are addressed by developing a computational framework to provide Complementary Situational Awareness (CSA) in a surgical assistant. This framework aims at improving the human-robot relationship by providing elaborate guidance and sensory feedback capabilities for the surgeon in complex MIS procedures. Unlike traditional teleoperation, this framework enables the user to telemanipulate the situational model in a virtual environment and uses that information to command the slave robot with appropriate admittance gains and environmental constraints. Simultaneously, the situational model is updated based on interaction of the slave robot with the task space environment. However, developing such a system to provide real-time situational awareness requires that many technical challenges be met. To estimate intraoperative organ information continuous palpation primitives are required. Intraoperative surface information needs to be estimated in real-time while the organ is being palpated/scanned. The model of the task environment needs to be updated in near real-time using the estimated organ geometry so that the force-feedback applied on the surgeon's hand would correspond to the actual location of the model. This work presents a real-time framework that meets these requirements/challenges to provide situational awareness of the environment in the task space. Further, visual feedback is also provided for the surgeon/developer to view the near video frame rate updates of the task model. All these functions are executed in parallel and need to have a synchronized data exchange. The system is very portable and can be incorporated to any existing telerobotic platforms with minimal overhead

Laser speckle contrast imaging for assessing microcirculation in diabetic foot disease

Diabetes Mellitus is one of the most common chronic diseases worldwide and it has been estimated that the number of people with diabetes will grow because of our lifestyle changes and longer life-expectancy. This development is disturbing because diabetes has a severe impact on the patient’s life. It can cause serious complications such as blindness, heart attacks, and strokes. Another severe and most frequently recognized complication of diabetes are diabetic foot ulcers. Diabetic foot ulcers are associated with high morbidity, mortality and healthcare costs. These consequences can even be more severe in case of diabetic foot ulcers with critical ischemia. Hence, an early and accurate diagnosis of this health condition is needed. Today, the most common diagnosis of (critical-) ischemia is determined in clinical practice, using non-invasive measurements of blood flow in the feet, by means of assessments of the ankle pressure, toe pressure or transcutaneous oxygen pressure. Yet, these currently used non-invasive measurement techniques have various disadvantages. Therefore, research into improved ways to assess the microcirculation in people with diabetic foot ulcers is needed. This thesis tried to fill this knowledge gap by looking into the potential of novel optical imaging techniques, and in particular in the potential of Laser Speckle Contrast Imaging (LSCI), for the assessment of the microcirculation in the diabetic foot and its applicability in the clinical setting.LSCI shows both similarities and differences with the currently used non-invasive blood pressure measurements, which is an indication that it measures perfusion in a novel and different way than the currently used techniques. However, in our cohort we have not been able to link perfusion as measured with LSCI to clinical outcome parameters such as ulcer healing or successful revascularization. Despite this current lack of applicability, this novel non-invasive optical imaging technique still offer potential to change clinical practice in the field of diabetic foot disease. For this, future research is needed to further investigate how LSCI can best be used to improve outcomes of diabetic foot ulcers.<br/

Ultrasound and Photoacoustic Techniques for Surgical Guidance Inside and Around the Spine

Technological advances in image-guidance have made a significant impact in surgical standards, allowing for safer and less invasive procedures. Ultrasound and photoacoustic imaging are promising options for surgical guidance given their real-time capabilities without the use of ionizing radiation. However, challenges to improve the feasibility of ultrasound- and photoacoustic-based surgical guidance persists in the presence of bone. In this thesis, we address four challenges surrounding the implementation of ultrasound- and photoacoustic-based surgical guidance in clinical scenarios inside and around the spine. First, we introduce a novel regularized implementation of short-lag spatial coherence (SLSC) beamforming, named locally-weighted short-lag spatial coherence (LW-SLSC). LW-SLSC improves the segmentation of bony structures in ultrasound images, thus reducing the hardware and software cost of registering pre and intra-operative volumes. Second, we describe a contour analysis framework to characterize and differentiate photoacoustic signals originating from cancellous and cortical bone, which is critical for a safety navigation of surgical tools through small bony cavities such as the pedicle. This analysis is also useful for localizing tool tips within the pedicle. Third, we developed a GPU approach to SLSC beamforming to improve the signal-to-noise ratio of photoacoustic targets using low laser energies, thus improving the performance of robotic visual servoing of tooltips and enabling miniaturization of laser systems in the operating room. Finally, we developed a novel acoustic-based atlas method to identify photoacoustic contrast agents and discriminate them from tissue using only two laser wavelengths. This approach significantly reduces acquisition times in comparison to conventional spectral unmixing techniques. These four contributions are beneficial for the transition of a combined ultrasound and photoacoustic-based image-guidance system towards more challenging scenarios of surgical navigation. Focusing on bone structures inside and surrounding the spine, the newly combined systems and techniques demonstrated herein feature robust, accurate, and real-time capabilities to register to preoperative images, localize surgical tool tips, and characterize biomarkers. These contributions strengthen the range of possibilities for spinous and transthoracic ultrasound and photoacoustic navigation, broaden the scope of this field, and shorten the road to clinical implementation in the operating room

Image analysis for extracapsular hip fracture surgery

PhD ThesisDuring the implant insertion phase of extracapsular hip fracture surgery, a surgeon visually inspects digital radiographs to infer the best position for the implant. The inference is made by “eye-balling”. This clearly leaves room for trial and error which is not ideal for the patient. This thesis presents an image analysis approach to estimating the ideal positioning for the implant using a variant of the deformable templates model known as the Constrained Local Model (CLM). The Model is a synthesis of shape and local appearance models learned from a set of annotated landmarks and their corresponding local patches extracted from digital femur x-rays. The CLM in this work highlights both Principal Component Analysis (PCA) and Probabilistic PCA as regularisation components; the PPCA variant being a novel adaptation of the CLM framework that accounts for landmark annotation error which the PCA version does not account for. Our CLM implementation is used to articulate 2 clinical metrics namely: the Tip-Apex Distance and Parker’s Ratio (routinely used by clinicians to assess the positioning of the surgical implant during hip fracture surgery) within the image analysis framework. With our model, we were able to automatically localise signi cant landmarks on the femur, which were subsequently used to measure Parker’s Ratio directly from digital radiographs and determine an optimal placement for the surgical implant in 87% of the instances; thereby, achieving fully automatic measurement of Parker’s Ratio as opposed to manual measurements currently performed in the surgical theatre during hip fracture surgery

Functional Organization of the Human Brain: How We See, Feel, and Decide.

The human brain is responsible for constructing how we perceive, think, and act in the world around us. The organization of these functions is intricately distributed throughout the brain. Here, I discuss how functional magnetic resonance imaging (fMRI) was employed to understand three broad questions: how do we see, feel, and decide? First, high-resolution fMRI was used to measure the polar angle representation of saccadic eye movements in the superior colliculus. We found that eye movements along the superior-inferior visual field are mapped across the medial-lateral anatomy of a subcortical midbrain structure, the superior colliculus (SC). This result is consistent with the topography in monkey SC. Second, we measured the empathic responses of the brain as people watched a hand get painfully stabbed with a needle. We found that if the hand was labeled as belonging to the same religion as the observer, the empathic neural response was heightened, creating a strong ingroup bias that could not be readily manipulated. Third, we measured brain activity in individuals as they made free decisions (i.e., choosing randomly which of two buttons to press) and found the activity within fronto-thalamic networks to be significantly decreased compared to being instructed (forced) to press a particular button. I also summarize findings from several other projects ranging from addiction therapies to decoding visual imagination to how corporations are represented as people. Together, these approaches illustrate how functional neuroimaging can be used to understand the organization of the human brain