Coronary Artery Segmentation and Motion Modelling

Conventional coronary artery bypass surgery requires invasive sternotomy and the use of a cardiopulmonary bypass, which leads to long recovery period and has high infectious potential. Totally endoscopic coronary artery bypass (TECAB) surgery based on image guided robotic surgical approaches have been developed to allow the clinicians to conduct the bypass surgery off-pump with only three pin holes incisions in the chest cavity, through which two robotic arms and one stereo endoscopic camera are inserted. However, the restricted field of view of the stereo endoscopic images leads to possible vessel misidentification and coronary artery mis-localization. This results in 20-30% conversion rates from TECAB surgery to the conventional approach. We have constructed patient-specific 3D + time coronary artery and left ventricle motion models from preoperative 4D Computed Tomography Angiography (CTA) scans. Through temporally and spatially aligning this model with the intraoperative endoscopic views of the patient's beating heart, this work assists the surgeon to identify and locate the correct coronaries during the TECAB precedures. Thus this work has the prospect of reducing the conversion rate from TECAB to conventional coronary bypass procedures. This thesis mainly focus on designing segmentation and motion tracking methods of the coronary arteries in order to build pre-operative patient-specific motion models. Various vessel centreline extraction and lumen segmentation algorithms are presented, including intensity based approaches, geometric model matching method and morphology-based method. A probabilistic atlas of the coronary arteries is formed from a group of subjects to facilitate the vascular segmentation and registration procedures. Non-rigid registration framework based on a free-form deformation model and multi-level multi-channel large deformation diffeomorphic metric mapping are proposed to track the coronary motion. The methods are applied to 4D CTA images acquired from various groups of patients and quantitatively evaluated

Computerized Analysis of Magnetic Resonance Images to Study Cerebral Anatomy in Developing Neonates

The study of cerebral anatomy in developing neonates is of great importance for the understanding of brain development during the early period of life. This dissertation therefore focuses on three challenges in the modelling of cerebral anatomy in neonates during brain development. The methods that have been developed all use Magnetic Resonance Images (MRI) as source data. To facilitate study of vascular development in the neonatal period, a set of image analysis algorithms are developed to automatically extract and model cerebral vessel trees. The whole process consists of cerebral vessel tracking from automatically placed seed points, vessel tree generation, and vasculature registration and matching. These algorithms have been tested on clinical Time-of- Flight (TOF) MR angiographic datasets. To facilitate study of the neonatal cortex a complete cerebral cortex segmentation and reconstruction pipeline has been developed. Segmentation of the neonatal cortex is not effectively done by existing algorithms designed for the adult brain because the contrast between grey and white matter is reversed. This causes pixels containing tissue mixtures to be incorrectly labelled by conventional methods. The neonatal cortical segmentation method that has been developed is based on a novel expectation-maximization (EM) method with explicit correction for mislabelled partial volume voxels. Based on the resulting cortical segmentation, an implicit surface evolution technique is adopted for the reconstruction of the cortex in neonates. The performance of the method is investigated by performing a detailed landmark study. To facilitate study of cortical development, a cortical surface registration algorithm for aligning the cortical surface is developed. The method first inflates extracted cortical surfaces and then performs a non-rigid surface registration using free-form deformations (FFDs) to remove residual alignment. Validation experiments using data labelled by an expert observer demonstrate that the method can capture local changes and follow the growth of specific sulcus

A Geometric Approach for Deciphering Protein Structure from Cryo-EM Volumes

Electron Cryo-Microscopy or cryo-EM is an area that has received much attention in the recent past. Compared to the traditional methods of X-Ray Crystallography and NMR Spectroscopy, cryo-EM can be used to image much larger complexes, in many different conformations, and under a wide range of biochemical conditions. This is because it does not require the complex to be crystallisable. However, cryo-EM reconstructions are limited to intermediate resolutions, with the state-of-the-art being 3.6A, where secondary structure elements can be visually identified but not individual amino acid residues. This lack of atomic level resolution creates new computational challenges for protein structure identification. In this dissertation, we present a suite of geometric algorithms to address several aspects of protein modeling using cryo-EM density maps. Specifically, we develop novel methods to capture the shape of density volumes as geometric skeletons. We then use these skeletons to find secondary structure elements: SSEs) of a given protein, to identify the correspondence between these SSEs and those predicted from the primary sequence, and to register high-resolution protein structures onto the density volume. In addition, we designed and developed Gorgon, an interactive molecular modeling system, that integrates the above methods with other interactive routines to generate reliable and accurate protein backbone models

Visual analytics methods for shape analysis of biomedical images exemplified on rodent skull morphology

In morphometrics and its application fields like medicine and biology experts are interested in causal relations of variation in organismic shape to phylogenetic, ecological, geographical, epidemiological or disease factors - or put more succinctly by Fred L. Bookstein, morphometrics is "the study of covariances of biological form". In order to reveal causes for shape variability, targeted statistical analysis correlating shape features against external and internal factors is necessary but due to the complexity of the problem often not feasible in an automated way. Therefore, a visual analytics approach is proposed in this thesis that couples interactive visualizations with automated statistical analyses in order to stimulate generation and qualitative assessment of hypotheses on relevant shape features and their potentially affecting factors. To this end long established morphometric techniques are combined with recent shape modeling approaches from geometry processing and medical imaging, leading to novel visual analytics methods for shape analysis. When used in concert these methods facilitate targeted analysis of characteristic shape differences between groups, co-variation between different structures on the same anatomy and correlation of shape to extrinsic attributes. Here a special focus is put on accurate modeling and interactive rendering of image deformations at high spatial resolution, because that allows for faithful representation and communication of diminutive shape features, large shape differences and volumetric structures. The utility of the presented methods is demonstrated in case studies conducted together with a collaborating morphometrics expert. As exemplary model structure serves the rodent skull and its mandible that are assessed via computed tomography scans

Automatic Affine and Elastic Registration Strategies for Multi-dimensional Medical Images

Medical images have been used increasingly for diagnosis, treatment planning, monitoring disease processes, and other medical applications. A large variety of medical imaging modalities exists including CT, X-ray, MRI, Ultrasound, etc. Frequently a group of images need to be compared to one another and/or combined for research or cumulative purposes. In many medical studies, multiple images are acquired from subjects at different times or with different imaging modalities. Misalignment inevitably occurs, causing anatomical and/or functional feature shifts within the images. Computerized image registration (alignment) approaches can offer automatic and accurate image alignments without extensive user involvement and provide tools for visualizing combined images. This dissertation focuses on providing automatic image registration strategies. After a through review of existing image registration techniques, we identified two registration strategies that enhance the current field: (1) an automated rigid body and affine registration using voxel similarity measurements based on a sequential hybrid genetic algorithm, and (2) an automated deformable registration approach based upon a linear elastic finite element formulation. Both methods streamlined the registration process. They are completely automatic and require no user intervention. The proposed registration strategies were evaluated with numerous 2D and 3D MR images with a variety of tissue structures, orientations and dimensions. Multiple registration pathways were provided with guidelines for their applications. The sequential genetic algorithm mimics the pathway of an expert manually doing registration. Experiments demonstrated that the sequential genetic algorithm registration provides high alignment accuracy and is reliable for brain tissues. It avoids local minima/maxima traps of conventional optimization techniques, and does not require any preprocessing such as threshold, smoothing, segmentation, or definition of base points or edges. The elastic model was shown to be highly effective to accurately align areas of interest that are automatically extracted from the images, such as brains. Using a finite element method to get the displacement of each element node by applying a boundary mapping, this method provides an accurate image registration with excellent boundary alignment of each pair of slices and consequently align the entire volume automatically. This dissertation presented numerous volume alignments. Surface geometries were created directly from the aligned segmented images using the Multiple Material Marching Cubes algorithm. Using the proposed registration strategies, multiple subjects were aligned to a standard MRI reference, which is aligned to a segmented reference atlas. Consequently, multiple subjects are aligned to the segmented atlas and a full fMRI analysis is possible

Modeling Brain Circuitry over a Wide Range of Scales

If we are ever to unravel the mysteries of brain function at its most fundamental level, we will need a precise understanding of how its component neurons connect to each other. Electron Microscopes (EM) can now provide the nanometer resolution that is needed to image synapses, and therefore connections, while Light Microscopes (LM) see at the micrometer resolution required to model the 3D structure of the dendritic network. Since both the topology and the connection strength are integral parts of the brain's wiring diagram, being able to combine these two modalities is critically important. In fact, these microscopes now routinely produce high-resolution imagery in such large quantities that the bottleneck becomes automated processing and interpretation, which is needed for such data to be exploited to its full potential. In this paper, we briefly review the Computer Vision techniques we have developed at EPFL to address this need. They include delineating dendritic arbors from LM imagery, segmenting organelles from EM, and combining the two into a consistent representation

A biomechanical approach for real-time tracking of lung tumors during External Beam Radiation Therapy (EBRT)

Lung cancer is the most common cause of cancer related death in both men and women. Radiation therapy is widely used for lung cancer treatment. However, this method can be challenging due to respiratory motion. Motion modeling is a popular method for respiratory motion compensation, while biomechanics-based motion models are believed to be more robust and accurate as they are based on the physics of motion. In this study, we aim to develop a biomechanics-based lung tumor tracking algorithm which can be used during External Beam Radiation Therapy (EBRT). An accelerated lung biomechanical model can be used during EBRT only if its boundary conditions (BCs) are defined in a way that they can be updated in real-time. As such, we have developed a lung finite element (FE) model in conjunction with a Neural Networks (NNs) based method for predicting the BCs of the lung model from chest surface motion data. To develop the lung FE model for tumor motion prediction, thoracic 4D CT images of lung cancer patients were processed to capture the lung and diaphragm geometry, trans-pulmonary pressure, and diaphragm motion. Next, the chest surface motion was obtained through tracking the motion of the ribcage in 4D CT images. This was performed to simulate surface motion data that can be acquired using optical tracking systems. Finally, two feedforward NNs were developed, one for estimating the trans-pulmonary pressure and another for estimating the diaphragm motion from chest surface motion data. The algorithm development consists of four steps of: 1) Automatic segmentation of the lungs and diaphragm, 2) diaphragm motion modelling using Principal Component Analysis (PCA), 3) Developing the lung FE model, and 4) Using two NNs to estimate the trans-pulmonary pressure values and diaphragm motion from chest surface motion data. The results indicate that the Dice similarity coefficient between actual and simulated tumor volumes ranges from 0.76±0.04 to 0.91±0.01, which is favorable. As such, real-time lung tumor tracking during EBRT using the proposed algorithm is feasible. Hence, further clinical studies involving lung cancer patients to assess the algorithm performance are justified

Fast capture of textured full-body avatar with RGB-D cameras

We present a practical system which can provide a textured full-body avatar within three seconds. It uses sixteen RGB-depth (RGB-D) cameras, ten of which are arranged to capture the body, while six target the important head region. The configuration of the multiple cameras is formulated as a constraint-based minimum set space-covering problem, which is approximately solved by a heuristic algorithm. The camera layout determined can cover the fullbody surface of an adult, with geometric errors of less than 5 mm. After arranging the cameras, they are calibrated using a mannequin before scanning real humans. The 16 RGB-D images are all captured within 1 s, which both avoids the need for the subject to attempt to remain still for an uncomfortable period, and helps to keep pose changes between different cameras small. All scans are combined and processed to reconstruct the photo-realistic textured mesh in 2 s. During both system calibration and working capture of a real subject, the high-quality RGB information is exploited to assist geometric reconstruction and texture stitching optimization

SHREC'16: partial matching of deformable shapes

Matching deformable 3D shapes under partiality transformations is a challenging problem that has received limited focus in the computer vision and graphics communities. With this benchmark, we explore and thoroughly investigate the robustness of existing matching methods in this challenging task. Participants are asked to provide a point-to-point correspondence (either sparse or dense) between deformable shapes undergoing different kinds of partiality transformations, resulting in a total of 400 matching problems to be solved for each method - making this benchmark the biggest and most challenging of its kind. Five matching algorithms were evaluated in the contest; this paper presents the details of the dataset, the adopted evaluation measures, and shows thorough comparisons among all competing methods