Anatomical Shape and Motion Reconstruction from Sparse Image Data

In current clinical practice, medical imaging plays a key role in diagnosis, therapy planning and therapy monitoring. Some of these modalities, such as CT, MRI, and 3D ultrasound, provide high resolution volumetric anatomical information, and more recently, 3D imaging in time. In certain practical situations, however, limitations with respect to imaging time, space, radiation dose, or ergonomics make it impossible to acquire such rich data. In such cases, imaging may be performed that is of lower dimensionality than the desired information, or is sparse in at least one of the dimensions. This type of sparse imaging is investigated in this thesis. Sparse imaging is typically employed in image guided interventions and surgeries, where high speed, high image resolution and an open acquisition setup are of major importance. The 3D position of the surgical instruments with respect to the 3D patient anatomy is then assessed through 2D imaging such as X-ray fluoroscopy or Ultra-soun. For similar reasons mono- and biplane X-ray fluoroscopy became the standard for kinematic analysis of joints, allowing the acquisition of a wide range of motions, such as running and jumping. In other situations, radiation dose and cost reduction play a mayor role in employing sparse imaging. Bone surface reconstruction from points pin-pointed during knee surgery e.g. has been investigated for replacing prior CT acquisition. Also, assessment of organ motion, such as cardiac or respiratory motion, can occur from temporally sparse CT images

Exploiting temporality for semi-supervised video segmentation

In recent years, there has been remarkable progress in supervised image segmentation. Video segmentation is less explored, despite the temporal dimension being highly informative. Semantic labels, e.g. that cannot be accurately detected in the current frame, may be inferred by incorporating information from previous frames. However, video segmentation is challenging due to the amount of data that needs to be processed and, more importantly, the cost involved in obtaining ground truth annotations for each frame. In this paper, we tackle the issue of label scarcity by using consecutive frames of a video, where only one frame is annotated. We propose a deep, end-to-end trainable model which leverages temporal information in order to make use of easy to acquire unlabeled data. Our network architecture relies on a novel interconnection of two components: a fully convolutional network to model spatial information and temporal units that are employed at intermediate levels of the convolutional network in order to propagate information through time. The main contribution of this work is the guidance of the temporal signal through the network. We show that only placing a temporal module between the encoder and decoder is suboptimal (baseline). Our extensive experiments on the CityScapes dataset indicate that the resulting model can leverage unlabeled temporal frames and significantly outperform both the frame-by-frame image segmentation and the baseline approach.Comment: Accepted as workshop paper at ICCV 201

Oriented Gaussian Mixture Models for Nonrigid 2D/3D Coronary Artery Registration

2D/3D registration of patient vasculature from preinterventional computed tomography angiography (CTA) to interventional X-ray angiography is of interest to improve guidance in percutaneous coronary interventions. In this paper we present a novel feature based 2D/3D registration framework, that is based on probabilistic point correspondences, and show its usefulness on aligning 3D coronary artery centerlines derived from CTA images with their 2D projection derived from interventional X-ray angiography. The registration framework is an extension of the Gaussian mixture model (GMM) based point-set registration to the 2D/3D setting, with a modified distance metric. We also propose a way to incorporate orientation in the registration, and show its added value for artery registration on patient datasets as well as in simulation experiments. The oriented GMM registration achieved a median accuracy of 1.06 mm, with a convergence rate of 81% for nonrigid vessel centerline registration on 12 patient datasets, using a statistical shape model. The method thereby outperformed the iterative closest point algorithm, the GMM registration without orientation, and two recently published methods on 2D/3D coronary artery registration

Registration of 3D+t Coronary CTA and Monoplane 2D+t X-Ray Angiography

A method for registering preoperative 3D + t coronary CTA with intraoperative monoplane 2D + t X-ray angiography images is proposed to improve image guidance during minimally invasive coronary interventions. The method uses a patient-specific dynamic coronary model, which is derived from the CTA scan by centerline extraction and motion estimation. The dynamic coronary model is registered with the 2D + t X-ray sequence, considering multiple X-ray time points concurrently, while taking breathing induced motion into account. Evaluation was performed on 26 datasets of 17 patients by comparing projected model centerlines with manually annotated centerlines in the X-ray images. The proposed 3D + t/2D + t registration method performed better than a 3D/2D registration method with respect to the accuracy and especially the robustness of the registration. Registration with a median error of 1.47 mm was achieved