Automatic Optimum Atlas Selection for Multi-Atlas Image Segmentation using Joint Label Fusion

Multi-atlas image segmentation using label fusion is one of the most accurate state of the art image segmentation techniques available for biomedical imaging applications. Motivated to achieve higher image segmentation accuracy, reduce computational costs and a continuously increasing atlas data size, a robust framework for optimum selection of atlases for label fusion is vital. Although believed not to be critical for weighted label fusion techniques by some works (Sabuncu, M. R. et al., 2010, [1]), others have shown that appropriate atlas selection has several merits and can improve multi-atlas image segmentation accuracy (Aljabar et al., 2009, [2], Van de Velde et al., 2016) [27]. This thesis proposed an automatic Optimum Atlas Selection (OAS) framework pre-label fusion step that improved image segmentation performance dice similarity scores using Joint Label Fusion (JLF) implementation by Wang et al, 2013, [3, 26]. A selection criterion based on a global majority voting fusion output image similarity comparison score was employed to select an optimum number of atlases out of all available atlases to perform the label fusion step. The OAS framework led to observed significant improvement in aphasia stroke heads magnetic resonance (MR) images segmentation accuracy in leave-one out validation tests by 1.79% (p = 0.005520) and 0.5% (p = 0.000656) utilizing a set of 7 homogenous stroke and 19 inhomogeneous atlas datasets respectively. Further, using comparatively limited atlas data size (19 atlases) composed of normal and stroke head MR images, t-tests showed no statistical significant difference in image segmentation performance dice scores using the proposed OAS protocol compared to using known automatic Statistical Parametric Mapping (SPM) plus a touchup algorithm protocol [4] for image segmentation (p = 0.49417). Thus, leading to the conclusions that the proposed OAS framework is an effective and suitable atlas selection protocol for multi-atlas image segmentation that improves brain MR image segmentation accuracy. It is comparably in performance to known image segmentation algorithms and can lead to reduced computation costs in large atlas data sets. With regards to future work, efforts to increase atlas data size and use of a more robust approach for determining the optimum selection threshold value and corresponding number of atlases to perform label fusion process can be explored to enhance overall image segmentation accuracy. Furthermore, for an unbiased performance comparison of the proposed OAS framework to other image segmentation algorithms, truly manually segmented atlas ground truth MR images and labels are needed

Adversarial Deformation Regularization for Training Image Registration Neural Networks

We describe an adversarial learning approach to constrain convolutional neural network training for image registration, replacing heuristic smoothness measures of displacement fields often used in these tasks. Using minimally-invasive prostate cancer intervention as an example application, we demonstrate the feasibility of utilizing biomechanical simulations to regularize a weakly-supervised anatomical-label-driven registration network for aligning pre-procedural magnetic resonance (MR) and 3D intra-procedural transrectal ultrasound (TRUS) images. A discriminator network is optimized to distinguish the registration-predicted displacement fields from the motion data simulated by finite element analysis. During training, the registration network simultaneously aims to maximize similarity between anatomical labels that drives image alignment and to minimize an adversarial generator loss that measures divergence between the predicted- and simulated deformation. The end-to-end trained network enables efficient and fully-automated registration that only requires an MR and TRUS image pair as input, without anatomical labels or simulated data during inference. 108 pairs of labelled MR and TRUS images from 76 prostate cancer patients and 71,500 nonlinear finite-element simulations from 143 different patients were used for this study. We show that, with only gland segmentation as training labels, the proposed method can help predict physically plausible deformation without any other smoothness penalty. Based on cross-validation experiments using 834 pairs of independent validation landmarks, the proposed adversarial-regularized registration achieved a target registration error of 6.3 mm that is significantly lower than those from several other regularization methods.Comment: Accepted to MICCAI 201

Label-driven weakly-supervised learning for multimodal deformable image registration

Spatially aligning medical images from different modalities remains a challenging task, especially for intraoperative applications that require fast and robust algorithms. We propose a weakly-supervised, label-driven formulation for learning 3D voxel correspondence from higher-level label correspondence, thereby bypassing classical intensity-based image similarity measures. During training, a convolutional neural network is optimised by outputting a dense displacement field (DDF) that warps a set of available anatomical labels from the moving image to match their corresponding counterparts in the fixed image. These label pairs, including solid organs, ducts, vessels, point landmarks and other ad hoc structures, are only required at training time and can be spatially aligned by minimising a cross-entropy function of the warped moving label and the fixed label. During inference, the trained network takes a new image pair to predict an optimal DDF, resulting in a fully-automatic, label-free, real-time and deformable registration. For interventional applications where large global transformation prevails, we also propose a neural network architecture to jointly optimise the global- and local displacements. Experiment results are presented based on cross-validating registrations of 111 pairs of T2-weighted magnetic resonance images and 3D transrectal ultrasound images from prostate cancer patients with a total of over 4000 anatomical labels, yielding a median target registration error of 4.2 mm on landmark centroids and a median Dice of 0.88 on prostate glands.Comment: Accepted to ISBI 201

Bayesian Spatial Binary Regression for Label Fusion in Structural Neuroimaging

Many analyses of neuroimaging data involve studying one or more regions of interest (ROIs) in a brain image. In order to do so, each ROI must first be identified. Since every brain is unique, the location, size, and shape of each ROI varies across subjects. Thus, each ROI in a brain image must either be manually identified or (semi-) automatically delineated, a task referred to as segmentation. Automatic segmentation often involves mapping a previously manually segmented image to a new brain image and propagating the labels to obtain an estimate of where each ROI is located in the new image. A more recent approach to this problem is to propagate labels from multiple manually segmented atlases and combine the results using a process known as label fusion. To date, most label fusion algorithms either employ voting procedures or impose prior structure and subsequently find the maximum a posteriori estimator (i.e., the posterior mode) through optimization. We propose using a fully Bayesian spatial regression model for label fusion that facilitates direct incorporation of covariate information while making accessible the entire posterior distribution. We discuss the implementation of our model via Markov chain Monte Carlo and illustrate the procedure through both simulation and application to segmentation of the hippocampus, an anatomical structure known to be associated with Alzheimer's disease.Comment: 24 pages, 10 figure

Movie Description

Audio Description (AD) provides linguistic descriptions of movies and allows visually impaired people to follow a movie along with their peers. Such descriptions are by design mainly visual and thus naturally form an interesting data source for computer vision and computational linguistics. In this work we propose a novel dataset which contains transcribed ADs, which are temporally aligned to full length movies. In addition we also collected and aligned movie scripts used in prior work and compare the two sources of descriptions. In total the Large Scale Movie Description Challenge (LSMDC) contains a parallel corpus of 118,114 sentences and video clips from 202 movies. First we characterize the dataset by benchmarking different approaches for generating video descriptions. Comparing ADs to scripts, we find that ADs are indeed more visual and describe precisely what is shown rather than what should happen according to the scripts created prior to movie production. Furthermore, we present and compare the results of several teams who participated in a challenge organized in the context of the workshop "Describing and Understanding Video & The Large Scale Movie Description Challenge (LSMDC)", at ICCV 2015