Deep learning cardiac motion analysis for human survival prediction

Motion analysis is used in computer vision to understand the behaviour of moving objects in sequences of images. Optimising the interpretation of dynamic biological systems requires accurate and precise motion tracking as well as efficient representations of high-dimensional motion trajectories so that these can be used for prediction tasks. Here we use image sequences of the heart, acquired using cardiac magnetic resonance imaging, to create time-resolved three-dimensional segmentations using a fully convolutional network trained on anatomical shape priors. This dense motion model formed the input to a supervised denoising autoencoder (4Dsurvival), which is a hybrid network consisting of an autoencoder that learns a task-specific latent code representation trained on observed outcome data, yielding a latent representation optimised for survival prediction. To handle right-censored survival outcomes, our network used a Cox partial likelihood loss function. In a study of 302 patients the predictive accuracy (quantified by Harrell's C-index) was significantly higher (p < .0001) for our model C=0.73 (95$\%$ CI: 0.68 - 0.78) than the human benchmark of C=0.59 (95$\%$ CI: 0.53 - 0.65). This work demonstrates how a complex computer vision task using high-dimensional medical image data can efficiently predict human survival

Comparing landmarking methods for face recognition

Good registration (alignment to a reference) is essential for accurate face recognition. We use the locations of facial features (eyes, nose, mouth, etc) as landmarks for registration. Two landmarking methods are explored and compared: (1) the Most Likely-Landmark Locator (MLLL), based on maximizing the likelihood ratio [1], and (2) Viola-Jones detection [2]. Further, a landmark-correction method based on projection into a subspace is introduced. Both landmarking methods have been trained on the landmarked images in the BioID database [3]. The MLLL has been trained for locating 17 landmarks and the Viola-Jones method for 5 landmarks. The localization error and effects on the equal-error rate (EER) have been measured. In these experiments ground- truth data has been used as a reference. The results are described as follows:\ud 1. The localization errors obtained on the FRGC database are 4.2, 8.6 and 4.6 pixels for the Viola-Jones, the MLLL, and the MLLL after landmark correction, respectively. The inter-eye distance of the reference face is 100 pixels. The MLLL with landmark correction scores best in the verification experiment.\ud 2. Using more landmarks decreases the average localization error and the EER

Fitting a 3D Morphable Model to Edges: A Comparison Between Hard and Soft Correspondences

We propose a fully automatic method for fitting a 3D morphable model to single face images in arbitrary pose and lighting. Our approach relies on geometric features (edges and landmarks) and, inspired by the iterated closest point algorithm, is based on computing hard correspondences between model vertices and edge pixels. We demonstrate that this is superior to previous work that uses soft correspondences to form an edge-derived cost surface that is minimised by nonlinear optimisation.Comment: To appear in ACCV 2016 Workshop on Facial Informatic

Automatic 3D bi-ventricular segmentation of cardiac images by a shape-refined multi-task deep learning approach

Deep learning approaches have achieved state-of-the-art performance in cardiac magnetic resonance (CMR) image segmentation. However, most approaches have focused on learning image intensity features for segmentation, whereas the incorporation of anatomical shape priors has received less attention. In this paper, we combine a multi-task deep learning approach with atlas propagation to develop a shape-constrained bi-ventricular segmentation pipeline for short-axis CMR volumetric images. The pipeline first employs a fully convolutional network (FCN) that learns segmentation and landmark localisation tasks simultaneously. The architecture of the proposed FCN uses a 2.5D representation, thus combining the computational advantage of 2D FCNs networks and the capability of addressing 3D spatial consistency without compromising segmentation accuracy. Moreover, the refinement step is designed to explicitly enforce a shape constraint and improve segmentation quality. This step is effective for overcoming image artefacts (e.g. due to different breath-hold positions and large slice thickness), which preclude the creation of anatomically meaningful 3D cardiac shapes. The proposed pipeline is fully automated, due to network's ability to infer landmarks, which are then used downstream in the pipeline to initialise atlas propagation. We validate the pipeline on 1831 healthy subjects and 649 subjects with pulmonary hypertension. Extensive numerical experiments on the two datasets demonstrate that our proposed method is robust and capable of producing accurate, high-resolution and anatomically smooth bi-ventricular 3D models, despite the artefacts in input CMR volumes

Distribution on Warp Maps for Alignment of Open and Closed Curves

Alignment of curve data is an integral part of their statistical analysis, and can be achieved using model- or optimization-based approaches. The parameter space is usually the set of monotone, continuous warp maps of a domain. Infinite-dimensional nature of the parameter space encourages sampling based approaches, which require a distribution on the set of warp maps. Moreover, the distribution should also enable sampling in the presence of important landmark information on the curves which constrain the warp maps. For alignment of closed and open curves in $\mathbb{R}^d, d=1,2,3$, possibly with landmark information, we provide a constructive, point-process based definition of a distribution on the set of warp maps of $[0,1]$ and the unit circle $\mathbb{S}^1$ that is (1) simple to sample from, and (2) possesses the desiderata for decomposition of the alignment problem with landmark constraints into multiple unconstrained ones. For warp maps on $[0,1]$, the distribution is related to the Dirichlet process. We demonstrate its utility by using it as a prior distribution on warp maps in a Bayesian model for alignment of two univariate curves, and as a proposal distribution in a stochastic algorithm that optimizes a suitable alignment functional for higher-dimensional curves. Several examples from simulated and real datasets are provided