Ball-Scale Based Hierarchical Multi-Object Recognition in 3D Medical Images

This paper investigates, using prior shape models and the concept of ball scale (b-scale), ways of automatically recognizing objects in 3D images without performing elaborate searches or optimization. That is, the goal is to place the model in a single shot close to the right pose (position, orientation, and scale) in a given image so that the model boundaries fall in the close vicinity of object boundaries in the image. This is achieved via the following set of key ideas: (a) A semi-automatic way of constructing a multi-object shape model assembly. (b) A novel strategy of encoding, via b-scale, the pose relationship between objects in the training images and their intensity patterns captured in b-scale images. (c) A hierarchical mechanism of positioning the model, in a one-shot way, in a given image from a knowledge of the learnt pose relationship and the b-scale image of the given image to be segmented. The evaluation results on a set of 20 routine clinical abdominal female and male CT data sets indicate the following: (1) Incorporating a large number of objects improves the recognition accuracy dramatically. (2) The recognition algorithm can be thought as a hierarchical framework such that quick replacement of the model assembly is defined as coarse recognition and delineation itself is known as finest recognition. (3) Scale yields useful information about the relationship between the model assembly and any given image such that the recognition results in a placement of the model close to the actual pose without doing any elaborate searches or optimization. (4) Effective object recognition can make delineation most accurate.Comment: This paper was published and presented in SPIE Medical Imaging 201

Groupwise Multimodal Image Registration using Joint Total Variation

In medical imaging it is common practice to acquire a wide range of modalities (MRI, CT, PET, etc.), to highlight different structures or pathologies. As patient movement between scans or scanning session is unavoidable, registration is often an essential step before any subsequent image analysis. In this paper, we introduce a cost function based on joint total variation for such multimodal image registration. This cost function has the advantage of enabling principled, groupwise alignment of multiple images, whilst being insensitive to strong intensity non-uniformities. We evaluate our algorithm on rigidly aligning both simulated and real 3D brain scans. This validation shows robustness to strong intensity non-uniformities and low registration errors for CT/PET to MRI alignment. Our implementation is publicly available at https://github.com/brudfors/coregistration-njtv

Adversarial Deformation Regularization for Training Image Registration Neural Networks

We describe an adversarial learning approach to constrain convolutional neural network training for image registration, replacing heuristic smoothness measures of displacement fields often used in these tasks. Using minimally-invasive prostate cancer intervention as an example application, we demonstrate the feasibility of utilizing biomechanical simulations to regularize a weakly-supervised anatomical-label-driven registration network for aligning pre-procedural magnetic resonance (MR) and 3D intra-procedural transrectal ultrasound (TRUS) images. A discriminator network is optimized to distinguish the registration-predicted displacement fields from the motion data simulated by finite element analysis. During training, the registration network simultaneously aims to maximize similarity between anatomical labels that drives image alignment and to minimize an adversarial generator loss that measures divergence between the predicted- and simulated deformation. The end-to-end trained network enables efficient and fully-automated registration that only requires an MR and TRUS image pair as input, without anatomical labels or simulated data during inference. 108 pairs of labelled MR and TRUS images from 76 prostate cancer patients and 71,500 nonlinear finite-element simulations from 143 different patients were used for this study. We show that, with only gland segmentation as training labels, the proposed method can help predict physically plausible deformation without any other smoothness penalty. Based on cross-validation experiments using 834 pairs of independent validation landmarks, the proposed adversarial-regularized registration achieved a target registration error of 6.3 mm that is significantly lower than those from several other regularization methods.Comment: Accepted to MICCAI 201

Real-time High Resolution Fusion of Depth Maps on GPU

A system for live high quality surface reconstruction using a single moving depth camera on a commodity hardware is presented. High accuracy and real-time frame rate is achieved by utilizing graphics hardware computing capabilities via OpenCL and by using sparse data structure for volumetric surface representation. Depth sensor pose is estimated by combining serial texture registration algorithm with iterative closest points algorithm (ICP) aligning obtained depth map to the estimated scene model. Aligned surface is then fused into the scene. Kalman filter is used to improve fusion quality. Truncated signed distance function (TSDF) stored as block-based sparse buffer is used to represent surface. Use of sparse data structure greatly increases accuracy of scanned surfaces and maximum scanning area. Traditional GPU implementation of volumetric rendering and fusion algorithms were modified to exploit sparsity to achieve desired performance. Incorporation of texture registration for sensor pose estimation and Kalman filter for measurement integration improved accuracy and robustness of scanning process

Automatic 3D bi-ventricular segmentation of cardiac images by a shape-refined multi-task deep learning approach

Deep learning approaches have achieved state-of-the-art performance in cardiac magnetic resonance (CMR) image segmentation. However, most approaches have focused on learning image intensity features for segmentation, whereas the incorporation of anatomical shape priors has received less attention. In this paper, we combine a multi-task deep learning approach with atlas propagation to develop a shape-constrained bi-ventricular segmentation pipeline for short-axis CMR volumetric images. The pipeline first employs a fully convolutional network (FCN) that learns segmentation and landmark localisation tasks simultaneously. The architecture of the proposed FCN uses a 2.5D representation, thus combining the computational advantage of 2D FCNs networks and the capability of addressing 3D spatial consistency without compromising segmentation accuracy. Moreover, the refinement step is designed to explicitly enforce a shape constraint and improve segmentation quality. This step is effective for overcoming image artefacts (e.g. due to different breath-hold positions and large slice thickness), which preclude the creation of anatomically meaningful 3D cardiac shapes. The proposed pipeline is fully automated, due to network's ability to infer landmarks, which are then used downstream in the pipeline to initialise atlas propagation. We validate the pipeline on 1831 healthy subjects and 649 subjects with pulmonary hypertension. Extensive numerical experiments on the two datasets demonstrate that our proposed method is robust and capable of producing accurate, high-resolution and anatomically smooth bi-ventricular 3D models, despite the artefacts in input CMR volumes

White Matter Structural Connectivity is Associated with Sensorimotor Function in Stroke Survivors

Purpose Diffusion tensor imaging (DTI) provides functionally relevant information about white matter structure. Local anatomical connectivity information combined with fractional anisotropy (FA) and mean diffusivity (MD) may predict functional outcomes in stroke survivors. Imaging methods for predicting functional outcomes in stroke survivors are not well established. This work uses DTI to objectively assess the effects of a stroke lesion on white matter structure and sensorimotor function. Methods A voxel-based approach is introduced to assess a stroke lesion\u27s global impact on motor function. Anatomical T1-weighted and diffusion tensor images of the brain were acquired for nineteen subjects (10 post-stroke and 9 age-matched controls). A manually selected volume of interest was used to alleviate the effects of stroke lesions on image registration. Images from all subjects were registered to the images of the control subject that was anatomically closest to Talairach space. Each subject\u27s transformed image was uniformly seeded for DTI tractography. Each seed was inversely transformed into the individual subject space, where DTI tractography was conducted and then the results were transformed back to the reference space. A voxel-wise connectivity matrix was constructed from the fibers, which was then used to calculate the number of directly and indirectly connected neighbors of each voxel. A novel voxel-wise indirect structural connectivity (VISC) index was computed as the average number of direct connections to a voxel\u27s indirect neighbors. Voxel-based analyses (VBA) were performed to compare VISC, FA, and MD for the detection of lesion-induced changes in sensorimotor function. For each voxel, a t-value was computed from the differences between each stroke brain and the 9 controls. A series of linear regressions was performed between Fugl-Meyer (FM) assessment scores of sensorimotor impairment and each DTI metric\u27s log number of voxels that differed from the control group. Results Correlation between the logarithm of the number of significant voxels in the ipsilesional hemisphere and total Fugl-Meyer score was moderate for MD (R2 = 0.512), and greater for VISC (R2 = 0.796) and FA (R2 = 0.674). The slopes of FA (p = 0.0036), VISC (p = 0.0005), and MD (p = 0.0199) versus the total FM score were significant. However, these correlations were driven by the upper extremity motor component of the FM score (VISC: R2 = 0.879) with little influence of the lower extremity motor component (FA: R2 = 0.177). Conclusion The results suggest that a voxel-wise metric based on DTI tractography can predict upper extremity sensorimotor function of stroke survivors, and that supraspinal intraconnectivity may have a less dominant role in lower extremity function