A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration

Image registration, the process of aligning two or more images, is the core technique of many (semi-)automatic medical image analysis tasks. Recent studies have shown that deep learning methods, notably convolutional neural networks (ConvNets), can be used for image registration. Thus far training of ConvNets for registration was supervised using predefined example registrations. However, obtaining example registrations is not trivial. To circumvent the need for predefined examples, and thereby to increase convenience of training ConvNets for image registration, we propose the Deep Learning Image Registration (DLIR) framework for \textit{unsupervised} affine and deformable image registration. In the DLIR framework ConvNets are trained for image registration by exploiting image similarity analogous to conventional intensity-based image registration. After a ConvNet has been trained with the DLIR framework, it can be used to register pairs of unseen images in one shot. We propose flexible ConvNets designs for affine image registration and for deformable image registration. By stacking multiple of these ConvNets into a larger architecture, we are able to perform coarse-to-fine image registration. We show for registration of cardiac cine MRI and registration of chest CT that performance of the DLIR framework is comparable to conventional image registration while being several orders of magnitude faster.Comment: Accepted: Medical Image Analysis - Elsevie

A New Methodology for Multiscale Myocardial Deformation and Strain Analysis Based on Tagging MRI

Myocardial deformation and strain can be investigated using suitably encoded cine MRI that admits disambiguation of material motion. Practical limitations currently restrict the analysis to in-plane motion in cross-sections of the heart (2D + time), but the proposed method readily generalizes to 3D + time. We propose a new, promising methodology, which departs from a multiscale algorithm that exploits local scale selection so as to obtain a robust estimate for the velocity gradient tensor field. Time evolution of the deformation tensor is governed by a first-order ordinary differential equation, which is completely determined by this velocity gradient tensor field. We solve this matrix-ODE analytically and present results obtained from healthy volunteers as well as from patient data. The proposed method requires only off-the-shelf algorithms and is readily applicable to planar or volumetric tagging MRI sampled on arbitrary coordinate grids

Registration of Brain MRI/PET Images Based on Adaptive Combination of Intensity and Gradient Field Mutual Information

Traditional mutual information (MI) function aligns two multimodality images with intensity information, lacking spatial information, so that it usually presents many local maxima that can lead to inaccurate registration. Our paper proposes an algorithm of adaptive combination of intensity and gradient field mutual information (ACMI). Gradient code maps (GCM) are constructed by coding gradient field information of corresponding original images. The gradient field MI, calculated from GCMs, can provide complementary properties to intensity MI. ACMI combines intensity MI and gradient field MI with a nonlinear weight function, which can automatically adjust the proportion between two types MI in combination to improve registration. Experimental results demonstrate that ACMI outperforms the traditional MI and it is much less sensitive to reduced resolution or overlap of images

Mjolnir: Extending HAMMER Using a Diffusion Transformation Model and Histogram Equalization for Deformable Image Registration

Image registration is a crucial step in many medical image analysis procedures such as image fusion, surgical planning, segmentation and labeling, and shape comparison in population or longitudinal studies. A new approach to volumetric intersubject deformable image registration is presented. The method, called Mjolnir, is an extension of the highly successful method HAMMER. New image features in order to better localize points of correspondence between the two images are introduced as well as a novel approach to generate a dense displacement field based upon the weighted diffusion of automatically derived feature correspondences. An extensive validation of the algorithm was performed on T1-weighted SPGR MR brain images from the NIREP evaluation database. The results were compared with results generated by HAMMER and are shown to yield significant improvements in cortical alignment as well as reduced computation time

A method for dynamic subtraction MR imaging of the liver

BACKGROUND: Subtraction of Dynamic Contrast-Enhanced 3D Magnetic Resonance (DCE-MR) volumes can result in images that depict and accurately characterize a variety of liver lesions. However, the diagnostic utility of subtraction images depends on the extent of co-registration between non-enhanced and enhanced volumes. Movement of liver structures during acquisition must be corrected prior to subtraction. Currently available methods are computer intensive. We report a new method for the dynamic subtraction of MR liver images that does not require excessive computer time. METHODS: Nineteen consecutive patients (median age 45 years; range 37–67) were evaluated by VIBE T1-weighted sequences (TR 5.2 ms, TE 2.6 ms, flip angle 20°, slice thickness 1.5 mm) acquired before and 45s after contrast injection. Acquisition parameters were optimized for best portal system enhancement. Pre and post-contrast liver volumes were realigned using our 3D registration method which combines: (a) rigid 3D translation using maximization of normalized mutual information (NMI), and (b) fast 2D non-rigid registration which employs a complex discrete wavelet transform algorithm to maximize pixel phase correlation and perform multiresolution analysis. Registration performance was assessed quantitatively by NMI. RESULTS: The new registration procedure was able to realign liver structures in all 19 patients. NMI increased by about 8% after rigid registration (native vs. rigid registration 0.073 ± 0.031 vs. 0.078 ± 0.031, n.s., paired t-test) and by a further 23% (0.096 ± 0.035 vs. 0.078 ± 0.031, p < 0.001, paired t-test) after non-rigid realignment. The overall average NMI increase was 31%. CONCLUSION: This new method for realigning dynamic contrast-enhanced 3D MR volumes of liver leads to subtraction images that enhance diagnostic possibilities for liver lesions

Segmentation of Myocardial Boundaries in Tagged Cardiac MRI Using Active Contours: A Gradient-Based Approach Integrating Texture Analysis

The noninvasive assessment of cardiac function is of first importance for the diagnosis of cardiovascular diseases. Among all medical scanners only a few enables radiologists to evaluate the local cardiac motion. Tagged cardiac MRI is one of them. This protocol generates on Short-Axis (SA) sequences a dark grid which is deformed in accordance with the cardiac motion. Tracking the grid allows specialists a local estimation of cardiac geometrical parameters within myocardium. The work described in this paper aims to automate the myocardial contours detection in order to optimize the detection and the tracking of the grid of tags within myocardium. The method we have developed for endocardial and epicardial contours detection is based on the use of texture analysis and active contours models. Texture analysis allows us to define energy maps more efficient than those usually used in active contours methods where attractor is often based on gradient and which were useless in our case of study, for quality of tagged cardiac MRI is very poor

3D Rigid Registration of Intraoperative Ultrasound and Preoperative MR Brain Images Based on Hyperechogenic Structures

The registration of intraoperative ultrasound (US) images with preoperative magnetic resonance (MR) images is a challenging problem due to the difference of information contained in each image modality. To overcome this difficulty, we introduce a new probabilistic function based on the matching of cerebral hyperechogenic structures. In brain imaging, these structures are the liquid interfaces such as the cerebral falx and the sulci, and the lesions when the corresponding tissue is hyperechogenic. The registration procedure is achieved by maximizing the joint probability for a voxel to be included in hyperechogenic structures in both modalities. Experiments were carried out on real datasets acquired during neurosurgical procedures. The proposed validation framework is based on (i) visual assessment, (ii) manual expert estimations , and (iii) a robustness study. Results show that the proposed method (i) is visually efficient, (ii) produces no statistically different registration accuracy compared to manual-based expert registration, and (iii) converges robustly. Finally, the computation time required by our method is compatible with intraoperative use

A Review on MR Image Intensity Inhomogeneity Correction

Intensity inhomogeneity (IIH) is often encountered in MR imaging, and a number of techniques have been devised to correct this artifact. This paper attempts to review some of the recent developments in the mathematical modeling of IIH field. Low-frequency models are widely used, but they tend to corrupt the low-frequency components of the tissue. Hypersurface models and statistical models can be adaptive to the image and generally more stable, but they are also generally more complex and consume more computer memory and CPU time. They are often formulated together with image segmentation within one framework and the overall performance is highly dependent on the segmentation process. Beside these three popular models, this paper also summarizes other techniques based on different principles. In addition, the issue of quantitative evaluation and comparative study are discussed

Automation of Hessian-Based Tubularity Measure Response Function in 3D Biomedical Images

The blood vessels and nerve trees consist of tubular objects interconnected into a complex tree- or web-like structure that has a range of structural scale 5 μm diameter capillaries to 3 cm aorta. This large-scale range presents two major problems; one is just making the measurements, and the other is the exponential increase of component numbers with decreasing scale. With the remarkable increase in the volume imaged by, and resolution of, modern day 3D imagers, it is almost impossible to make manual tracking of the complex multiscale parameters from those large image data sets. In addition, the manual tracking is quite subjective and unreliable. We propose a solution for automation of an adaptive nonsupervised system for tracking tubular objects based on multiscale framework and use of Hessian-based object shape detector incorporating National Library of Medicine Insight Segmentation and Registration Toolkit (ITK) image processing libraries

3D Kidney Segmentation from Abdominal Images Using Spatial-Appearance Models

Kidney segmentation is an essential step in developing any noninvasive computer-assisted diagnostic system for renal function assessment. This paper introduces an automated framework for 3D kidney segmentation from dynamic computed tomography (CT) images that integrates discriminative features from the current and prior CT appearances into a random forest classification approach. To account for CT images’ inhomogeneities, we employ discriminate features that are extracted from a higher-order spatial model and an adaptive shape model in addition to the first-order CT appearance. To model the interactions between CT data voxels, we employed a higher-order spatial model, which adds the triple and quad clique families to the traditional pairwise clique family. The kidney shape prior model is built using a set of training CT data and is updated during segmentation using not only region labels but also voxels’ appearances in neighboring spatial voxel locations. Our framework performance has been evaluated on in vivo dynamic CT data collected from 20 subjects and comprises multiple 3D scans acquired before and after contrast medium administration. Quantitative evaluation between manually and automatically segmented kidney contours using Dice similarity, percentage volume differences, and 95th-percentile bidirectional Hausdorff distances confirms the high accuracy of our approach