The demons algorithm is a fast non-parametric non-rigid registration method. In recent years great efforts have been made to improve the approach; the state of the art version yields symmetric inverse-consistent largedeformation diffeomorphisms. However, only limited work has explored inter-modal similarity metrics, with no practical evaluation on multi-modality data. We present a diffeomorphic demons implementation using the analytical gradient of Normalised Mutual Information (NMI) in a conjugate gradient optimiser. We report the first qualitative and quantitative assessment of the demons for inter-modal registration. Experiments to spatially normalise real MR images, and to recover simulated deformation fields, demonstrate (i) similar accuracy from NMI-demons and classical demons when the latter may be used, and (ii) similar accuracy for NMI-demons on T1w-T1w and T1w-T2w registration, demonstrating its potential in multi-modal scenarios

Fox, N.C.

Hawkes, D.J.

Modat, M.

Ourselin, S.

Ridgway, G.R.

Vercauteren, T.

English

UCL Discovery

Diﬀeomorphic Demons using Normalised MutualInformation, Evaluation on Multi-Modal Brain MR ImagesM. Modata, T. Vercauterenb, G.R. Ridgwaya, D.J. Hawkesa, N.C. Foxa and S. OurselinaaUniversity College London, UK. bMauna Kea Technologies, Paris, France.ABSTRACTThe demons algorithm is a fast non-parametric non-rigid registration method. In recent years great eﬀortshave been made to improve the approach; the state of the art version yields symmetric inverse-consistent large-deformation diﬀeomorphisms. However, only limited work has explored inter-modal similarity metrics, withno practical evaluation on multi-modality data. We present a diﬀeomorphic demons implementation using theanalytical gradient of Normalised Mutual Information (NMI) in a conjugate gradient optimiser. We report theﬁrst qualitative and quantitative assessment of the demons for inter-modal registration. Experiments to spatiallynormalise real MR images, and to recover simulated deformation ﬁelds, demonstrate (i) similar accuracy fromNMI-demons and classical demons when the latter may be used, and (ii) similar accuracy for NMI-demons onT1w–T1w and T1w–T2w registration, demonstrating its potential in multi-modal scenarios.1. INTRODUCTIONThe demons algorithm1 is a well-established technique for non-rigid registration. Its key advantage over com-peting algorithms like ﬂuid2 and Free-Form Deformation (FFD)3 is computational eﬃciency. Recent work hasimproved the demons’ registration quality and its applicability. The diﬀeomorphic approach4 makes the al-gorithm suitable for statistical morphometric analyses that require topology-preservation. The symmetric andinverse-consistent formulation5 enhances reliability for longitudinal data, and permits consistent forward andinverse transformations (e.g. allowing both voxel-wise and vertex-wise data to be transformed). These modi-ﬁcations have been accomplished without unduly increasing the demons’ computation time, thus maintainingits speed advantage. However, the demons framework has not yet been shown to be suitable for multi-modalapplications; while some papers have proposed Mutual Information as a similarity measure,6–8 they have shownno successful results on practical inter-modal examples.We present a novel implementation of the diﬀeomorphic demons using Normalised Mutual Information (NMI)with its analytical gradient in a non-linear conjugate gradient optimiser. Our evaluation employs simulated datafrom BrainWeb,9 using its perfectly aligned T1- and T2-weighted images to perform several related comparisons.Firstly, we compare the performance of atlas construction when registering T1w images to the T1w BrainWebtemplate using the conventional mean-squared error (MSE) demons algorithm and our new NMI-demons. Wethen compare the latter to the novel use of the NMI-demons on registering the same T1w images to the T2wtemplate. In a third experiment, we ﬁrst use an FFD algorithm to register the BrainWeb T1w image to anumber of diﬀerent subjects’ T1w scans, we then apply the estimated transformations to the T1w BrainWebimage, simulating a set of images with known ground-truth correspondence to BrainWeb. This allows quanti-tative measurement of the error in registering these images back to either the T1w or T2w BrainWeb originals,evaluating both intra- and inter-modal accuracy.2. METHOD2.1 Diﬀeomorphic demonsFor ﬁxed image F and moving image M, typical non-linear registration algorithms seek a spatial transformationg that optimizes a weighted average of an image similarity term Sim(F,M ◦ g) and a regularization term Reg(g).Coupling of these terms usually leads to computationally intensive optimization. In contrast, Thirion proposedto follow an unregularized optical-ﬂow step by a decoupled regularization using Gaussian smoothing.1 BySend correspondence to Marc Modat, E-mail: m.modat@ucl.ac.ukintroducing a hidden variable c that represents observed point correspondences, the demons algorithm can beseen as an optimization of a global energy,10 where a distance term dist(g,c) ensures g remains close to c:E(c,g) =1σ2iSim(F,M ◦ c) +1σ2xdist(g,c)2 +1σ2TReg(g). (1)The auxiliary variable justiﬁes the demons’ two step procedure as a well-posed alternate optimization overc and g. The correspondence step optimizes 1σ2iSim(F,M ◦ c) + 1σ2x dist(g,c)2, with respect to c, for ﬁxedg. This is typically done by a gradient descent step starting from c = g. The regularization step optimizes1σ2x dist(g,c)2 + 1σ2TReg(g), with respect to g, for ﬁxed c, typically by simple Gaussian smoothing. σi, σx and σTare weights that are not explicitly used in the practical implementation.The classical demons algorithm works on the complete space of dense non-parametric spatial transformationsand uses a second-order Newton-type optimisation (see §2.3) of an MSE criterion, Sim(F,M ◦ c) =  F − M ◦ c 2.This approach was extended4 to generate smooth topology-preserving maps by working on a space of diﬀeomor-phisms. Eﬃcient implementation is achieved by using diﬀeomorphisms that are the exponential map of stationaryvelocity ﬁelds, and hence can be computed rapidly using a scaling and squaring approach. In practice, the dif-feomorphic demons involves both the above-mentioned Gaussian smoothing (of the displacement ﬁeld, analogousto diﬀusion regularisation) and a second Gaussian smoothing of the velocity ﬁeld — a visco-diﬀusion model11analogous to the ﬂuid algorithm.2 There are two tunable parameters for the Gaussian widths, σdiﬀn and σﬂuid.2.2 Normalised mutual information and its gradientThe Normalised Mutual Information (NMI) similarity measure12 aims to quantify the amount of information oneimage has about an other. It is suitable for multi-modal applications as it does not rely on a simple functionalrelationship between intensities. NMI = (H(F) + H(M ◦ g))/H(F,M ◦ g), where H(F) and H(M ◦ g) are themarginal entropies of the ﬁxed and warped images, and H(F,M ◦g) is their joint entropy. To optimize NMI weuse its analytical gradient ∂NMI/∂µ for each degree of freedom µ, which we have previously derived.132.3 OptimisationAs previously mentioned, the classical demons optimises the MSE using a Newton-type method (multiplyingthe gradient by the inverse-Hessian). Because MSE is a local similarity measure, and the demons has a ‘non-parametric’ voxel-wise transformation, the Hessian consists only of 3 × 3 blocks on the diagonal. This can beseen by considering the gradient with respect to movement of one voxel; the rates of change of the x-, y- andz-components at that voxel may depend on each other but not on the motion of other voxels.This approach performs well, but it cannot be applied in the multi-modal case. NMI is a global metric whereall voxels with the same intensity are linked through the joint histogram, so the Hessian is much less sparse thanfor MSE. The Hessian’s size (square in the number of voxels — e.g. 106 ×106), therefore makes inversion almostimpossible, and we have found simple approximations of the NMI Hessian are inadequate. We therefore proposeconjugate gradient (CG) optimisation, requiring only the ﬁrst derivative of NMI. The interleaving of descentsteps and Gaussian smoothing means it is not theoretically clear that CG will work well, but in practice, wehave found that it signiﬁcantly improves on gradient descent.A disadvantage of CG compared to Newton-type optimisation is that an appropriate step-size must be found.Furthermore, the scale of the NMI gradient is not compatible with realistic displacement; in one example on 3Dbrain MRI, the update ﬁeld of MSE-demons showed displacements of ∼ 1voxel, while the maximal NMI-demonsstep was just 10−6. We therefore rescale the NMI gradient to have maximal displacement norm of 1voxel.3. EXPERIMENTS3.1 Comparison of the MSE- and NMI-demonsThe diﬀerence between the MSE criterion and the NMI is assessed using T1w brain images. Forty diagnosedAlzheimer’s disease (AD) patient scans and 18 age-matched control scans are registered aﬃnely and then non-rigidly to the (ﬁxed) BrainWeb T1w image. Scans were acquired on a 1.5T GE Signa scanner with an inversion-prepared fast SPGR sequence (TE=6.4ms, TI=650ms, TR=3s, bandwith=16kHz, 256×256×128 matrix, ﬁeldof view 240×240×186mm). We used AD patients and control subjects to provide a large variety of deformationsto recover (controls are quite similar to the target image, whereas the AD patients have much larger ventricles).Prior to any registration, a histogram-equalisation procedure non-linearly rescaled the voxel intensities between0 and 64 for every image (to match the ﬁxed image intensity histogram). Both MSE-demons and NMI-demonsused 3 coarse-to-ﬁne resolution levels. The number of iterations has been set to 50 for each level to ensureconvergence in both algorithms. Diﬀerent smoothing parameters have been selected for the two algorithms,based upon visual assessment of result and Jacobian images from test registrations. σﬂuid, has been set to 2 forboth versions of the demons whereas σdiﬀn is 2 for MSE-demons but 0.8 for NMI-demons.3.2 Comparison of mono- and multi-modal NMI-demonsThe 58 moving images described previously were all registered to the BrainWeb T2w image (aligned with the T1wﬁxed image used previously). The NMI-demons parameters were identical to the ones in the former experiment.3.3 Assessment on multi-modal casesUsing 10 controls and 10 diagnosed AD scans from the 58 previously described, we generated known deformationﬁelds. Firstly, the BrainWeb T1w image has been registered to each scan using the FFD. Secondly, the warpedimage was considered as ﬁxed and, using the MSE- and NMI-demons, the deformation ﬁeld was recovered fromeither the T1w or the T2w image. The T1w and T2w image being perfectly aligned, we had the possibility todirectly compare the obtained result with a ground truth deformation ﬁeld.4. RESULTS AND DISCUSSION4.1 Registration metricsThe MSE and NMI values were computed for each registered image and are summarized in the ﬁrst part of Table 1,along with the overall extrema of the Jacobian determinant for each method and modality. As might be expected,the NMI-demons leads to higher NMI than the MSE-demons and vice-versa, i.e. each registration is ‘best’ by itsown metric. The similarity of the ranges of the jacobian determinants suggests that the regularisation eﬀects aresimilar, supporting our choice of diﬀerent σﬂuid for MSE- and NMI-demons. Note that negative Jacobians canstill arise in the diﬀeomorphic demons due to discretisation eﬀects.4Aﬃne MSE-demons NMI-demons NMI-demons(T1w-T1w) (T1w-T1w) (T1w-T1w) (T1w-T2w)Registration of 58 T1wimages to BrainWebT1w/T2w templateMSE value 72.6 (11.3) 27.3 (3.9) 30.8(5.2)NMI value 1.094 (0.057) 1.149 (0.07) 1.161 (0.07) 1.159 (0.07)|Jac| extrema [-0.93 9.03] [0.04 8.30] [0.05 7.31]Recovery of 20 FFDsimulated deformations Absolute error0.4677 0.1025 0.5473(0.0235) (0.0318) (0.0255)Table 1. Quantitative registration results, presented as ‘mean (standard deviation)’ or ‘[minimum maximum]’.4.2 Atlasing resultsThe warped images from all subjects were averaged within-method and are displayed in Figure 1; all three arevery similar, demonstrating that both intra- and inter-modal NMI-demons give similar results to MSE-demons.It appears that the MSE-demons result preserves slightly less detail in ﬁner cortical folds. However, we onlyclaim there is no evidence for marked inferiority of the NMI-demons. We cannot suggest that the NMI criterionoutperforms the MSE one, because there are numerous diﬀerences in the two implementations: in σdiﬀn; in theuse of CG or Newton-type optimisers; and in their moving-image interpolation (our NMI-demons code uses cubicspline, while the original MSE-demons uses linear). It seems inevitable that MSE-demons would be preferredwhen the data truly satisfy the assumption of additive Gaussian white noise, but it is conceivable that NMI-demons might be better even for intra-modal data if there are minor acquisition diﬀerences, e.g. across scanners.BrainWeb T1w MSE mean NMI-T1w mean NMI-T2w mean BrainWeb T2wFigure 1. (b-d) Averages of 58 T1w brain MRI registered to (a) T1w, or (e) T2w, BrainWeb templates.4.3 Registration errorsThe bottom half of Table 1 presents results for the absolute errors at each voxel integrated over the brain area.It can be seen that the NMI-demons within modality performed better than the other two methods, which arethemselves statistically signiﬁcantly diﬀerent, but in practical terms would be considered very close for mostapplications. The error between the ground truth and the recovered deformation ﬁeld can be explained by thediﬀerence of deformation (and regularisation) model between the demons algorithm and the FFD.4.4 ConclusionOur novel NMI-demons performs similarly on intra- and inter-modal examples, and is comparable to the classicalMSE-demons on examples where the latter is applicable. There is no signiﬁcant diﬀerence in computation time.Future work could involve the exploration of theoretical and practical aspects of the optimisation scheme. Inter-modal capability is a valuable addition to the growing spectrum of fast demons-based methods.REFERENCES[1] Thirion, J.-P., “Image matching as a diﬀusion process: An analogy with Maxwell’s demons,” Med. ImageAnal. 2(3), 243–260 (1998).[2] Christensen, G. E., Rabbitt, R. D., and Miller, M. I., “Deformable templates using large deformationkinematics,” IEEE Trans. Image Process. 5, 1435–1447 (1996).[3] Rueckert, D., Sonoda, L., Hayes, C., Hill, D., Leach, M., and Hawkes, D., “Nonrigid registration usingfree-form deformations: Application to breast MR images,” IEEE Trans. Med. Imag. 18, 712–721 (1999).[4] Vercauteren, T., Pennec, X., Perchant, A., and Ayache, N., “Diﬀeomorphic demons: Eﬃcient non-parametric image registration,” Neuroimage 45, S61–S72 (Mar. 2009).[5] Vercauteren, T., Pennec, X., Perchant, A., and Ayache, N., “Symmetric log-domain diﬀeomorphic registra-tion: A demons-based approach,” in [Proc. MICCAI’08], 754–761 (2008).[6] Guimond, A., Roche, A., Ayache, N., and Meunier, J., “Multimodal brain warping using the demonsalgorithm and adaptative intensity corrections,” IEEE Trans. Med. Imag. 20, 58–69 (Jan. 2001).[7] Hermosillo, G., Chefd’Hotel, C., and Faugeras, O., “Variational methods for multimodal image matching,”Int. J. Comput. Vis. 50, 329–343 (Dec. 2002).[8] Tao, G., He, R., Datta, S., and Narayana, P. A., “Symmetric inverse consistent nonlinear registration drivenby mutual information,” Comput. Methods Programs Biomed. (2009). In press.[9] Cocosco, C., Kollokian, V., Kwan, R., and Evans, A., “Brainweb: Online interface to a 3D MRI simulatedbrain database,” Neuroimage 5(4), 425 (1997).[10] Cachier, P., Bardinet, E., Dormont, D., Pennec, X., and Ayache, N., “Iconic feature based nonrigid regis-tration: The PASHA algorithm,” Comput. Vis. Image Underst. 89, 272–298 (Feb. 2003).[11] Modersitzki, J., [Numerical methods for image registration], Oxford University Press (2004).[12] Studholme, C., Hill, D. L. G., and Hawkes, D. J., “An overlap invariant entropy measure of 3D medicalimage alignment,” Pattern Recognit 32, 71–86 (1999).[13] Modat, M., Ridgway, G. R., Taylor, Z. A., Hawkes, D. J., and Fox, N. C., “A parallel-friendly normalizedmutual information gradient for free-form registration,” Proceedings of SPIE (Jan 2009).

[Numerical methods for image registration],

A parallel-friendly normalized mutual information gradient for free-form registration,”

An overlap invariant entropy measure of 3D medical image alignment,”

Brainweb: Online interface to a 3D MRI simulated brain database,”

Deformable templates using large deformation kinematics,”

Diﬀeomorphic demons: Eﬃcient nonparametric image registration,”

Iconic feature based nonrigid registration:

Image matching as a diﬀusion process: An analogy with Maxwell’s demons,”

Multimodal brain warping using the demons algorithm and adaptative intensity corrections,”

Nonrigid registration using free-form deformations: Application to breast MR images,”

Symmetric inverse consistent nonlinear registration driven by mutual information,” Comput. Methods Programs Biomed.

Symmetric log-domain diﬀeomorphic registration: A demons-based approach,”

Variational methods for multimodal image matching,”

Diffeomorphic demons using normalized mutual information, evaluation on multimodal brain MR images

Diffeomorphic demons using normalized mutual information, evaluation on multimodal brain MR images

Abstract

Similar works

Full text

Available Versions

UCL Discovery