This paper presents a new method for multiple structure segmentation,

using a maximum a posteriori (MAP) estimation framework,

based on prior shape densities involving nonparametric multivariate

kernel density estimation of multiple shapes. Our method is motivated

by the observation that neighboring or coupling structures

in medical images generate configurations and co-dependencies

which could potentially aid in segmentation if properly exploited.

Our technique allows simultaneous segmentation of multiple objects,

where highly contrasted, easy-to-segment structures can help

improve the segmentation of weakly contrasted objects. We demonstrate

the effectiveness of our method on both synthetic images and

real magnetic resonance images (MRI) for segmentation of basal

ganglia structures

Cetin, Mujdat

Ercil, Aytul

Erçil, Aytül

Unal, Gozde

Uzunbas, Mustafa Gokhan

Uzunbaş, Mustafa Gökhan

Çetin, Müjdat

Ünal, Gözde

English

Crossref

Coupled nonparametric shape priors for segmentation of multiple basal ganglia structures

Sabanci University Research Database

COUPLED NONPARAMETRIC SHAPE PRIORS FOR SEGMENTATION OF MULTIPLEBASAL GANGLIA STRUCTURESGokhan Uzunbas, Mujdat Cetin, Gozde Unal, and Aytul ErcilSabanci UniversityFaculty of Engineering and Natural SciencesTuzla, 34956Istanbul,TurkeyABSTRACTThis paper presents a new method for multiple structure segmenta-tion, using a maximum a posteriori (MAP) estimation framework,based on prior shape densities involving nonparametric multivariatekernel density estimation of multiple shapes. Our method is mo-tivated by the observation that neighboring or coupling structuresin medical images generate configurations and co-dependencieswhich could potentially aid in segmentation if properly exploited.Our technique allows simultaneous segmentation of multiple ob-jects, where highly contrasted, easy-to-segment structures can helpimprove the segmentation of weakly contrasted objects. We demon-strate the effectiveness of our method on both synthetic images andreal magnetic resonance images (MRI) for segmentation of basalganglia structures.Index Terms— MRI, brain, Basal Ganglia, curve evolution,multi object image segmentation, shape priors, nonparametric shapedensity.1. INTRODUCTIONMedical image segmentation has improved the diagnosis of manyhuman diseases through more robust and accurate extraction of rel-evant tissues from medical image data. One of the main motivationsfor the segmentation of subcortical structures in the brain MR imagesis the analysis of chemicals in various basal ganglia (BG) structuresfor diagnosis of neurodegenerative diseases. The amount of chem-ical accumulation and shape deformations in subcortical structuresare thought to provide important clues in early diagnosis. However,the information obtained from these images does not often provideenough contrast or clear boundary patterns of subcortical structures.Under such conditions, use of prior information based on shape isgaining increased attention in MR image segmentation algorithms.Introducing shape priors into deformable models is not a nov-elty. There are numerous existing automatic segmentation methodsthat enforce constraints about the underlying shapes. In a typical ac-tive contour-based image segmentation, a curve length penalty termis often used for regularization, which makes the assumption thatshorter curves are in general more likely than the longer ones [1].However, in many applications, more structured prior informationabout the shapes is available. In [2] [3], principal component anal-ysis (PCA) of the signed distance functions of training data is usedto capture the variability of shapes. However, these techniques canThis work was partially supported by the European Commission un-der Grants FP6-2004-ACC-SSA-2 (SPICE), MIRG-CT-2006-041919 andMTKI-CT-2006-042717be applied only when the shape variability is rather small. Sec-ondly, these techniques can handle only unimodal, Gaussian-likeshape densities. In particular, these methods cannot deal with mul-timodal shape densities, which involve multiple classes of shapes.As a solution to the limitation of PCA and unimodal Gaussian dis-tribution models, techniques based on nonparametric shape densitieslearned from example training shapes have been proposed in [4], [5].Although a nonparametric prior is very adequate to capture non-linear shape variability, some challenging applications in medicalimage segmentation still require some additional information to beused. In many cases, objects to be segmented, have one or moreneighboring structures, whose location and shape provide informa-tion about the local geometry that can aid in the delineation. In thisfashion, the relative shape arrangements among these neighbors canbe modeled based on statistical information from a training set. In[6], a joint prior based on a parametric shape model, is proposedto capture co-variations shared among the different shape classes,which improves the performance of segmentation. With a similarapproach in a Bayesian framework, in [7], joint prior information ofthe multiple objects is used to capture the dependencies among thedifferent shapes where multiple objects with clearer boundaries areused as reference objects to provide constraints in the segmentationof poorly contrasted objects. Another coupled shape prior model,which is based on the cumulative distribution function of shape fea-tures, is proposed in [8]. In [8], relative inter-object distances aredefined as a shape feature to capture information about the interac-tion between multiple objects.In this paper, having the inspiration of capturing co-variationsand dependencies between different shapes, we propose a newmethod that extends nonparametric shape priors to the problemof coupled segmentation of multiple objects. We use multivariatedensity estimation to estimate unknown joint distribution of mul-tiple shapes which effectively captures coupling between distinctshapes in a scene. This key property of our method allows segment-ing multiple objects simultaneously where the new prior providesautomatical, coupled constraints to be used in challenging imagescenarios. Moreover, as compared to existing methods [6], [7]which are based on prior of multiple objects, our approach takesthe advantage of using nonparametric density estimate, in order tocapture non-linear shape variability.2. SEGMENTATION FRAMEWORK BASED ON COUPLEDSHAPE PRIORSIn a typical active contour model, the segmentation process involvesan iterative algorithm for minimization of an energy functional. Wedefine our energy (cost) functional in a MAP estimation frameworkasE(C) = − log p(data|C)− log pC(C) (1)where C is a set of evolving curves{C1, ..., Cm}that match theboundary of the shapes. The first term, log p(data|C), called thelikelihood term, enforces some form of data fidelity; and log pC(C)is a joint prior density for the shapes involved. In particular, wechoose the likelihood term as in [9], which is a common techniqueused in previous works such as [3], [5], [6], [7]. In this paper, wefocus on building the coupled shape prior pC(C).2.1. Coupled Prior Model of Multiple ObjectsHere the essential idea is to construct coupled nonparametric shapeprior information pC(C) for m different class of objects. Let us as-sume we are given N example training samples {C1, ...,CN}. Each”sample” consists of multiple shapes, each of which corresponds toone of m objects, i.e Ci ={C1i , ...Cmi}. The pose variation in thetraining data is removed by an alignment operation, as in [3]. In thisoperation we calculate a set of similarity transformation parameters(translation, scaling and rotation) for each sample in the training setto align binary shapes with each other. We align each sample of themth object within its own members and we obtain{C˜1, ..., C˜N}.After alignment, any variability due to pose differences is removedand what remains is shape variation. Fig. 1 shows binary alignedshape representations for 2 objects (head of caudate and putamen)from 12 different slices.(a) (b)Fig. 1. Aligned Training Shapes for (a) head of caudate and (b)putamen extracted from multi-modality sequences.To build a joint prior model for multiple objects, we choose levelsets as the representation of shapes [10] and we use multivariateParzen density estimation (see [11]) to estimate the unknown jointshape distribution. For the sake of simplicity of exposition and with-out loss of generality, let us take m = 2, and then, define the jointkernel density estimate of two shapes as,pC˜1,C˜2(C˜1, C˜2) =1NN∑i=1m=2∏k=1k(d(φC˜k, φC˜ki), σk) (2)where k(., σk) is a Gaussian kernel with standart deviation σk. Inthis equation, φC˜kis the candidate signed distance function and φC˜kiis the signed distance function for the ith training shape of the kthobject. Note that, given a distance measure d(., .), we can constructthe kernel function for joint density estimation, by multiplying sepa-rate kernels k(., σk) for each object. Our nonparametric shape priorin (2) can be used with a variety of distance metrics. In this work,we consider two specific metrics, namely the template metric and theL2 distance dL2 between signed distance functions. For the kernelsize σk, for the kth object, we use maximum likelihood kernel sizewith leave-one-out [12].As compared to existing single shape prior based approaches [3],[5] in this framework, we produce much more accurate joint shapedensities in cases where there are shape dependencies between themultiple objects involved. This is a phenomenon we observe in basalganglia structures.2.2. Gradient Flow for the Coupled Shape PriorIn this part, we compute the gradient flow of the joint prior in equa-tion (2) for the two curves which are represented implicitly by theircorresponding signed distance functions. We adress how the gradi-ent flow for the coupled shape prior is obtained with gradient descentoptimization.In differentiating the logarithm of the expession given in (2), weuse shorthand notation, kσk for k(dL2(φC˜k , φC˜ki), σk). Note thatφC˜kis a function of time t and φC˜kis a shorthand notation for theevolving level set function φC˜k(t). Using these, we obtain∂∂tlog pC˜1,C˜2(C˜1, C˜2) =1NN∑i=1{k′σ1kσ2 + kσ1k′σ2}pC˜1,C˜2(C˜1, C˜2)(3)Then, we compute the gradient direction that increases most rapidlyfor each object curve. Using the L2 distance in kernels, we find thatthe gradient directions for the curves C˜k, where k = 1, 2, are∂φC˜k∂t=1σk2N∑i=1λi(C˜1, C˜2)(φC˜ki(x)− φC˜k(x)) (4)where λi(C˜1, C˜2) =k(dL2 (φC˜1,φC˜1i),σ1)k(dL2 (φC˜2,φC˜2i),σ2)NpC˜1,C˜2(C˜1,C˜2), andN∑i=1λi(C˜1, C˜2) = 1. The final expression (4), evolves the curvestoward shapes at the local maximum of the coupled shape prior oftwo objects. Note that, training shapes that are closer to the evolv-ing curve get more weight. Furthermore, the weighting function λidepends on each curve in exactly the same way. In particular, dueto the coupled nature of this weight, given a pair (C˜1, C˜2) in theevolution process training shape pairs in which the second trainingshape is closer to C˜2 get relatively more weight in the evolution ofthe first curve as well. This shows one aspect of the coupled natureof our shape-based segmentation aproach.2.3. Iterative Segmentation AlgorithmGiven an arbitrary test image, the objects to be segmented in thescene are not necessarily aligned with the shapes in our training set.Due to the pose differences, we relate the density estimates of candi-date curves, pC(C) with density estimates of aligned curves, pC˜(C˜).In order to do that, describing each candidate curve by its shape andpose, we interpret C˜ as the shape of C after being transformed by T :C˜ = T [p]C (5)where T is a similarity transformation (translation, scaling and ro-tation) with parameters p = p1, ..., pm for each object and m is thenumber of objects. Then during the evolution process, we align eachcandidate curve with its training shapes. For a review of the poseupdate algorithm we refer the reader to [5]. We can summarize theprincipal steps of our segmentation algorithm as follows:1. Evolve C = C1, ..., Cm until iteration time to without shapeprior2. At t = t0, for all m, compute the pose p = p1, ..., pm3. while (t0 < iterations < maxIteration)a ) fix p, and for all m- compute C˜ = T [p]C using (Eqn. (5))- compute∂φC˜k∂tfor log pC(C) using (Eqn. (4))- relate C = T−1[p]C˜b ) Update C using both 1 data and shape forcec ) fix C and update p3. EXPERIMENTAL RESULTSIn this section we show segmentation results for the head of caudateand the putamen which are two subcortical structures of the brain.We first consider 2D synthetic images created by adding noise overbinary images representing the brain structures in Fig.1. These im-ages are obtained through manual segmentation of true MR images.In training the set, we use 11 of these slices (with size 512x448)from four different patients and we perform segmentation on im-ages which are not in the training set. We work with high contrastedcaudate and low contrasted, occluded putamen to focus on the cou-pling effect of using the joint shape prior. We compare our results onputamen with those of [5], which uses independent priors for the twostructures. Segmentation results with single prior and coupled priorfor both structures are shown in Fig.2. The results for our method onputamen, visually show better segmentation than the results obtainedby using a single prior.(a) (b)Fig. 2. Segmentation of the head of caudate (blue curve) and oc-cluded putamen (red curve) (a) with single prior information and,(b) coupled prior information.We also show quantitative comparison of the two methods bycomputing several performance measures such as false positive rate:FPR =FPFP + TN(6)false negative rate:FNR =FNFN + TP(7)mean ±σ PutamenFPR (%) FNR (%) 1 - D.C. (%)Single Prior 0.76 ±2 13.5 ±5.3 16.8 ±5.9Coupled Prior 0.70 ±2 13.1 ±5.0 13.8 ±4.2Table 1. Average performance results on 12 synthetic images.Initial Middle (t = t0) FinalFig. 3. Three steps in the segmentation of head of caudate and puta-men in an MR slice with no prior (top row), with single prior (middlerow) and with coupled prior information (bottom row).Dice Coefficient (DC) (see [13]):DC =2TPFP + FN + 2TP(8)where FP is false positive pixels inside the segmenting curve butout of the true shape, and FN is false negative pixels inside the trueshape but out of the segmenting curve. TN is true negative pixels outof true shape and segmenting curve. TP is true positive pixels insideboth true shape and segmenting curve. We present average valuesover 12 tests for 12 different images, in Table. 1.We then consider Proton Density (PD) sequences of real MRdata sets with 2 subcortical structures of interest, head of caudateand putamen. We use a training set of 11 pairs of shapes. Fig. 3top row shows initial, middle and final steps of the segmentationusing only gray level information as proposed in [9], where the headof caudate and left putamen cannot be segmented well, due to lowcontrast. The middle row in the Fig. 3 shows the result of using1Data force can be obtained by computing the gradient flow of the firstterm in Eqn. (1)independent shape prior for each structure [5]. In this case, due tothe leakage in putamen and mismatch in the right part of caudate,the result is not satisfactory. Fig. 3 bottom row shows the results ofour coupled shape prior model, where the coupling between putamenand caudate improves the results.In Table. 2, it can be observed that our method achieves a muchlower FPR for putamen and FNR for caudate than the technique in[5]. According to (1 - DC) measure, our method performs better thanthe other methods for both objects.%, %, %Putamen Head of CaudateFPR FNR 1-D.C. FPR FNR 1-D.CWithout Prior 1.10 5 12 4.2 8 22Single Prior 1.08 4 13 0.29 20 17Coupled Prior 0.88 4 8 0.35 14 13Table 2. Performance results on the real MR data set.4. CONCLUSIONSIn this paper, we proposed an approach for introducing coupledshape prior into the segmentation process. We use multivariateParzen density estimation to estimate the unknown joint distributionof multiple shapes. We introduce a new mechanism which automat-ically captures the coupling between distinct shapes in a scene. Weevolve multiple curves in parallel with the influence of joint prior tosegment poorly contrasted difficult shapes.Our preliminary experiments demonstrated that our method hasmuch promise for multi-object segmentation in medical imaging ap-plications. This current work does not include any pose constraintson relative positions of coupled shapes. Moreover, the data term thatwe use, can be incorporated with some statistical priors based on in-tensity characteristics of the tissues. We are currently working onincorporating these approaches to our pricipal approach in order toimprove our segmentation results in more challenging scenarios. Weare working on segmentation of other subcortical structures, as well.5. ACKNOWLEDGMENTThe authors would like to thank Junmo Kim for providing his codefor nonparametric shape prior. The authors would also like to thankDr. Kutlay Karaman, radiologist at Anadolu Medical Centre, forvery helpful conversations and for providing the MR data.6. REFERENCES[1] Mumford D. and Shah J., “Boundary detection by minimiz-ing functionals,” IEEE Conferance Computer Vision PatternRecognition, 1985.[2] M. Leventon, E. Grimson, and O. Faugeras, “Statistical shapeinfluence in geodesic active contours,” IEEE Conferance Com-puter Vision Pattern Recognition, 2000, pp. 316–323.[3] A. Yezzi, A. Tsai, W. Wells, C. Tempany, D. Tucker, A. Fan,W.E. Grimson, and A. Willsky, “A shape-based approach tothe segmentation of medical imagery using level sets,” IEEETransactions Medical Imaging, vol. 22, pp. 137–154, 2003.[4] D. Cremers, S.J. Osher, and S. Soatto, “Kernel density es-timation and intrinsic alignment for shape priors in level setsegmentation,” Internat. J. Comp. Vision, vol. 69, no. 3, pp.335–351, 2006.[5] J. Kim, M. Cetin, and A.S. Willsky, “Nonparametric shapepriors for active contour-based image segmentation,” SignalProcessing, vol. 87, pp. 3021 – 3044, 2007.[6] A. Tsai, W. Wells, C. Tempany, W.E. Grimson, and A. Willsky,“Mutual information in coupled multi-shape model for medicalimage segmentation,” Medical Image Analysis, vol. 8, pp. 429–445, 2004.[7] J. Yang and J. Duncan, “Joint prior models of neighboring ob-jects for 3-d image segmentation,” IEEE Conferance ComputerVision Pattern Recognition, Washington, DC, 2004.[8] A. Litvin and W.C. Karl, “Coupled shape distribution-basedsegmentation of multiple objects,” Information Processing inMedical Imaging, 2005, pp. 345–356.[9] T. Chan and L. Vese, “Active contours without edges,” IEEETransactions Image Processing., vol. 2, pp. 266–277, 2001.[10] S. Osher and R. Fedkiw, “Level set methods and dynamic im-plicit surfaces,” Springer, Berlin, 2003.[11] D. Erdogmus, R. Jenssen, Y.N. Rao, and J.C. Principe, “Gaus-sianization: An efficient multivariate density estimation tech-nique for statistical signal processing,” Journal of VLSI SignalProcessing, vol. 45, pp. 67–83, 2006.[12] B.W. Silverman, “Density estimation for statistics and dataanalysis,” Chapman Hall, London, 1986.[13] L. Dice, “Measures of the amount of ecologic association be-tween species,” Ecology, vol. 26, pp. 297–302, 1945.

http://research.sabanciuniv.edu/10528/1/uzunbas_ISBI08.pdf

Coupled nonparametric shape priors for segmentation of multiple basal ganglia structures

Abstract

Similar works

Full text

Available Versions

Crossref

Sabanci University Research Database