This paper presents a method for automatic segmentation of the prostate from transversal T2-weighted images based on 3D Active Appearance Models (AAM). The algorithm consist of two stages. Firstly, Shape Context based non-rigid surface registration of the manual segmented images is used to obtain the point correspondence between the given training cases. Subsequently, an AAM is used to segment the prostate on 50 training cases. The method is evaluated using a 5-fold cross validation over 5 repetitions. The mean Dice similarity coefficient and 95% Hausdorff distance are 0.78 and 7.32 mm respectively

Heijden, Ferdi van der

Maan, Bianca

English

van der Heijden, Ferdinand

NARCIS 

Prostate MR image segmentation using 3D active appearance models

University of Twente Research Information

Prostate MR image segmentation using 3DActive Appearance ModelsBianca Maan and Ferdi van der HeijdenMIRA Institute for Biomedical Technology and Technical Medicine, University ofTwente, the Netherlands;Abstract. This paper presents a method for automatic segmentationof the prostate from transversal T2-weighted images based on 3D Ac-tive Appearance Models (AAM). The algorithm consist of two stages.Firstly, Shape Context based non-rigid surface registration of the man-ual segmented images is used to obtain the point correspondence betweenthe given training cases. Subsequently, an AAM is used to segment theprostate on 50 training cases. The method is evaluated using a 5-foldcross validation over 5 repetitions. The mean Dice similarity coefficientand 95% Hausdorff distance are 0.78 and 7.32 mm respectively.1 BackgroundProstate segmentation is essential for calculating prostate volume, image fusion,creating patient-specific prostate anatomical models, and as a pre-processingstep for many computer aided diagnosis algorithms. Furthermore, informationabout the size, volume, shape and location of the prostate relative to adjacentorgans is an essential part of planning for minimally invasive therapies and biop-sies.Because manual segmentation of the prostate is time-consuming and highlysubjective, (semi-)automatic segmentation methods are preferable. However, seg-menting the prostate in MR images is challenging due to the large variations ofprostate shape between subjects, the lack of clear prostate boundaries and thesimilar intensity profiles of the prostate and surrounding tissues.The 2012 MICCAI challenge: “Prostate MR Image Segmentation” involvessegmentation of the prostate on transversal T2-weighted images. The goal of thechallenge is to evaluate segmentation algorithms on images from multiple centersand multiple MRI device vendors.Only a few prostate segmentation methods for T2-weighted MR images cur-rently exist. Klein et al. [1] proposed a method based on non-rigid registration ofa set of pre-labeled atlas images, against the target patients image, using mutualinformation. Subsequently, the segmentation is obtained as the average of thebest matched registered atlas sets. Multiple modifications are published on thisatlas based prostate segmentation method [2–4].The methods presented by Toth et al. [5] and Ghose et al. [6, 7] are basedon statistical shape models. Toth et al. used a levelset-based statistical shapemodel – which is landmark free and fully 3D – in combination with an statisticalappearance model. In order to segment the prostate in a new image, a Bayesianclassifier is employed to pre-classify the image voxels as belonging to the prostateor the surroundings. Subsequently the shape model is fitted to the segmentationfollowed by an adaptive update of the statistical appearance model until conver-gence. Ghose et al. presented several segmentation methods based on 2D activeappearance models (AAM). Active appearance models include both shape andappearance of the prostate. In [6], they presented a texture enhanced 2D AAM.In [7] they applied 2D AAM with a 3D shape restriction imposed by rigidlyregistering the obtained volume to a 3D average model of the prostate.It has been demonstrated that the combination of both shape and inten-sity prior information improves the segmentation accuracy. We present a novelframework for automated prostate segmentation based on 3D active appearancemodels (AAM) in combination with Shape Context based non-rigid surface reg-istration to obtain a point distribution model.2 MethodologyOur proposed method is built on the work presented by Kroon et al. [8]. Theyused a Shape Context based non-rigid surface registration in combination withActive Shape Models to segment knee cartilage. We adapted the framework touse the Shape Context registration with Active Appearance Models (AAM) toinclude the texture of the prostate.An overview of the proposed system is shown in Figure 1 and consists ofa training and a testing part. AAM contain a statistical model of the shapeand grey-level appearance of the organ which can generalise to almost any validexample. AAM learn what are valid shape and intensity variations from theirtraining images. During the testing part, the model is fitted to a new imagewhereafter the results for the segmentation are compared with the providedground truth segmentation.2.1 Shape context registrationThe first step in AAM training is describing the prostate surface in each trainingcase by a set of n landmark points. Every landmark in a training case must have acorresponding landmark in all other training cases. These point correspondencesallow Principle Component Analysis (PCA) to extract the principal modes ofshape variations.We want to use the vertices of the prostate surfaces to train the AAM.Therefore all surfaces must contain an equal number of vertices, and every vertexmust have a corresponding vertex at relatively the same location in all trainingcases. To obtain the corresponding points we use Shape Context based non-rigidregistration of the binary segmentation surfaces. We will describe the methodbriefly, more details are published in [8] and [9].Fig. 1. Schematic overview of our prostate segmentation approach. The upper bluebox shows the training of the AAM. The lower blue box shows the test part of theAAM.The method consists of four steps: the first step is extracting the surface ofthe segmented prostate. The second step is constructing a descriptor, the shapecontext, for each point in both data sets. The Shape Context describes everypoint on the surface by the vectors originating from that point to all other pointson the surface. For every point on the surface, n − 1 vectors are obtained. Thefull Shape Context representation is too detailed. Therefore, for every point pion the surface, a coarse histogram of the relative coordinates of the remainingn − 1 points is defined to be the shape context.The third step is calculating the matrix Ci,j = C(pi, qj) which containsthe costs between each pair of points. Considering points pi and point qj withhistograms hi and hj respectively, the costs can be calculated as:C(pi, qj) =12K∑k=1[hi(k) − hj(k)]2[hi(k) + hj(k)](1)The fourth step is matching the points by minimizing the total matching cost.The total Shape Context based non-rigid registration works as follows [8]:One prostate surface is selected as reference surface. This surface is registered toa surface of another training case. Subsequently the Shape Context is calculatedfor all points in both shapes (reference and training). 3D Shape Context isused [8,10], which is an extension to the original 2D Shape Context [9]. For everypoint on the reference surface, we search for a point in the training surface withthe minimum cost, and also from the training surface to the reference surface.The obtained point correspondences are then used to construct a B-spline gridwhich warps the reference surface to the shape of the training surface. We repeatthis process –using a coarse to fine grid– until the reference surface is transformedinto the shape of the training surface. In some regions, the transformed surfacepoints can still deviate from the real boundary by one pixel. Therefore, thealgorithm uses a modified iterative closest point (ICP) algorithm to calculatethe vectors from the vertices to the closest surface. Using these vectors the finalB-spline grid is updated.2.2 3D Active Appearance ModelAfter obtaining the corresponding points, all point sets are aligned into a com-mon reference frame. Let each point set be represented by a vector x. Thenprinciple component analysis (PCA) is applied to determine the principal modesof the shape variations.x = [x1, x2, ..., xn, y1, y2, ..., yn, z1, z2, ..., zn]Tx̃ = x̄ + Φsbs(2)with x̃ an estimate of the shape, x̄ the mean shape, Φs the t eigenvectorscorresponding to the largest eigenvalues and bs a set of shape deformation pa-rameters [11].The appearance model can be described in a similar way. First we warp eachtraining image –using a piece-wise affine warp– so that its points correspond tothe mean shape points. Then we sample the grey-level information of the regioncovered by the mean shape. After applying PCA to the normalised appearancedata we obtain the modelg = [g1, g2, ..., gm]Tg̃ = ḡ + Φgbg(3)with g representing the appearance vector of m voxels, g̃ an estimation of thegrey-level appearance, ḡ the mean grey-level appearance, Φg the t eigenvectorsand bg a set of appearance variation parameters.Shape and appearance are often correlated, therefore we can combine bothmodels and apply a third PCA:b =(Wsbsbg)=(WsΦsT (x − x̄)ΦgT (g − ḡ))b = Qc(4)where Ws is a diagonal matrix of weights for each shape parameter, allowing forthe difference in units (distance vs. intensity) between the shape and grey models.Q are the eigenvectors and c is a vector of appearance parameters controllingboth shape and grey-level appearance. Now the shape and grey-levels can beexpressed as function of c usingx = x̄ + ΦsW−1s Qscg = ḡ + ΦgQgc.(5)The AAM is optimized by minimizing the difference between the test imageand the synthesized images [11]. Since the appearance models can have manyparameters, the fitting to a new image appears to be a difficult optimisationproblem. However, each attempt to fit the model to an image is actually a sim-ilar optimisation problem. Therefore, prior knowledge about how to adjust themodel parameters during image search is acquired by perturbing the combinedmodel and the pose parameters with some known offsets, and recording thecorresponding changes in the appearance.The mean model is initialized by manual selecting the center of the prostatebased on visual inspection. Subsequently, the AAM is performed using two res-olutions with each 15 iterations.3 Experimental Design3.1 MR ImagesThe training data comprise 50 anonymized patient cases which include transver-sal T2-weighted MR images of the prostate. The training data is multi-center andmulti-vendor, and has different acquisition protocols. This means that there isa difference in slice thickness, slice orientation, images with/without endorectalcoil etcetera.Furthermore, a reference segmentation of the prostate is provided for eachcase in the training data. The images are downloaded in .mhd format from theMICCAI challenge website: http://promise12.grand-challenge.org/Pre-processing The scans are not uniformly sampled, therefore we re-samplethe scans and segmentations to a voxel resolution of 0.5 x 0.5 x 0.5 mm, usingcubic interpolation for the MRI data and nearest neighbor interpolation for thelabel data. The orientation of the slices differ per case in the training data.Therefore the images are rotated to obtain images with equal orientations.3.2 ExperimentsFor evaluation of our proposed method we perform a 5-fold cross-validation over5 repetitions. During one 5-fold cross-validation experiment, the training datais randomly partitioned into five sub sets. Of the five sub sets, four sub sets areused as training data, while the remaining sub set is retained as testing datafor the model. The process is then repeated five times, with each of the sub setsused exactly once as test data.Furthermore, we splitted the training cases in two groups; the first groupcontaining the cases with endorectal coil, the second group containing the caseswithout endorectal coil. To evaluate our algorithm using two seperate models,we performed a 3-fold cross validation over 5 repetitions on these two groups.The segmentations are evaluated using the Dice coefficient (DSC) and the95% Haussdorf distance. Moreover, slices of MR images with their segmentationare shown, providing a qualitative evaluation.4 Results and DiscussionTable 4 shows the mean and median DSC and 95% Hausdorff distance evaluatedusing a 5 fold cross validation over 5 repetitions. It is generally accepted that aDSC value > 0.7 represents a good agreement. The mean DSC overlap for oursegmentation was 0.78 (sd = 0.12, calculated over all cases and all iterations),with a median DSC of 0.82, which confirms overall satisfactory results of thesegmentation. However, the algorithm was unable to segment the prostate incase 23 which contains a very large prostate that exceeds the boundaries ofthe MR image. Discarding this outlier result, a mean DSC of 0.80 (sd=0.08) isobtained.The presence of an endorectal coil has influence on the prostate shape andimage intensities. Therefore we decided to train two separate models for thecases with and without coil. The results in Table 4 show that the mean DSC isincreased to 0.81 (sd = 0.12), with a median DSC of 0.83. From this results wecan conclude that training the AAM on images per institute, and per imagingprotocol, will slightly improve the segmentation results. However, a generallyapplicable segmentation method is preferable. Therefore we decided to segmentthe test cases with one AAM instead of two separate models.Although our DSC values are slightly higher than the values reported byDowling et al. [3] (mean: 0.73) and Gubern-Merida et al. [4] (median: 0.79),they are lower compared to several other articles [1, 5–7, 12]. However, it is tobe noted that they mostly use training and testing data from only one institute,containing less variation.The mean 95% HD value for all datasets is 7.32 mm (sd = 4.91) with amedian value of 6.18 mm. As with the DSC values, splitting the model in twoparts (with and without coil) improves the HD values to 6.43 mm (sd = 4.63)with median 5.19 mm. Our HD values are lower than the values reported byGubern-Merida et al. [4].Table 1. Table with mean evaluation values over all training cases and five iterations.One shape model Two shape modelsmean DSC 0.78 ± 0.12 0.81 ± 0.12median DSC 0.82 0.83mean 95% HD [mm] 7.32 ± 4.91 6.43 ± 4.63median 95% HD [mm] 6.18 5.19Figure 2 shows a surface mesh generated from the segmentation result of case30 (DSC = 0.84) with a colormap indicating the vertex distance to the groundtruth. Visual inspection of these meshes for all cases indicate that the regionwith the highest segmentation error is not consistent. Figure 3 shows 4 slicesfrom different prostates (with DSC > 0.7) with the ground truth contour inred and the segmentation result in green. The automatic segmentation contoursshow good agreement with the manual ground truth segmentations.Fig. 2. Surface mesh from case 30. Thecolormap indicates the vertex distance (inmm) between the segmentation result andthe ground truth.Fig. 3. Segmentation result for case 01, 11(top), 27 and 46 (bottom). The red line rep-resents the ground truth outline while thegreen line represents the segmentation re-sult.The automatic segmentation required approximately 3.8 minutes on a stan-dard desktop PC (Intel Core(TM) i7 CPU 870 @ 2.93GHz). See Table 2 for moreinformation about the efficiency of our algorithm.Table 2. Details about the algorithm.Parameter ValueAlgorithmLanguage: Matlab 7.13.0.564Libraries/Packages: Image Analysis Toolkit + 2 packages fromMathworks File exchange [10, 13]GPU Optimizations: -Multi-Threaded: YesUser Interaction: (1) Selection of the middle of the prostateMachine CPU Clock Speed: 2.93 GHzMachine CPU Count: 4Machine Memory: 16 GBMemory Used During Segmentation: ± 2 GBTimePreprocessing time: ± 2 minutes (per case)Shape Context registration: ± 3.5 minutes (per case)Training Time: ± 3 hours (for 50 training cases)Segmentation Time: ± 1.8 minutes (per case)User Interaction Time: ± 50 seconds (per case)5 Concluding RemarksEvaluation of our segmentation method has been performed using 50 transversalT2-weighted MR images. A 5-fold cross validation over 5 repetitions is performedgiving a mean DSC of 0.78 with median 0.82. The 95% HD and visual inspectionof the results also show good agreement of the prostate segmentation.The method could be further improved. For example by adding texture infor-mation to the AAM [6], adding additional preprocessing steps such as removingthe bias field, and using several initial pose parameters during the initialisationof the model.References1. Klein, S., van der Heide, U.a., Lips, I.M., van Vulpen, M., Staring, M., Pluim,J.P.W.: Automatic segmentation of the prostate in 3D MR images by atlas match-ing using localized mutual information. Medical Physics 35(4) (2008) 14072. Martin, S., Daanen, V., Troccaz, J.: Atlas-based prostate segmentation using anhybrid registration. International Journal of Computer Assisted Radiology andSurgery 3(6) (September 2008) 485–4923. Dowling, J., Fripp, J., Greer, P., Ourselin, S.: Automatic atlas-based segmentationof the prostate : A MICCAI 2009 Prostate Segmentation Challenge entry (2009)4. Gubern-merida, A., Marti, R.: Atlas Based Segmentation of the prostate in MRimages (2009)5. Toth, R., Bulman, J., Patel, A.D., Bloch, B.N., Genega, E.M., Rofsky, N.M., Lenk-inski, R.E., Madabhushi, A.: Integrating an adaptive region-based appearancemodel with a landmark-free statistical shape model: application to prostate MRIsegmentation. Proceedings of SPIE Medical Imaging: Image processing 7962(May)(2011) 79622V–79622V–126. Ghose, S., Oliver, a., Marti, R., Llado, X., Freixenet, J., Vilanova, J.C., Meri-audeau, F.: Prostate Segmentation with Texture Enhanced Active AppearanceModel. 2010 Sixth International Conference on Signal-Image Technology and In-ternet Based Systems (December 2010) 18–227. Ghose, S., Oliver, A., Marti, R., Llado, X., Freixenet, J., Mitra, J., Vilanova, J.C.,Meriaudeau, F.: A hybrid framework of multiple active appearance models andglobal registration for 3D prostate segmentation in MRI. In: SPIE Medical Imaging;Image processing. Volume 8314. (2012) 83140S–18. Kroon, D.J., Kowalski, P., Tekieli, W., Reeuwijk, E., Saris, D., Slump, C.H.: MRIbased knee cartilage assessment. Proceedings of SPIE Medical Imaging: Computer-Aided Diagnosis 8315(0) (2012) 83151V–83151V–109. Belongie, S., Malik, J.: Shape matching and object recognition using shape con-texts. Pattern Analysis and Machine 24(24) (2002) 509–52210. Kroon, D.J.: Shape Context Based Corresponding Point Models11. Cootes, T., Edwards, G.: Active appearance models. Proceedings of EuropeanConference on Computer Vision 2 (1998) 484–49812. Makni, N., Puech, P., Lopes, R., Dewalle, a.S., Colot, O., Betrouni, N.: Combininga deformable model and a probabilistic framework for an automatic 3D segmenta-tion of prostate on MRI. International journal of computer assisted radiology andsurgery 4(2) (March 2009) 181–813. Kroon: Active Shape Model (ASM) and Active Appearance Model (AAM)

A.: Integrating an adaptive region-based appearance model with a landmark-free statistical shape model: application to prostate MRI segmentation.

Active appearance models.

Active Shape Model (ASM) and Active Appearance Model (AAM)

Atlas Based Segmentation of the prostate in MR images

Atlas-based prostate segmentation using an hybrid registration.

Automatic atlas-based segmentation of the prostate : A MICCAI

Automatic segmentation of the prostate in 3D MR images by atlas matching using localized mutual information.

C.H.: MRI based knee cartilage assessment.

Combining a deformable model and a probabilistic framework for an automatic 3D segmentation of prostate on MRI. International journal of computer assisted radiology and surgery 4(2)

F.: A hybrid framework of multiple active appearance models and global registration for 3D prostate segmentation in MRI. In: SPIE Medical Imaging; Image processing. Volume 8314.

Prostate Segmentation with Texture Enhanced Active Appearance Model.

Shape Context Based Corresponding Point Models

Shape matching and object recognition using shape contexts.

Prostate MR image segmentation using 3D active appearance models

Abstract

Similar works

Full text

Available Versions

NARCIS

University of Twente Research Information