The prediction of the position of a given volume sample in a full body atlas, also known as a volume localization, is a part of an initial stage of image retrieval in most of

the dedicated CAD systems. In this paper we present two methods for volume localization, namely histogram matching and classifier regression. Since the histogram matching method ignores the spatial orientation, it is used when the orientation of the volume cubes are not the same. On the other hand the classifier regression is much faster and can be used as a quick estimation and as a tool to reduce the scope of the initial problem. Both presented methods were tested on a dataset with 3962 volumes of a human body atlas. The accuracy and the speed of execution was compared and is presented in this pap

Dobravec, Tomaž

RADOJEVIĆ, Miroslav

Šajn, Luka

ePrints.FRI

3D volume localization using miniaturesLuka Sˇajn1, Miroslav Radojevic2 and Tomazˇ Dobravec11Faculty of Computer and Information ScienceUniversity of Ljubljana, Slovenija2Biomedical Imaging Group of Erasmus MCUniversity Medical Center Rotterdam, NetherlandsAbstract—The prediction of the position of a given volumesample in a full body atlas, also known as a volume localization,is a part of an initial stage of image retrieval in most ofthe dedicated CAD systems. In this paper we present twomethods for volume localization, namely histogram matchingand classifier regression. Since the histogram matching methodignores the spatial orientation, it is used when the orientationof the volume cubes are not the same. On the other handthe classifier regression is much faster and can be used as aquick estimation and as a tool to reduce the scope of the initialproblem. Both presented methods were tested on a dataset with3962 volumes of a human body atlas. The accuracy and thespeed of execution was compared and is presented in this paper.I. INTRODUCTION3D volume localization problem is a problem of estimatingthe atlas coordinates of a given sample 3D volume (i.e.object, miniature). This is usually done by comparing thecharacteristics of a sample volume with the characteristics ofother volumes with known coordinates. The coordinates of themost similar volume are returned as a result, as shown in Fig.1.To achieve this, one can use the method which comparesthe sample volume with the other volumes from the datasetand the coordinates are estimated as the interpolation of thecoordinates of the most similar volumes, weighted by thedegree of similarity. Another approach attempts to estimatethe coordinates using classifier regression. Part of the problemis to find the matching volumes - this task can be viewedas the contents based image retrieval (CBIR) problem. Oneof the models to solve the CBIR is to treat the images ascollections or histograms of the features [2] and compare thedistances (dissimilarities) between the feature vectors. Theresulting feature is the one with the lower distances, hencehigher similarity. Potential features used for grey-level imagesFig. 1. Estimating the coordinates of the most similar volumecan be the very pixels, features that correspond to humanvisual perception, texture features such as global descriptors orGabor features, or local descriptors such as SIFT or corner de-tectors [2]. Shape features such as Fourier descriptors, momentinvariants and finite element models were surveyed in [12],graph based shape features were presented in [1]. A method forretrieving 3D datasets based on Local Binary Patterns (LBP)features [4] was introduced in [11] and compared with featuressuch as 3D Wavelet Transforms and 3D co-occurence matrices.In this paper we use and compare two methods for thecoordinate prediction, namely the histogram matching, whichis presented in Section II-A, and the classifier regression,presented in Section II-B. The results of the comparison aregiven in Section III. We conclude the paper in Section IV.II. METHODSTwo methods for the coordinate prediction are used: histogrammatching and the classifier regression.A. Histogram matchingThe orientation of the sample volume is usually not the sameas the orientation of the volumes in the dataset. Hence, itis necessary either to encounter for their proper alignment,or to use a measure independent to the orientation of thevolumes. Such measures include histogram, rotation invariantimage moments and local features. The first method presentedin this paper compares the histogram of the samples withthe histograms of all the other miniatures. The drawback ofusing histograms is the loss of spatial intensity distributioninformation. Some other rotation invariant feature extractionmethod such as SIFT or LBP in 3D [4] would be potentiallysuitable to overcome the problem of describing the spatialrelations between voxels.Figure 2 depicts three different volume samples in both 3Dand cross-section version, their histograms and the detectedlocations within the human body atlas using the histogrammatching method.1) Histogram dissimilarity measures: There are several cross-bin and bin-to-bin dissimilarity measures to distinguish amonghistograms being compared [6]. Generally, cross-bin distancessuch as Earth Movers Distance (EMD) tend to be morerobust and more discriminative than bin-to-bin measures whencomparing two histograms [9].Let L and M represent histograms being compared and eachvalue M(i) the count of the number of observations that fallFig. 2. Three volumes with their histograms and detected location within human body atlas.into one of the disjoint intensity intervals. If n be the totalnumber of observations and k the total number of bins, thenthe following holds:n =k∑i=1M(i)Among the most popular bin-to-bin measures let us mention:histogram intersection, L1 and L2 norm, χ2, and JeffreysDivergence with calculation formulas (in respective order)• h(L,M) =∑jmin(L(j),M(j)))∑jM(j)• L1(L,M) =∑jabs(L(j)−M(j))N• L2(L,M) =∑j(L(j)−M(j))2N• χ2(L,M) =∑j(L(j)−M(j))2∑jL(j)+M(j)• Jf(L,M) =∑j(L(j)log 2L(j)L(j)+M(j) +M(j)log2M(j)L(j)+M(j))An important representative of the cross-bin histogram com-parison method is called the Earth Movers Distance (EMD)[8], [9]. Fast implementation of the cross-bin histogram com-parison EMD was used in our tests [8].2) Weighting the coordinates: We have performed the testsby calculating all of the inter-histogram dissimilarities. Afterthis calculation, ten most similar volumes were queried outof the base and used to estimate the coordinate. Estimationwas based on weighted averaging where weights sum up tounit and each weight corresponds to the similarity level di andfollows negative exponential function with slope determinedby normalized variance of 10 selected dissimilarities.The coordinates of the ten best matches (ranked with i) wereweighted by exponentially decaying weights, derived fromthe variance of their dissimilarity measures. The resultingcoordinates [xest, yest, zest] were calculated by[xest, yest, zest] =10∑i=1wi[xiyizi]where wi = e−λi and λ = max(di−mean(di)var(di)).Such weighting was used to overcome the influence of theoutliers on the estimation since outliers can cause highervariance in general as shown in Fig. 3.B. Classifier regressionAnother approach to predict the coordinate values uses classi-fier regression. Classifiers such as 10-NN and Random FernsFig. 3. The weights of the outliers are smaller than the weights of theneighboring candidates.[7] use the image features and carry out the regression insteadof usual crisp classification. PRTools Matlab library [3] isused for 10-NN implementation while Piotrs Matlab Toolbox[10] is used for Random Ferns implementation. Classifierscores are used to interpolate coordinates of those that wereclassified as being close to the test sample in a way typical foreach classifier. Namely, averaging of the 10 nearest neighborscoordinates will be 10-NN regression. Similarly, in the caseof SVM classifier usage, coordinates of the most similar onesseparated by SVM will be interpolated. Features used forclassifier regression are: mean, standard deviation, median,volume (actual number of pixels in the volume), centroid (1storder image moments), central moments (2nd order imagemoments), and the voxels themselves can be used as features,too. In cases when voxel intensities are used (Random Fernsclassification), binary features need to be extracted - hencenumber of combinations can grow high. Therefore, volumeswere resized using Gaussian pyramid before calculating thebinary features.III. RESULTSWe tested the methods on a dataset which consist of 3926volumes (miniatures representing body regions). Each of thevolumes was resampled to 32 × 32 × 32 image cube. Voxelintensities of the image cube are real values ranging from 0 to4095. Each volume corresponds to certain spatial regions ofthe body as it was shown with dots (center points of the region)in Fig. 5. Volumes are actually resembling different regionsof the body and they are taken from different orientations(transverse, coronal, sagital).Results are presented for both approaches introduced in theSection II. ”Histogram comparison” turns out to be quiteprecise at finding similar patches. However, the choice of thedissimilarity metric is very important. As mentioned, cross-bin dissimilarity is preferred since it is more robust. Theperformance of different metrics when matching histogramsfor query volume (object nr. 1540) shown together with itstop 10 most similar patches from the database. Process finds10 most similar volumes and interpolates their coordinates.Fig. 4. Human atlas with 3926 volumes.Fig. 5. Voxel representing a region of a human body.Dissimilarity measure h L1 L2 χ2 Jf EMDEstimated position error 16.5792 3.7436 129.8316 1.3632 1.2895 1.2466TABLE IESTIMATED POSITION ERRORS FOR DIFFERENT HISTOGRAM COMPARISON MEASURES (INTERSECTION, L1 AND L2 NORM, χ2 , JEFFREY DIVERGENCEAND EMD) FOR THE OBJECT NR. 1540Fig. 6. Results of volume matching using histogram comparison for dissimilarity measures L1 norm, L2 norm, EMD, χ2, Jeffrey divergence, and Intersect(from left to right, top to bottom).EMD dissimilarity measure turns out to be the one thatperforms the best both in terms of accuracy and robustness,however it is slower and features used for matching are depen-dent on the database. Downside of its usage is computationalcost, nevertheless fast implementation of EMD [8] used inthis experiment provides satisfactory performance. What isstill a potential problem with this approach is the fact thatit is dependent on the data from the training set - since itis based on comparing the input with the training dataset. Inthat sense - classifier regression based approach with invariantstatistical/shape/moment based features would be interesting.Figure 6 show the results of volume matching using histogramcomparison for dissimilarity measures: L1 norm, L2 norm,EMD, and χ2, Jeffrey divergence, and intersection (from leftto right, top to bottom). Predicted value is represented withwhite square, while the true value is described with red circle.Top 10 matching positions were shown as dots in jet colorspace - ranging from red ones that represent high ranking tillthe blue ones with low ranking. Estimation fails in case metricwas not properly chosen. EMD turns out to be the one thatperforms the best both in terms of accuracy and robustness.Fig. 7. Statistics for the absolute distance error between the position estimateand the actual position.Finally, histogram matching was performed individually on 50randomly chosen objects while the rest were used for matchingwith the test sample each time. Figure 7 shows the statisticsfor the absolute distance error between the position estimateand the actual position. Majority of discrepancies fall into 0-50 voxel distance range which suggests that the object wastargeted. Usually, distances up to 50 absolute voxel valuesdo properly guess the object since the main task is roughlocalization of the body region.”Classifier regression” was based on independent features(such as volume mean, standard deviation, image moments,volume). As stated before - classifier score was the basisfor the data regression. 10-NN classifier was used to findthe neighbors in feature space and average the coordinatesassociated with them in order to carry out the regression.This process works much faster since it takes short time toextract the features and there is no need for comparison. Itaccomplishes the retrieval but the performance is lower asshown in the figure 8.Regression accuracy is worse than the one obtained usinghistogram comparison and one of the causes is probably plainaveraging of the feature-space-nearest-neighbor coordinatesFig. 8. Results of classifier regression (10-NN) based on independent features(no weighting). The features used can influence the choiceof the nearest neighbors. Finally, Random Ferns [7] weretested on position regression. They performed slightly slowerthan 10-NN regression, but had an overall lower averageerror calculated over 50 randomly chosen miniatures. Voxelintensities were used as the source for making binary featuresthat this classification algorithm uses. Usage of statistic/shapefeatures that were supplied to 10-NN did not perform as wellas voxel intensities did. Binary features were chosen to be -1or +1 for each possible pair of voxels. In case the first onefrom the pair was higher - feature was set to +1 and -1 wasfor the opposite case. Each miniature needed to be resized to8×8×8 using Gaussian pyramides and even after the reduction- quarter of the total number (every 4th) of the voxels wasused for matching in the experiment due to processing timeconstrains. Results proved to be promising in spite of all theconstrains.IV. CONCLUSIONThis paper summarizes the results obtained while develop-ing 3D content based image localization framework. Twoapproaches have been examined - one based on comparingthe histograms and another one based on the classifier regres-sion. 10-NN and Random Ferns classifiers were used for theregression. The best performance in terms of accuracy wasachieved by comparing histograms using the EMD (EarthMovers Distance) method. Classification regression using 10-NN proved to be slightly less precise but significantly faster,and the same conclusion can be made for Random Fernsthat found their best performance by taking voxel values asfeatures and compared the 3D patches for similarity. Averageerror on position prediction calculated on 50 random sampleswas lower than the one obtained using 10-NN. Neverthe-less, Random Ferns classification was significantly constrainedwith reducing the feature number for the computational timepurposes, therefore its performance can certainly improve.Future work can include clustering of the coordinates intocrisp classes so that further evaluation of position predictionperformance can be estimated in a more appropriate way. 10-NN regression can be improved by using weighted averageof the nearest neighbors or some additional features. RandomFerns regression can perform better with more elegant wayof reducing features it uses. Finally, local feature extractionmethods such as 3D SIFT or 3D LBP can be used.REFERENCES[1] M.-C. Chang and B.B. Kimia. Measuring 3d shape similarity by graph-based matching of the medial scaffolds. Computer Vision and ImageUnderstanding, 115(5):707 720, 2011. Special issue on 3D Imaging andModelling.[2] T. Deselaers, D. Keysers and H. Ney. Features for image retrieval:an experimental comparison. Information Retrieval 11:77107, 2008.10.1007/s10791-007- 9039-3.[3] R.P.W.Duin, P.Juszczak, P.Paclik, E.Pekalska, D.deRidder, D.M.J.Tax,and S.Verzakov. Pr-tools4.1, a matlab toolbox for pattern recognition.http://prtools.org, 2007.[4] J.Fehr and H.Burkhardt. 3d rotation invariant local binary patterns. InPattern Recognition, 2008. ICPR 2008. 19th International Conferenceon, pages 1–4, dec. 2008.[5] J. Flusser, T. Suk, and B. Zitov. Moments and Moment Invariants inPattern Recognition. John Wiley & Sons, Ltd, 2009.[6] F.D. Jou, K.C. Fan, and Y.L. Chang. Efficient matching of large-sizehistograms. Pattern Recognition Letters, 25(3):277 286, 2004.[7] M. Ozuysal, M. Calonder, V. Lepetit, and P. Fua. Fast keypoint recog-nition using random ferns. Pattern Analysis and Machine Intelligence,IEEE Transactions, 32(3):448 461, march 2010.[8] O. Pele and M. Werman. Fast and robust earth movers distances. InComputer Vision, 2009 IEEE 12th International Conference on, pages460 467, 29 2009-oct. 2 2009.[9] O. Pele and M. Werman. The quadratic-chi histogram distance family.Computer Vision ECCV 2010, volume 6312 of Lecture Notes inComputer Science, pages 749762. Springer Berlin / Heidelberg, 2010.[10] D. Piotr. Piotrs image and video matlab toolbox (pmt).http://vision.ucsd.edu/ pdol- lar/toolbox/doc/index.html.[11] Y. Qian, X. Gao, M. Loomes, R. Comley, B. Barn, R. Hui, and Z.Tian. Content-based retrieval of 3d medical images. In eTELEMED2011, The Third International Conference on eHealth, Telemedicine,and Social Medicine, pages 712, 2011.[12] Y. Rui, T.S. Huang, and S.F. Chang. Image retrieval: Current techniques,promising directions, and open issues. Journal of Visual Communicationand Image Representation, 10(1):39 62, 1999.

3D volume localization using miniatures

Abstract

Similar works

Full text

Available Versions

ePrints.FRI