Segmentation of white blood cells (i.e. leukocytes) is a crucial step toward the development of haematological images
analysis of peripheral blood smears due to the complex nature of the different types of white blood cells and their
large variations in shape, texture, color, and density. This study addresses this issue and presents a single fully automatic segmentation framework for both nuclei and cytoplasm of the five classes of leukocytes in a microscope blood smears. The proposed framework integrates a priori information of enhanced nuclei color with Gram-Schmidt orthogonalization, and multi-scale morphological enhancement to localize the nuclei, whereas clustering-based seed extraction and watershed are utilized to segment the cytoplasm. The experimental results on two different datasets show that the proposed method works successfully for both nuclei and cytoplasm segmentation, and achieves more accurate segmentation results compared to the other methods in the literature

Cai, Weidong

Chen, Mei

Feng, Dagan

Song, Yang

Tareef, Afaf

Afaf Tareef

Yang Song

Dagan Feng

Mei Chen

Weidong Cai

Crossref

Automated multi-stage segmentation of white blood cells via optimizing color processing

Sydney eScholarship

AUTOMATED MULTI-STAGE SEGMENTATION OF WHITE BLOOD CELLS VIAOPTIMIZING COLOR PROCESSINGAfaf Tareef*1 Yang Song1 Dagan Feng 1 Mei Chen2,3 Weidong Cai11 School of Information Technologies, University of Sydney, Australia2 Computer Engineering Department, University of Albany State University of New York, USA3 Robotics Institute, Carnegie Mellon University, USAABSTRACTSegmentation of white blood cells (i.e. leukocytes) is acrucial step toward the development of haematological im-ages analysis of peripheral blood smears due to the complexnature of the different types of white blood cells and theirlarge variations in shape, texture, color, and density. Thisstudy addresses this issue and presents a single fully auto-matic segmentation framework for both nuclei and cytoplasmof the five classes of leukocytes in a microscope blood smears.The proposed framework integrates a priori information ofenhanced nuclei color with Gram-Schmidt orthogonalization,and multi-scale morphological enhancement to localize thenuclei, whereas clustering-based seed extraction and water-shed are utilized to segment the cytoplasm. The experimen-tal results on two different datasets show that the proposedmethod works successfully for both nuclei and cytoplasm seg-mentation, and achieves more accurate segmentation resultscompared to the other methods in the literature.Index Terms— White blood cells, nuclei and cytoplasmsegmentation, color reduction, Gram-Schmidt orthogonaliza-tion, marker-controlled watershed.1 IntroductionThe visual examination of the patient’s peripheral bloodsmear under the microscope is an important preliminary stepin the diagnosis of various blood related diseases, in whichthe changes in the number or morphologies of the blood cellsprovide an indicator of the nature and severity of the dis-ease. Human blood consists of five types of white blood cells(WBCs), also known as leukocytes or inflammatory cells,which are neutrophils with 40-70% percentage, eosinophilsof 5%, basophils of 0.5%, lymphocytes of 20-50% and mono-cytes of 1-5%. Differential Blood Count (DBC), which countsthe percentages of different blood cells in a blood sample, iscurrently a manual procedure carried out by the Hematolo-gists. Many automatic techniques have been developed togive a faster and more accurate ways to count the blood cells.These counting techniques generally consist of three steps:(a) (b) (c) (d) (e)Fig. 1: The five WBCs classes in blood smear: (a) basophil, (b) eosinophil,(c) lymphocyte, (d) monocyte, and (e) neutrophil.segmentation, feature extraction, and classification. The ac-curate segmentation of WBCs is a difficult and challengingproblem due to the complexities of cell structures, incoherentnature of cell cytoplasm, the high accumulation of red bloodcells and their interference with the WBC boundaries, and thelarge variations in shape, texture, color, and density amongthe five major types of WBCs, as shown in Fig. 1.The earliest approaches for WBCs segmenting are basedon thresholding [1], morphological operations [2, 3], and re-gion growing [4]. These methods, however, lead to unsatis-factory segmentation results due to the existence of the redblood cells. Some methods have been developed based on de-formable models, such as level set [5], and parametric activecontour [6–8]. However, these methods require good contourinitialization to obtain good segmentation results.Several methods that can be categorized into color-basedmethods have been proposed in the literature to exploit theadvantages of the color variation between different cell com-ponents in segmentation [9–17]. For instance, Gram-Schmidtmethod has been used in [10–12, 16] for leukocyte nucleussegmentation. These methods, however, are semi-automatic,where the desired and undesired color vectors used by Gram-Schmidt method need to be tuned manually by the users. Inaddition, the same nucleus may have different intensities andsaturation levels, as shown in Fig. 1 (d) and (e). Automaticcolor-based methods have been introduced in [13–15, 17]based on color and texture enhancement techniques. How-ever, these methods in [13, 14, 17] are unable to segment thecytoplasm, and the segmentation results of [15] still need tobe improved. This paper presents a fully automatic methodfor leukocyte nucleus and cytoplasm segmentation based oncolor characteristics of the white blood cells’ components.Fig. 2: The flowchart of the proposed method: (a) the original color blood image, (b) the processed nuclei image fNu, (c) the processed cytoplasm image fCy ,(d) the result of Gram-Schmidt method, (e) the result of SMMT, (f) the scaled image, (g) the corresponding histogram with the peak represents the thresholdused for thresholding, (h) the nuclei and cytoplasm masks during post-processing, and (i) the final segmentation.2 MethodThe designed segmentation framework consists of four steps:clustering-based color enhancement and reduction, nucleisegmentation, cytoplasm segmentation, and post-processing.Fig. 2 shows the steps of the proposed method.2.1 Color Enhancement/ ReductionThe color blood smear image provides important informationthat can help to get more accurate segmentation for nucleiand cytoplasm, whose intensities are very similar in the grayimage. In addition, different color channels provide differentlevel of information about the cell’s component. For instance,the red channel provides more information about the nuclei,whereas the green channel can better represent the WBC cy-toplasm.The challenge facing the color-based WBC segmentationmethods is the similarities in the color of some parts of the nu-clei and cytoplasm, and the similarities between the color ofthe lymphocytes and the red blood cell. To address this issue,we design a color enhancement and reduction technique, thatcan reduce the color range while preserving the cell contour,based on simple linear iterative clustering algorithm [18]. Inparticular, median filtering and intensity adjustment are ap-plied on the color image I to remove the noise and highlightthe colors. Clustering is then applied on the pre-processed im-age (Î) to divide the blood smear image into coherent regions.For each cluster, the median value for each color channel iscomputed and assigned to all pixels within the cluster. Thenuclei have a distinct color in the resulted image as shown inFig. 2 (b). Let’s represent this enhanced image by fNu, whichis used in the nuclei segmentation step.Segmentation of WBCs cytoplasm is more complex thannuclei segmentation due to the poor contrast and interfacingwith the red blood cells. Thus, further enhancement pro-cedure is performed. Specifically, decorrelation stretch en-hancement method [19] is applied on Î before applying thecolor enhancement procedure to enhance the cytoplasm con-trast. Then, the designed clustering-based color reductionmethod is applied on the green channel of the resulted im-age to obtain the cytoplasm image fCy used for cytoplasmsegmentation, as shown in Fig. 2 (c).2.2 Nucleus SegmentationFor nuclei segmentation, a priori information about eachWBC components color is used. A sample of images ispre-processed as described earlier, and used to define threecolor vectors: the nuclei color vector VNu, the cytoplasmcolor vector VCy , and the background color vector VBK bycomputing the mean pixel intensities of the RGB channels ofthe color blood image for each component. Then, the ’target’color vector, i.e., VNu, and the other vectors are passed tothe Gram-Schmidt orthogonalization method [20] to computea weighting vector w having the maximum orthogonalitywith the target vector VNu. The weighting vector w is thenused to highlight the target region and weaken the other re-gions in fNu, by simply performing the inner product of wwith fNu. The resulted image, shown in Fig. 2 (d), has thehighest intensities assigned to the nuclear region and lowerintensities to the other regions in the image. To smooth theimage and remove the fine details affecting the segmentationresults, self-dual multi-scale morphological toggle (SMMT)filter [21] is applied on the resulted image, and then, an-tistrophic diffusion filter (ADF) is utilized to highlight thesmoothed nuclei contour. Binarization is finally used to ob-tain the nuclei segmentation.2.3 Cytoplasm SegmentationMarker-controlled watershed is used for cytoplasm segmenta-tion using the cytoplasm image fCy . For seed extraction, thecolor range of the processed image Î is reduced and used toscale the cytoplasm image fCy to a new domain as shownin Fig. 2 (f). Fig. 2 (g) shows the histogram of scaledimage, where there is only one peak which represents thebackground, and the other intensities show the foregroundincluding the WBCs cytoplasm. Using the peak intensityas a threshold, the seed image is generated, and passed tothe marker-controlled watershed method to get the cytoplasmsegmentation.2.4 Post-processingThe post-processing step is three-fold: regularization, refine-ment, and filtering. For regularization, a disk-based majorityfilter [22] is applied on the nuclei candidates to reduce the fineartifacts and smooth their boundaries, whereas, a set of mor-phological operations (i.e., closing, filling, and dilation witha flat and disk-shaped structuring element) is performed toconnect the segmented cytoplasm and generate a solid mask.The refinement step uses level set [23] to smooth the candi-date contour and reduce the false pixels belonging to the redblood cells. According to our observation, the nuclei alwayshave low values in the red channel and clearly appear as fore-ground, whereas the green channel represents the cytoplasm.Therefore, the red channel is used in level set evolution fornuclei contour refinement and the green channel is used forcytoplasm contour refinement. Lastly, the false nuclei andcytoplasm candidates are removed based on the size and ec-centricity properties. Also, the nuclei candidates that are notsurrounded by cytoplasm, or the cytoplasm candidates with-out nuclei, are filtered out.3 Experimental ResultsThe performance of the proposed method was evaluated us-ing two WBC image datasets with different color, texture, andcell sizes: the first dataset [14] consists of 365 (640 × 480pixels) blood images, with the nuclei ground truth, whereasthe second dataset [24] includes 242 (720 × 576 pixels) bloodimages, with ground truth delineating the nucleus and cyto-plasm segmentation. The images in both datasets are charac-terized by different illumination and contrast conditions, anddifferences in shape, size, and spatial distribution of the cells.The images were also grouped into five leukocyte classes:basophil, eosinophil, lymphocyte, monocyte, and neutrophil.The results were quantitatively evaluated using the similaritymetrics (Ts), given by:Ts = 100×Oseg ∩Ogtmax (Oseg , Ogt)(1)where Oseg represents the segmented object by the proposedmethod and Ogt represents the ground truth. Moreover, thenuclei segmentation performance of the proposed method wascompared with the results of six nuclei segmentation meth-ods [10,13–17], and the cytoplasm segmentation performancewas compared with [15] on the second dataset, using the samesimilarity metrics.Table 1 presents our nucleus segmentation performancefor each type of WBCs. The proposed approach achievedlarge improvement over the other approaches on the firstdataset, with total on-average improvement of 12%. Thehighest nucleus segmentation result obtained by our approachTable 1: Evaluation of nuclei segmentation on Dataset 1 and 2.Dataset 1Baso. Eosi. Lymph. Mono. Neut. Overall[10] 75.7 88.9 79.6 89.6 82.3 83.2[13] 64.3 44.8 58.9 57.4 57.0 55.9[14] 80.4 69.3 83.8 86.3 80.3 79.7[16] 78.6 90.1 78.3 83.0 85.8 85.4[17] 81.0 71.0 84.5 85.9 81.2 80.6[15] 82.6 81.2 87.1 88.7 86.1 85.1Our app. 85.2 93.8 85.0 94.8 91.8 90.1Dataset 2[15] - 77.6 83.6 85.1 86.2 83.5Our app. 89.3 81.2 87.2 89.3 84.0 86.8Table 2: Evaluation of cytoplasm segmentation on Dataset 2.Baso. Eosi. Lymph. Mono. Neut. Overall[15] 90.6 91.0 79.7 90.4 91.6 88.7Our app. 93.6 92.7 87.2 81.6 93.1 89.6was for monocytes of 94.8% on the first dataset, and 89.3%on the second dataset, owing to the colourlessness and fuzzi-ness characteristic of the monocytes cytoplasm, which makesit easier to delineate the nucleus contour. The lowest accu-racy obtained by our approach was for lymphocyte of 85% onthe first dataset, and for eosinophils of 81.2% on the seconddataset. The is because of the color similarity between thenuclei and cytoplasm of these two kind of WBCs (see Fig.1(b) and (c)). However, the proposed approach provided agreat improvement in eosinophil segmentation of ∼20% onaverage, over other methods on the first dataset, and 4% onthe second dataset. The proposed method showed a good im-provement over two Gram-Schmidt-based methods [10, 16],with total 7% over [10], and 5% improvement over [16]. Theresults demonstrated that our clustering-based color enhance-ment procedure along with color learning could effectivelyimprove the nuclei segmentation results.Table 2 shows our cytoplasm segmentation performance.According to the table, the highest improvement achievedby our approach was for lymphocyte, with 8% improvementover [15]. Monocyte was found to have the lowest cytoplasmsegmentation accuracy of 81.6%, due to the ground-glass(translucent) nature of the monocyte cytoplasm, and ba-sophil has the highest accuracy of 93.6%, because of thedistinguishable color of this kind of WBCs cytoplasm. Thedesigned color reduction procedure succeeds in highlightingthe cytoplasm pixels, resulted in improving the cytoplasmsegmentation performance.Furthermore, the performance of the nuclei and cytoplasmsegmentation was qualitatively assessed, and a visual inspec-tion of the results presented in Fig. 3. As shown in the figure,the segmented nuclei and cytoplasm contours were close totheir ground truth contours.4 ConclusionsSegmentation and counting of white blood cells plays an im-portant and vital role in the diagnosis of various blood re-lated diseases, such as blood cancer. This paper introduces(a) (b) (c)Fig. 3: Nuclei and cytoplasm segmentation of basophil, eosinophil, lym-phocyte, monocyte, neutrophil, and mixture of WBCs, respectively. (a) theoriginal image, (b) haematologist segmentation, (c) our segmentation.a fully automatic segmentation framework for five types ofwhite blood cells in the peripheral blood smears. The pro-posed approach applies Gram-Schmidt method using priorinformation of leukocyte nuclei, cytoplasm, and backgroundcolors after clustering-based color enhancement process, andmulti-scale simplification procedure to segment the WBCsnuclei. For cytoplasm segmentation, marker-controlled wa-tershed with clustering-based seed extraction is performed.Finally, post-processing step is utilized to regularize, refine,and filter the segmented nuclei and cytoplasm areas. Twodifferent datasets of microscopic blood images were used inevaluation, and the proposed framework managed to obtain88.5% accuracy for the localization of the nuclei and 89.6%for cytoplasm segmentation. The results obtained indicatethat the proposed method can be effectively used for WBCsnuclei and cytoplasm localization with different datasets withsuperior performance than prior works, and subsequently, aid-ing the diagnosis of blood-related diseases.5 References[1] I. Cseke, “A fast segmentation scheme for white blood cell images,” inICPR. IEEE, 1992, pp. 530–533.[2] C Di Rubeto, A. Dempster, S. Khan, and B. Jarra, “Segmentation ofblood images using morphological operators,” in ICPR. IEEE, 2000,vol. 3, pp. 397–400.[3] D. Anoraganingrum, “Cell segmentation with median filter and mathe-matical morphology operation,” in ICIAP. IEEE, 1999, pp. 1043–1046.[4] J. Chassery and C. Garbay, “An iterative segmentation method basedon a contextual color and shape criterion,” IEEE Trans. Pattern Anal.Mach. Intell., , no. 6, pp. 794–800, 1984.[5] L. B. Dorini, R. Minetto, and N. Leite, “Semiautomatic white blood cellsegmentation based on multiscale analysis,” IEEE J. Biomed. HealthInform., vol. 17, no. 1, pp. 250–256, 2013.[6] M. Hamghalam, M. Motameni, and A. Kelishomi, “Leukocyte seg-mentation in giemsa-stained image of peripheral blood smears basedon active contour,” in ICSPS. IEEE, 2009, pp. 103–106.[7] F. Zamani and R. Safabakhsh, “An unsupervised gvf snake approachfor white blood cell segmentation based on nucleus,” in ICSP. IEEE,2006, vol. 2.[8] G. Ongun, U. Halici, K. Leblebicioglu, V. Atalay, M. Beksac, andS. Beksac, “An automated differential blood count system,” in EMBC.IEEE, 2001, vol. 3, pp. 2583–2586.[9] J. Wu, P. Zeng, Y. Zhou, and C. Olivier, “A novel color image segmen-tation method and its application to white blood cell image analysis,”in ICSP. IEEE, 2006, vol. 2.[10] S. H. Rezatofighi, H. Soltanian-Zadeh, R. Sharifian, and R. A. Zoroofi,“A new approach to white blood cell nucleus segmentation based ongram-schmidt orthogonalization,” in ICDIP. IEEE, 2009, pp. 107–111.[11] S. H. Rezatofighi and H. Soltanian-Zadeh, “Automatic recognition offive types of white blood cells in peripheral blood,” Comput. Med.Imaging Graph., vol. 35, no. 4, pp. 333–343, 2011.[12] P. R. Tabrizi, S. H. Rezatofighi, and M. J. Yazdanpanah, “Using pcaand lvq neural network for automatic recognition of five types of whiteblood cells,” in EMBC. IEEE, 2010, pp. 5593–5596.[13] H. Madhloom, S. Kareem, H. Ariffin, A. Zaidan, H. Alanazi, andB. Zaidan, “An automated white blood cell nucleus localization andsegmentation using image arithmetic and automatic threshold,” J. Appl.Sci., vol. 10, pp. 959–966, 2010.[14] M. Mohamed, B. Far, and A. Guaily, “An efficient technique for whiteblood cells nuclei automatic segmentation,” in SMC. IEEE, 2012, pp.220–225.[15] A. Tareef, Y. Song, W. Cai, Y. Wang, D. Feng, and M. Chen, “Auto-matic nuclei and cytoplasm segmentation of leukocytes with color andtexture-based image enhancement,” in ISBI. IEEE, 2016, pp. 935–938.[16] M. Mohamed and B. Far, “A fast technique for white blood cells nucleiautomatic segmentation based on gram-schmidt orthogonalization,” inICTAI. IEEE, 2012, vol. 1, pp. 947–952.[17] M. Mohamed and B. Far, “An enhanced threshold based technique forwhite blood cells nuclei automatic segmentation,” in Healthcom. IEEE,2012, pp. 202–207.[18] R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Susstrunk,“SLIC superpixels compared to state-of-the-art superpixel methods,”IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 11, pp. 2274–2282, 2012.[19] D. A. Rothery, “Cover decorrelation stretching as an aid to image in-terpretation,” Int. J. Remote Sens., vol. 8, no. 9, pp. 1253–1254, 1987.[20] H. Soltanian-Zadeh, D. Peck, D. Hearshen, and R. Lajiness-O’Neill,“Model-independent method for fmri analysis,” IEEE Trans. Med.Imag., vol. 23, no. 3, pp. 285–296, 2004.[21] L. Dorini and N. Leite, “A scale-space toggle operator for morpholog-ical segmentation,” ISMM, pp. 101–112, 2007.[22] T. Lillesand, R. Kiefer, and J. Chipman, Remote sensing and imageinterpretation, John Wiley & Sons, 2014.[23] I. Rianto and P. Pranowo, “Distance regularized level set evolution formedical image segmentation,” in CITACEE, 2013, vol. 1, pp. 49–51.[24] “LISC: Leukocyte Images for Segmentation and Classification,”http://users.cecs.anu.edu.au/hrezatofighi/Data/Leukocyte%20Data.htm.

Automated Multi-Stage Segmentation of White Blood Cells Via Optimizing Color Processing

https://ses.library.usyd.edu.au/bitstream/2123/29783/1/ISBI_2017_Afaf_v6-FINAL.pdf

Automated Multi-Stage Segmentation of White Blood Cells Via Optimizing Color Processing

Abstract

Similar works

Full text

Available Versions

Crossref

Sydney eScholarship