Jensen, Sebastian H. Nesgaard

Moeslund, Thomas B.

Rankl, Christian

English

In this paper, we propose an approach for achieving generalized segmentation of microorganisms in mi- croscopy images. It employs a pixel-wise classification strategy based on local features. Multilayer percep- trons are utilized for classification of the local features and is trained for each specific segmentation problem using supervised learning. This approach was tested on five different segmentation problems in bright field, differential interference contrast, fluorescence and laser confocal scanning microscopy. In all instance good results were achieved with the segmentation quality scoring a Dice coefficient of 0.831 or higher

    Aalborg UniversitetGeneral Purpose Segmentation for Microorganisms in Microscopy ImagesJensen, Sebastian H. Nesgaard; Moeslund, Thomas B.; Rankl, ChristianPublished in:9th International Conference on Computer Vision Theory and ApplicationsPublication date:2014Document VersionEarly version, also known as pre-printLink to publication from Aalborg UniversityCitation for published version (APA):Jensen, S. H. N., Moeslund, T. B., & Rankl, C. (2014). General Purpose Segmentation for Microorganisms inMicroscopy Images. In 9th International Conference on Computer Vision Theory and Applications Institute forSystems and Technologies of Information, Control and Communication.General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.            ? Users may download and print one copy of any publication from the public portal for the purpose of private study or research.            ? You may not further distribute the material or use it for any profit-making activity or commercial gain            ? You may freely distribute the URL identifying the publication in the public portal ?Take down policyIf you believe that this document breaches copyright please contact us at vbn@aub.aau.dk providing details, and we will remove access tothe work immediately and investigate your claim.Downloaded from vbn.aau.dk on: December 27, 2020General Purpose Segmentation for Microorganisms in MicroscopyImagesS.N. Jensen and T.B. MoeslundVisual Analysis of People LabAalborg University, Denmarktbm@create.aau.dkChristian RanklAgilent Technologies, Measurement Research Laboratory, AustriaKeywords: Cell Segmentation, Microscopy Image Analysis, Object Detection, Pixel ClassificationAbstract: In this paper, we propose an approach for achieving generalized segmentation of microorganisms in mi-croscopy images. It employs a pixel-wise classification strategy based on local features. Multilayer percep-trons are utilized for classification of the local features and is trained for each specific segmentation problemusing supervised learning. This approach was tested on five different segmentation problems in bright field,differential interference contrast, fluorescence and laser confocal scanning microscopy. In all instance goodresults were achieved with the segmentation quality scoring a Dice coefficient of 0.831 or higher.1 INTRODUCTIONMicroscopy is the art of observing objects whichare normally too small to be seen by the unaidedhuman eye. It is one of the most important infor-mation gathering tools in many different fields suchas microbiology and have remained so since it’sconception over a hundred years ago. One of themain applications of microscopy is the observationof microorganisms, an important endeavor in mi-crobiology and medical science. Through visualmagnification of a 100 times or more, a wealth ofvisual information can be extracted from even thetiniest specimens or samples. In fact information canbe so plentiful that a thorough analysis can be quitea time consuming task. Combining microscopy withadditional techniques such as time-lapse videos andz-layering, can make the amount of information evenmore staggering. This calls for means by which theanalysis tasks may be partially or fully automated.Fortunately many microscopy analysis tasks can,on their base level, be boiled down to locatinginstances of one or several specific classes withinan image, examples of this includes cancer (Wienertet al., 2012) and malaria diagnostics (F. Boray Tekand Kale, 2009a). This means that the bulk of theanalysis work can be automated by developing ageneral method for detection microorganisms withinmicroscopy images. Unfortunately this can be aquite challenging problem due to visual variation,which stems from the employed microscopy type andspecific species of microorganism. A few examplesof this behavior can be observed in figure 1.In this positional paper we will present the pre-liminary work for a single segmentation method thatis capable of handling many of these visually varyingproblems.2 Related WorkAutomatic microscopy image analysis is by nomeans a new topic of research and material on thematter have been published for more than 50 years.Regarding the specific subject of cell segmentation, asurvey is illustrated in figure 2 which nicely detailsthe number article published on cell segmentationevery 5 years as well as their overall methodologyfor solving the problem. Now it clearly shows thatthe amount of research gone into cell segmentationhas been steadily increasing over the last 50 years,indicating a clear interest in solving the problem.However the lower part of figure 2 illustrates a majorissue with the research that has been performed; itsuffers from a very divergent methodology. AmongstFigure 1: Examples of the visual variety posed by differ-ent species of microorganisms and types of microscopy.Contains images produced using bright field(Wienert et al.,2012)(bin Abdul Jamil et al., 2012), differential interfer-ence contrast, fluorescence(Institute, 2013) and phase con-trast microscopy(Kane et al., 2013).the utilized methods for achieving microorganismdetection are variations of watershed segmentation(Lebrun et al., 2007) (Ao et al., 2011) (Cheng andRajapakse, 2009) (bin Abdul Jamil et al., 2012), con-tour and shape based segmentation (Wienert et al.,2012) (Zhou, 2007) (Kujiper and Heise, 2008), colorand intensity based segmentation (Zhaozhen Yingand Kanade, 2010) (F. Boray Tek and Kale, 2009b)(F. Boray Tek and Kale, 2009a), wavelet based de-tection (Ariel J. Bernal and Bernal, 2008) and shapebased detection (Kevin Smith and Lepetit, 2009).Additionally there is a clear trend towards specializedsolutions, as stated in (Meijering, 2012) and whichmay be observed in the previously mentioned work.The program CellProfiler is capable of handlingmany microorganism detection problems, however itdoes so by simply letting the user manually choosethe specific segmentation method which is to beemployed(Carpenter et al., 2006).Figure 2: This figure illustrates the progression of researchin cell segmentation over the past 50 years, both in termsof quantity and methodological distribution. It shows thatno real consensus has been achieved on the approach forcell segmentation. The graph was obtained from (Meijer-ing, 2012).This article will contribute to the overall stateof the art by introducing a single segmentationmethod that may determine an optimal segmentationrule for a specific microorganism detection problem.Segmentation is chosen as the base of the detectionstrategy as it is capable of dealing with the greatvariations in shape, orientation and appearancethat microorganism naturally exhibit in microscopyimages.3 MethodThe main principle of this work is the collectionand classification of pixel-wise local features. This isaccomplished by having a machine learning algorithmdetermine the optimal classification rule through su-pervised learning. The reason for this approach is thatit provides a simple framework with a constant struc-ture capable of handling a wide array of microorgan-ism segmentation problems, as long as the objects arelocally distinguishable.3.1 FeaturesIt is difficult to ensure that a given feature set can dis-tinguish between background and object in all plausi-ble situation. However, it was experimentally foundthat in many specific problems good segmentationcould be achieved using a feature set based on colorand gradient information. It is defined below,F̄(x,y) =Ī(x,y)E [Ī(x,y)]Var [Ī(x,y)]ḡ(x,y)E [ḡ(x,y)]Var [ḡ(x,y)] (1)Where,F̄(x,y) is the pixel feature-vector at (x,y),Ī(x,y) is the pixel value at (x,y),ḡ(x,y) is the pixel gradient magnitude at (x,y),Both the pixel value and gradient magnitude functionsare denoted as being vectors as pixel values are as-sumed to be defined in the HSV color space. Theexpectation and variance operators both refer to ele-ment wise operations. The feature is evaluated on asquare area centered on the pixel in question. Eval-uating both variance and mean can get quite com-putationally expensive for larger areas. To remedythis problem a moving average technique is utilizedto drastically improve real-time performance. For ex-ample the mean may be efficiently evaluated using thefollowing equation,µ̄(x+1,y) =µ̄(x,y)+1(2w+1)2[w∑j=−wĪ(x+w+1,y+ j)−w∑j=−wĪ(x−w,y+ j)](2)Where,µ̄I(x,y) is the mean value of pixel (x,y),Ī(x,y) is the value of pixel (x,y),w is the width of the square evalutation area3.2 ClassificationMultilayer perceptrons with sigmoid activation func-tions was utilized for pixel classification in thismethod, which contained a single input, output andhidden layer. Each neuron within the network is de-fined as,a j = f(c j ∑iwi, jai)(3)Where,a j is the output of the jth neuron,f is an activation function,c j is the neuron gain,ai is the ith input to the neuron,wi, j is the weight of the edge from ith to jth neuron.As this network is used for classification, the activa-tion function for each neuron within the hidden andoutput layer is a sigmoid function which is definedbelow,f (z) =11+ e−z(4)In this particular work the multilayer perceptrons isused to estimate the likelihood of a given feature be-longing to either the background or object class. Thismeans that the network has two output nodes for eachclass, each of which outputs the likelihood for oneof the given classes. In the training phase, the out-put corresponding to a given training sample featureis either [0, 1] or [1, 0] depending on the class whichthe feature belongs to. The final classification is thenperformed using Bayesian decision,class(x̄) ={O if P(O)P(x̄|O)> P(B)P(x̄|B)B else(5)Where,x̄ is an input feature,O is the object class,B is the background class,A binary image is achieved by numerically represent-ing the background and object class respectively as 0and 1. The network is trained using a variation of thestandard backpropagation, whose main difference lieson that it uses both first and second order derivativesto estimate the optimal search direction during opti-mization of neuron weight. At iteration n the optimalsearch direction is,p̄n =−H−1n ḡw,n. (6)Where,p̄n is the optimal search direction,Hn is the Hessian matrix,ḡw,n is the gradient.Now this search direction estimate leads to fewer iter-ation before convergence, but both the Hessian andit’s inverse are relative expensive to estimate(Nawiet al., 2006). To remedy this problem the Broyden-Fletcher-Goldfarb-Shanno (BFGS) recursive iterationscheme is used to accurately approximate the Hes-sian using the following set of equations(Nawi et al.,2006),s̄n = w̄n+1− w̄n (7)ȳn = ḡw,n+1− ḡw,n (8)H−1n+1 = H−1n +(1+ȳTn H−1n ȳns̄Tn ȳn)s̄ns̄Tns̄Tn ȳn− s̄nȳTn H−1ns̄Tn ȳn(9)Where,w̄n is a vector containing the neuron weights.As long as H−1n is initialized as a positive definite ma-trix, the above will converge towards the true Hes-sian(Nawi et al., 2006). The training algorithm is de-scribed in detail in (Nawi et al., 2006). It is importantthat the training algorithm is supplied with an equalamount of samples from each class. Having one classvastly overrepresented leads to classification bias, aneffect which is studied in detail in (Brain, 2003). Wetherefore subsample the overrepresented class.4 Experimental ResultsThe method has been tested on four image setscontaining different microorganisms and producedusing different microscopy types. Each image setcontains approximately 20 images and the resultshave been achieved by running the segmentationmethod on all of them and performing comparisonsto expert produced ground truths. The main purposeis to illustrate the viability of the method on a varietyof different detection problems. Segmentation qualityis gauged using the Dice coefficient rather than pixel-wise classification error, it is defined below (Ao et al.,2011),D =2 · |Sauto⋂Smanual||Sauto +Smanual|(10)Where,D is the Dice coeffecient,Sauto is the set of segmentation object pixels,Smanual is the set of ground truth object pixels.Image Set Dice SubsampleFluorescence 0.831Bright Field 0.851Laser Scanning 0.952DIC Wound 0.963Table 1: Table of experimental test results.Now the dice coefficient for each image set was es-timated by utilizing a k-fold procedure with 4 folds.A example image along with an example segmenta-tion for each image set can be seen in figure 3, 4,5 and 6. The image sets were obtained from onlinemicroscopy image databases provided by (Institute,2013) and (for Bio-Image Informatics, 2013). The re-sults of each test can be observed in table 1. In all testhigh Dice coefficients were achieved, this indicatesthat the method is very versatile and can handle awide range of segmentation problems. The results ob-tained from the DIC Wound image set can be directlycompared to those achieved by (Zaritsky et al., 2011),who designed a segmentation algorithm for this par-ticular problem. In their work they achieved an aver-age pixel-wise segmentation accuracy of 0.922, whichis very much comparable to the Dice coefficient of0.963 obtained by our method. In (Ao et al., 2011)a method is presented for segmenting cancer cells inmicroscopy images. They achieved an average Dicecoefficient of 0.9 which was deemed to be an accept-able degree of accuracy. While the data set used in(Ao et al., 2011) was not available for testing ourmethod thus preventing a direct comparison, it doesindicate the needed level of precision. As the resultsobtained with our method are very close to the re-sults obtained in (Ao et al., 2011), this demonstratesthat our method is capable of producing segmentationquality on par with others, without being specificallydesigned for a particular problem.5 ConclusionIn this paper it has been shown that a wide rangeof segmentation problems in microscopy can besolved through pixel-wise classification of localfeatures. These are based on the mean and varianceof color and gradient magnitude evaluated in an areacentered locally on each pixel. A classifier, in thiscase multilayer perceptrons, is automatically trainedto each specific segmentation problem, using userlabeled sample features. In order to document theviability of the method, it was tested on five differentimage set produced using bright field, fluorescence,differential interference contrast and laser confocalscanning microscopy. In all cases good results wereobtained with a Dice coefficient ranging from 0.831to 0.963.In future work this method will be subjected tomore systematic tests in order to closely investigateit’s properties. Additionally more data producedusing different kinds of microscopy and microor-Figure 3: Example segmentation of an image from theBright Field set. Input image at the top, ground truth inthe middle and segmentation at the bottom.ganisms will be utilized in order to fully gauge thepotential and limits of the method.REFERENCESAo, J., Mitra, S., Long, R., Nutter, B., and Antani, S. (2011).A hybrid watershed method for cell image segmenta-tion. IEEE Southwest Symposium on Image Analysisand Interpretation.Ariel J. Bernal, S. E. F. and Bernal, L. J. (2008). Cell recog-nition using wavelet templates. Canadian Conferenceon Electrical and Computer Engineering.bin Abdul Jamil, M. M., Sharif, J. M., Miswan, M. F.,Ngadi, M. A., and Salam, M. S. H. (2012). Red bloodFigure 4: Example segmentation of an image from the Flu-orescence set. Input image at the top, ground truth in themiddle and segmentation at the bottom.cell segmentation using masking and watershed algo-rithm: A preliminary study. International Conferenceon Biomedical Engineering.Brain, D. (2003). Learning From Large Data: Bias, Vari-ance, Sampling and Learning Curves. PhD thesis,Deakin University.Carpenter, A., Jones, T., Lamprecht, M., and et al (2006).CellProfiler: image analysis software for identifyingand quantifying cell phenotypes. Genome Biology.Cheng, J. and Rajapakse, J. C. (2009). Segmentationof clustered nuclei with shape markers and markingfunction. IEEE Transactions on Biomedical Engineer-ing.F. Boray Tek, A. G. D. and Kale, I. (2009a). ComputerVision for Microscopy Diagnosis of Malaria. MalariaJournal.F. Boray Tek, A. G. D. and Kale, I. (2009b). Malaria Par-asite Detection in Peripheral Blood Images. IEEE In-ternational Conference on Acoustics, Speech and Sig-nal Processing.for Bio-Image Informatics, C. (2013). Ucsb bio-segmentation benchmarking.Institute, B. (2013). Broad bioimage benchmark collection.Kane, C., Iwasa, J., Orloff, D., and Wong, W. (2013). Thecell: An image library.Figure 5: Example segmentation of an image from the LaserScanning set. Input image at the top, ground truth in themiddle and segmentation at the bottom.Kevin Smith, A. C. and Lepetit, V. (2009). Fast ray featuresfor learning irregular shapes. Internation Conferenceon Computer Vision.Kujiper, A. and Heise, B. (2008). An automated cell seg-mentation method for differential interference con-trast microscopy. International Conference on PatternRecognition.Lebrun, G., Charrier, C., Lezoray, O., Meurie, C., and Car-dot, H. (2007). A Fast And Efficient SegmentationScheme For Cell Microscopic Image. Cellular andMolecular Biology.Meijering, E. (2012). Cell segmentation: 50 years down theroad. IEEE Signal Processing Magazine.Nawi, N. M., Ransing, M. R., and Ransing, R. S. (2006). Animproved learning algorithm based on the broyden-fletcher-goldfarb-shanno (bfgs) method for back prop-agation neural networks. International Conference onIntelligent Systems Design and Applications.Wienert, S., Heim, D., Saeger, K., Stenzinger, A., Beil, M.,Hufnagl, P., Dietel, M., Denkert, C., and Klauschen,F. (2012). Detection and segmentation of cell nucleiin virtual microscopy images; a minimum-model ap-proach. Scientific Reports.Zaritsky, A., Natan, S., Horev, J., Hecht, I., Wolf, L., Ben-Jacob, E., and Tsarfaty, I. (2011). Cell motility dy-namics: A novel segmentation algorithm to quantifymulti-cellular bright field microscopy images. PLoSONE.Figure 6: Example segmentation of an image from the DICWound set. Input image at the top, ground truth in the mid-dle and segmentation at the bottom.Zhaozhen Ying, Ryoma Bise, M. C. and Kanade, T. (2010).Cell segmentation in microscopy imagery using a bagof local bayesian classifiers. The IEEE InternationalSymposium on Biomedical Imaging.Zhou, Y. (2007). Cell segmentation using level set method.Technical report, Institute for Computational and Ap-plied Mathematics, Johannes Kepler University, Linz.

General Purpose Segmentation for Microorganisms in Microscopy Images

https://vbn.aau.dk/ws/files/146869112/GeneralSegmentation_no_ack.pdf

General Purpose Segmentation for Microorganisms in Microscopy Images

Abstract

Similar works

Full text

Available Versions

VBN