Deep learning techniques bring together key advantages in biomedical image
segmentation. They speed up the process, increase the reproducibility, and reduce the workload
in segmentation and classification. Deep learning techniques can be used for analysing cell
concentration, cell viability, as well as the size and form of each cell. In this study, we develop
a deep learning model for automated segmentation of fluorescence cell images, and apply it
to fluorescence images recorded with a home-built epi-fluorescence microscope. A deep neural
network model based on U-Net architecture was built using a publicly available dataset of
cell nuclei images [1]. A model accuracy of 97.3% was reached at the end of model training.
Fluorescence cell images acquired with our home-built microscope were then segmented using
the developed model. 141 of 151 cells in 5 images were successfully segmented, revealing a
segmentation success rate of 93.4%. This deep learning model can be extended to the analysis
of different cell types and cell viability

Acılan, Ceyda

Aydın, Musa

Eren, Furkan

Kiraz, Alper

Kiraz, Berna

Morova, Berna

Uysallı, Yiğit

Özcan, Selahattin Can

English

DSpace@FSM Vakif University

Journal of Physics: Conference SeriesPAPER • OPEN ACCESSA Deep Learning Model for AutomatedSegmentation of Fluorescence Cell imagesTo cite this article: Musa Aydn et al 2022 J. Phys.: Conf. Ser. 2191 012003 View the article online for updates and enhancements.You may also likeElectrochemical Impedance SpectroscopicStudy on Polypyrrole/BariumTitanate/Poly(acrylonitrile-co-methylacrylate) NanoparticlesF. Z. Engin Sagirli, E. S. Kayali and A. S.Sarac-Comment on “Underlining possible effectsof Coriolis acceleration in experiments witha Mössbauer source” by Benedetto Elmoet al.Alexander Kholmetskii, Tolga Yarman,Ozan Yarman et al.-Analysis of atherosclerosis by inductivelycoupled plasma optical emissionspectroscopy and scanning acousticmicroscopyBukem Tanoren, Murat Ugurlucan andFatma Ates Alkan-This content was downloaded from IP address 95.0.60.60 on 04/03/2022 at 07:21Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence. Any further distributionof this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.Published under licence by IOP Publishing LtdDERELI-FS-2021Journal of Physics: Conference Series 2191 (2022) 012003IOP Publishingdoi:10.1088/1742-6596/2191/1/0120031A Deep Learning Model for AutomatedSegmentation of Fluorescence Cell imagesMusa Aydın1∗, Berna Kiraz1,2, Furkan Eren2, Yiğit Uysallı2,3, BernaMorova3, Selahattin Can Ozcan4, Ceyda Acilan4,5, and AlperKiraz2,3,4,61Department of Computer Engineering, Fatih Sultan Mehmet Vakif University, Istanbul,Turkey2Optofil Inc., Istanbul, Turkey3Department of Physics, Koç University, Istanbul, Turkey4Koç University Research Center for Translational Medicine (KUTTAM), Istanbul, 34450,Turkey5School of Medicine, Koç University, Istanbul, 34450, Turkey6Department of Electrical and Electronics Engineering, Koç University, Istanbul, TurkeyE-mail: maydin@fsm.edu.tr1∗, bkiraz@fsm.edu.tr1, akiraz@ku.edu.tr3Abstract. Deep learning techniques bring together key advantages in biomedical imagesegmentation. They speed up the process, increase the reproducibility, and reduce the workloadin segmentation and classification. Deep learning techniques can be used for analysing cellconcentration, cell viability, as well as the size and form of each cell. In this study, we developa deep learning model for automated segmentation of fluorescence cell images, and apply itto fluorescence images recorded with a home-built epi-fluorescence microscope. A deep neuralnetwork model based on U-Net architecture was built using a publicly available dataset ofcell nuclei images [1]. A model accuracy of 97.3% was reached at the end of model training.Fluorescence cell images acquired with our home-built microscope were then segmented usingthe developed model. 141 of 151 cells in 5 images were successfully segmented, revealing asegmentation success rate of 93.4%. This deep learning model can be extended to the analysisof different cell types and cell viability.1. IntroductionManual image segmentation performed by an expert is considered as the gold standard; however,it is a time-consuming process, which constantly requires expert intervention and can result inhigh variations due to different interpretations of experts. Besides, it is prone to human errorsdue to the background noise that may reduce contrast of the objects to be segmented. Theseshortcomings of manual segmentation can be overcome by deep learning techniques which havebeen widely used for biomedical image segmentation to speed up the process, increase thereproducibility, and reduce the workload in manual segmentation.Convolutional neural network (CNN) is a deep neural network that extracts importantfeatures of an input with the application of different kernels. CNNs have been applied to manybiomedical image segmentation problems such as cell segmentation, brain vessel segmentation,and brain tumor segmentation [2, 3, 4, 5, 6]. Among different CNN architectures, U-Net modelDERELI-FS-2021Journal of Physics: Conference Series 2191 (2022) 012003IOP Publishingdoi:10.1088/1742-6596/2191/1/0120032is often selected as the architecture of choice for biomedical image segmentation due to its highperformance and accuracy [7, 8].In this study, we develop a deep learning model based on U-Net architecture for automatedsegmentation of fluorescence cell images. We trained our model using a publicly available datasetthat contains images of cell nuclei as well as corresponding ground truths [1], and obtained atraining accuracy of 97.3%. We then applied the trained model to automated segmentationof cell images obtained with a home-built epi-fluorescence microscope. Fluorescence imageswere acquired using U2OS cells stained with DAPI (6-diamidino-2-phenylindole) that enablesimaging of nuclear DNA. 141 of 151 cells were successfully segmented in 5 fluorescence images,corresponding to a segmentation success rate of 93.4%.2. Experimental StudyIn this section we describe the dataset used for segmentation of cell images with the deepneural network together with the deep learning method and algorithm. For the images in thedataset, image pre-processing was applied before training in the deep neural network. Thepre-processing steps were histogram matching and median filtering. Histogram matching is theprocess of transforming the histogram of one image to match with the histogram of anotherimage [9]. In our implementation, the histogram of each image in the dataset was matched tothe histogram function distribution of the image with the highest average light intensity value.This way, contrast differences between images were reduced. After that, a 3 × 3 median filterwas applied to all images in order to eliminate the background noise. The dataset used in thisstudy and all the developed codes can be accessed at github1.2.1. DatasetIn this study we used the publicly available dataset of cell nuclei images [1] which consisted of670 images encoded with Portable Network Graphics format (.png) defined in RGB color spacewith corresponding binary ground truths that are manually created by experts. Figure 1 showssample images and ground truths from the dataset.Figure 1. Exemplary input images and the corresponding ground truths from the dataset usedfor cell segmentation.The sizes of the images in the cell dataset are different from each other. Thus, all images wereresized to 256× 256 in the image pre-processing step. 80% of the 670 images in the cell dataset1 https://github.com/msaaydin/U-net-SegmentationDERELI-FS-2021Journal of Physics: Conference Series 2191 (2022) 012003IOP Publishingdoi:10.1088/1742-6596/2191/1/0120033was randomly selected for model training while the remaining 20% of was used for tests. Eachimage selected for training or tests also has ground truths available in the dataset. For groundtruths, merging was done in the image pre-processing step. An image found in the original dataset was manually segmented as follows. Initially, multiple ground truth images were created andsaved corresponding to each cell in a reference image. As a result, more than one segmentedground truth images were created for a reference image. The ground truths corresponding toall images were then merged into a single image during the image pre-processing step, and thedata set was updated. This way a new updates dataset was obtained. The new updated datasetcan be accessed on github1.2.2. MethodIn this study we employed the U-Net convolutional neural network that has a proven trackrecord in numerous biomedical image segmentation studies in the literature [10]. The U-Netmodel employed, consisted of two main parts: Encoder and decoder. The main purpose of theencoder and decoder architecture used in the U-Net model is to enable concatenation of theinformation at different abstraction levels. Encoder reduces the dimensions of the features mapsproduced by convolutional layers using max pooling operations. This way low resolution, higherlevel information maps are constructed. These maps contain detailed features of the objectsin the image which are sufficient for defining them while lacking many details. These detailsare then recovered by concatenating the information layers generated by the encoder with highresolution, lower level information maps generating with upsampling operations of the decoder.Figure 2. Diagram showing the encoder and decoder parts of the U-Net model developed forcell segmentation.The first part, i.e. encoder, is known as the contracting path. Contracting path is aconvolutional network containing steps where convolutions are performed, reducing the inputDERELI-FS-2021Journal of Physics: Conference Series 2191 (2022) 012003IOP Publishingdoi:10.1088/1742-6596/2191/1/0120034dimensions. At each step of the contracting path, the data size is downsampled with max poolingoperations (stride=2). Each step in the contracting path contains 2 convolutional layers with3 × 3 filter size, employing rectified linear unit (ReLU) activation with zero padding. Featuresize at the input layer is 16 and this number is doubled at each step of the contracting path.The second part of the U-Net model, i.e. decoder, is the expansive path where, at each step,the results obtained from 2 × 2 upsampling (stride=2) are concatenated with feature channelsof the contructing path that are obtained from the same steps. Following concatenation, eachstep of the expansive path employs 2 convolutional layers with 3× 3 filter size, employing ReLUactivation with zero padding. After each upsampling step in the expansive path, the numberof feature channels is halved. Hence, the contracting and expansive paths are symmetrical.In the final step of the U-Net model, a 1 × 1 convolution is implemented in order to extractthe segmentation map from 16 feature maps. Due to the limited data size, each convolutionallayer is followed by dropout operation with 0.2 ratio in order to prevent overfitting. All theoperations (e.g. convolution, max pooling, upsampling, donwsapmling) used in the U-Net modelconstructed in this work are shown in Fig. 2.Computational experiments were performed using the Google colab infrastructure and keras-tensorflow (version 2.4.1, python version = 3.7) library. In our U-Net network architecture, 16feature maps were extracted in the first step, and propagated with 4 downsampling steps wherethe size of the feature map is doubled at each step (Fig. 2). U-Net model hyperparameterswere selected as: Optimizer: Adam, epoch: 40, loss: binary crossentropy, metrics: [accuracy],learning rate: 0.001. 10% of the dataset was used for model validation together with a batchsize of 4 at each iteration.3. Results and DiscussionFigure 3. Changes of (a) accuracy and (b) loss with number of batches in cell segmentationduring U-Net model training.Figure 3 shows the accuracy and loss curves as a function of number of batches obtainedduring training and validation cycles. A very high accuracy of 97.3% was obtained with testsat the end of model training. The accuracy value was obtained as the average of accuraciesobtained from 10 tests conducted.DERELI-FS-2021Journal of Physics: Conference Series 2191 (2022) 012003IOP Publishingdoi:10.1088/1742-6596/2191/1/0120035Figure 4. Exemplary input images and ground truths used for U-Net model training togetherwith U-Net segmentation results.Exemplary input images and U-Net segmentation results obtained with our model togetherwith corresponding ground truths are shown in Fig. 4. A very good match is visible betweenthe U-Net segmentation results and ground truths, as expected from the high accuracy valuesobtained in model training. We then applied our trained model to five fluorescence microscopycell images recorded in our laboratory. These fluorescence images are recorded using a customepi-fluorescence microscope which employs LED excitation at 4 colors (wavelengths of LEDs are405, 488, 532, 633 nm), a 20X microscope objective (Nikon 20X, NA=0.4), and a CMOS cameradetector (Basler daA3840-45um). Fluorescence images were collected from U2OS cells whichFigure 5. Input images obtained using our home-built epi-fluorescence microscope and thecorresponding segmented images obtained as a result of segmentation with the U-Net modeldeveloped in this study.were grown on coverslips and fixed with ice cold methanol at -20◦C for 10 min, washed 3 timesDERELI-FS-2021Journal of Physics: Conference Series 2191 (2022) 012003IOP Publishingdoi:10.1088/1742-6596/2191/1/0120036with PBS, and mounted on glass slides using Vectashield mounting medium containing DAPI(4,6-diamidino-2-phenylindole) (Vector Laboratories; H-1000-10). Segmentation results of theseimages are shown in Fig. 5. A very high accuracy in cell segmentation was also achieved withthis dataset of 5 images. Out of a total of 151 cells in the images shown in Fig. 5, 141 of themwere correctly segmented, corresponding to a segmentation success rate of 93.4%.In conclusion, in this study we developed a U-Net model for cell segmentation and trainedour model using a publicly available dataset of cell images. An accuracy of 97.3% was obtainedat the end of model training. We then employed this trained model to cell segmentation using5 fluorescence images recorded in our experimental setup, and successfully segmented 141 of151 cells, corresponding to a segmentation success rate of 93.4%. This work can be extendedto segmentation of images that contain different cell types and to classification for cell viabilityanalysis. We are also planning to use such a U-Net based image segmentation approach to otherbiomedical image processing problems including brain vasculature analysis.AcknowledgmentsWe acknowledge financial supports from TÜBİTAK (Project No: 7190434) and KOSGEB. A.Kiraz acknowledges partial support from the Turkish Academy of Sciences (TÜBA).References[1] “Cell nuclei data set,” https://www.kaggle.com/c/data-science-bowl-2018/data, last access: 01-03-2021.[2] R. Hollandi, A. Szkalisity, T. Toth, E. Tasnadi, C. Molnar, B. Mathe, I. Grexa, J. Molnar, A. Balind,M. Gorbe et al., “nucleaizer: a parameter-free deep learning framework for nucleus segmentation usingimage style transfer,” Cell Systems, vol. 10, no. 5, pp. 453–458, 2020.[3] M. Habibzadeh, M. Jannesari, Z. Rezaei, H. Baharvand, and M. Totonchi, “Automatic white blood cellclassification using pre-trained deep learning models: Resnet and inception,” in Tenth internationalconference on machine vision (ICMV 2017), vol. 10696. International Society for Optics and Photonics,2018, p. 1069612.[4] M. Delgado-Ortet, A. Molina, S. Alférez, J. Rodellar, and A. Merino, “A deep learning approach forsegmentation of red blood cell images and malaria detection,” Entropy, vol. 22, no. 6, p. 657, 2020.[5] M. R. K. Mookiah, S. Hogg, T. J. MacGillivray, V. Prathiba, R. Pradeepa, V. Mohan, R. M. Anjana, A. S.Doney, C. N. Palmer, and E. Trucco, “A review of machine learning methods for retinal blood vesselsegmentation and artery/vein classification,” Medical Image Analysis, p. 101905, 2020.[6] M. Havaei, A. Davy, D. Warde-Farley, A. Biard, A. Courville, Y. Bengio, C. Pal, P.-M. Jodoin, andH. Larochelle, “Brain tumor segmentation with deep neural networks,” Medical image analysis, vol. 35,pp. 18–31, 2017.[7] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical imagesegmentation,” in International Conference on Medical image computing and computer-assistedintervention. Springer, 2015, pp. 234–241.[8] M. Livne, J. Rieger, O. U. Aydin, A. A. Taha, E. M. Akay, T. Kossen, J. Sobesky, J. D. Kelleher,K. Hildebrand, D. Frey et al., “A u-net deep learning framework for high performance vessel segmentationin patients with cerebrovascular disease,” Frontiers in neuroscience, vol. 13, p. 97, 2019.[9] S. P. Ehsani, H. S. Mousavi, and B. H. Khalaj, “Iterative histogram matching algorithm for chromosomeimage enhancement based on statistical moments,” in 2012 9th IEEE International Symposium onBiomedical Imaging (ISBI). IEEE, 2012, pp. 214–217.[10] Z. Liu, “Retinal vessel segmentation based on fully convolutional networks,” arXiv preprint arXiv:1911.09915,2019.

A Deep Learning Model for Automated Segmentation of Fluorescence Cell images

http://acikerisim.fsm.edu.tr/xmlui/bitstream/11352/4069/1/Ayd%c4%b1n.pdf

A Deep Learning Model for Automated Segmentation of Fluorescence Cell images

Abstract

Similar works

Full text

Available Versions

DSpace@FSM Vakif University