Motivation
						An automated counting of beads is required for many high-throughput experiments such as studying mimicked bacterial invasion processes. However, state-of-the-art algorithms under- or overestimate the number of beads in low-resolution images. In addition, expert knowledge is needed to adjust parameters.
						Results
						In combination with our image labeling tool, BeadNet enables biologists to easily annotate and process their data reducing the expertise required in many existing image analysis pipelines. BeadNet outperforms state-of-the-art-algorithms in terms of missing, added and total amount of beads.
						Availability and implementation
					BeadNet (software, code and dataset) is available at https://bitbucket.org/t_scherr/beadnet. The image labeling tool is available at https://bitbucket.org/abartschat/imagelabelingtool

Bartschat, Andreas

Böhland, Moritz

Mikut, Ralf

Orian-Rousseau, Véronique

Reischl, Markus

Scherr, Tim

Stegmaier, Johannes

Streule, Karolin

English

Publikationsserver der RWTH Aachen University

BeadNet : deep learning-based bead detection and counting in low-resolution microscopy images

KITopen

Bioimage informaticsBeadNet: deep learning-based bead detection andcounting in low-resolution microscopy imagesTim Scherr 1,*, Karolin Streule2, Andreas Bartschat, Moritz Böhland1, JohannesStegmaier3, Markus Reischl1, Véronique Orian-Rousseau2 and Ralf Mikut1,*1Institute for Automation and Applied Informatics and 2Institute of Biological and Chemical Systems – Functional Molecular Systems,Karlsruhe Institute of Technology, Eggenstein-Leopoldshafen 76344, Germany and 3Institute of Imaging and Computer Vision, RWTHAachen University, Aachen 52074, Germany*To whom correspondence should be addressed.Associate Editor: Jinbo XuReceived on January 20, 2020; revised on May 20, 2020; editorial decision on June 16, 2020; accepted on June 18, 2020AbstractMotivation: An automated counting of beads is required for many high-throughput experiments such as studyingmimicked bacterial invasion processes. However, state-of-the-art algorithms under- or overestimate the number ofbeads in low-resolution images. In addition, expert knowledge is needed to adjust parameters.Results: In combination with our image labeling tool, BeadNet enables biologists to easily annotate and processtheir data reducing the expertise required in many existing image analysis pipelines. BeadNet outperforms state-of-the-art-algorithms in terms of missing, added and total amount of beads.Availability and implementation: BeadNet (software, code and dataset) is available at https://bitbucket.org/t_scherr/beadnet. The image labeling tool is available at https://bitbucket.org/abartschat/imagelabelingtool.Contact: tim.scherr@kit.edu or ralf.mikut@kit.eduSupplementary information: Supplementary data are available at Bioinformatics online.1 IntroductionLigand-coupled beads are often used in in vitro experiments tomimic bacterial invasion processes (Braun et al., 1998; Hebert et al.,2017; Jung et al., 2009). In the experiments that led to the develop-ment of BeadNet, red fluorescent latex beads of 1 mm in size werechemically coupled with a bacterial surface ligand, and their intern-alization into human cells was investigated. To distinguish internal-ized beads and beads that remained outside of cells, the cells werefixed without permeabilization, and external beads were recognizedusing a ligand-specific antibody coupled to a green fluorophore(Fig. 1a).The complexity of bead detection in low-resolution images maylead biologists to either use an inefficient and user-biased manualquantification or to use a user-friendly but inaccurate method. Inaddition, the parameters of sophisticated analysis pipelines need tobe adjusted in case of changes in the experimental design. Even forexperts, it is often easier to annotate new images instead of adjustingparameters for low-resolution and low signal-to-noise-ratio data.Thus, we designed the bead detection and counting softwareBeadNet to avoid parameter adjustments. For domain adaptation,biologists only need to create new training images using a built-infunctionality and annotate them using our labeling tool. Despite itssimple use, BeadNet enables a nearly error-free bead detection on amanually annotated low-resolution bead dataset. To our knowledge,this is the first annotated 2D dataset and software tackling the prob-lem of counting beads in low-resolution images with high accuracy.The software combines a crucial upsampling preprocessing with adeep learning-based detection. In addition, BeadNet provides agraphical user interface for easy use, e.g. for generating new trainingdata. The upsampling step and the training data generation are oftenmissing in toolboxes, leaving users alone with the upsampling, thenormalization and the creation of new training samples. BeadNetalso provides the automatic calculation of evaluation measures andoverlays of the detection results.2 Materials and methodsBeadNet consists of an upsampling preprocessing step for low-resolution images and a deep learning-based bead detection withsubsequent counting. The bilinear upsampling, hereby, enables dis-tinguishing touching beads (Fig. 1b and c). Furthermore, it eases theannotation of a low-resolution dataset that is required for supervisedlearning (Fig. 1d). Due to the upsampling preprocessing, centers oftouching beads do not share an edge anymore, and the bead detec-tion can be treated as a semantic segmentation task. Thus, for thebead detection, an adapted U-Net trained with a combination ofVC The Author(s) 2020. Published by Oxford University Press. 4668This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permitsunrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.Bioinformatics, 36(17), 2020, 4668–4670doi: 10.1093/bioinformatics/btaa594Advance Access Publication Date: 26 June 2020Applications NoteDownloaded from https://academic.oup.com/bioinformatics/article/36/17/4668/5863246 by KIT Library user on 03 December 2020binary cross-entropy and Dice loss is used (Milletari et al., 2016;Ronneberger et al., 2015). Thereby, the traditional upsampling uti-lizes prior knowledge enforcing the network to work at reasonablescales. To improve the generalization ability of the trained model,specific training data augmentations are applied online during train-ing. To get a more robust seed detection, a morphological dilationwith a cross-like mask can be applied to the upsampled traininglabel images. This enlarges the seeds and reduces the number ofmissing seed detections. However, depending on the resolution andthe seeds’ distances the number of missing detections may alsoincrease.In the post-processing, the raw predictions are binarized using aglobal threshold of 0.5, and the centroids of predicted beads are cal-culated. Finally, the detected beads are counted. Additional informa-tion concerning the processing steps and the U-Net model used canbe found in the Supplementary Information.To use our BeadNet software, a user needs to select a predefinedbead diameter range required for the upsampling preprocessing. If aretraining for domain adaptation is needed, i.e. the resulting seed de-tection overlays are erroneous, new training data crops can be gener-ated and annotated. Then, multiple models with and without thedilation step are trained and evaluated. For inference, the on the testdataset best performing model is selected automatically.3 Low-resolution bead datasetWe annotated a low-resolution bead dataset consisting of 60 train-ing, 15 validation and 25 test images with 2587 beads in total. Thebead diameters range from about 2 px to 4 px (Fig. 1b).Neighboring seeds can touch each other and the center of beads isnot well-defined. This makes manual annotation of seeds in the ori-ginal images very difficult. The application of a fourfold bilinearupsampling and the use of 2  2 px seeds enable to annotate beadsquickly, to avoid touching seeds and to hit the center easily (Fig. 1cand d).4 ResultsFor performance measurements, the ground truth seeds are enlarged(Fig. 1d). Then, the normalized amount of missing (no predictedseed in a ground truth bead), split (multiple predicted seeds in aground truth bead) and added beads (predicted seed in the groundtruth background) can be counted. All shown metrics do not countadded seeds at the border area (only fully visible beads are anno-tated, see Supplementary Information for the applied bordercorrection).Table 1 shows quantitative results of BeadNet, of the Laplacianof Gaussian-based seed detection of TWANG (Bartschat et al.,2016; Stegmaier et al., 2014), of a Hough-transform-based detectionusing MATLABs imfindcircles (Yuen et al., 1990) and of a simpleOtsu thresholding (Otsu, 1979) with Euclidean distance transformon the (upsampled) test dataset. The median BeadNet predictionoutperforms the other methods in nearly every metric. More detailedresults including the single initializations, and a qualitative compari-son with the FISH-quant software for the automatic counting oftranscripts in FISH images (Mueller et al., 2013) is provided in theSupplementary Information.5 ConclusionBeadNet outperforms traditional bead detection methods, whichneed expert knowledge to adjust them. The high detection rateshows that no additional preprocessing toolbox is needed, e.g. a 2Dadaptation of the image restoration toolbox CSBDeep for thedenoising and isotropic recovery of 3D microscopy data (Weigertet al., 2018). Combined with the integrated training data generationenables the high detection rate a user-friendly end-to-end use ofBeadNet for the detection of poorly resolved objects, e.g. of ligand-coupled beads or other spherical or non-spherical objects.FundingThis work was supported by the Helmholtz Association in the programBioInterfaces in Technology and Medicine (BIFTM), and by a grant from theDeutsche Forschungsgemeinschaft [OR124/16-1 to V.O.-R. and K.S.].Conflict of Interest: none declared.ReferencesBartschat,A. et al. (2016) XPIWIT – an XML pipeline wrapper for the insighttoolkit. Bioinformatics, 32, 315–317.Fig. 1. Exemplary application case and detection results of BeadNet. (a) A maximum intensity projection of red fluorescent beads (additional green fluorophore for beads out-side of cells) and blue fluorescent cell nuclei. The 32  32 px test image in (b) is taken from the red fluorescent channel of (a). The upsampled test image is shown in (c). Thewhite 2  2 px seeds in (d) are enlarged (gray) for the calculation of the evaluation metrics. The predictions of BeadNet are shown in (e) (white in gray ground truth)Table 1. Results on the 25 test images of the bead dataset (670 beads)Method F-Score QF Precision QP Recall QR Split Qsplit (%) Missing Qmiss (%) Added Qadd (%) Detections (%)BeadNet 0.971 0.977 0.954 0.45 4.63 1.79 97.76BeadNet (w. dilation) 0.977 0.979 0.976 0.15 2.39 2.09 99.70TWANG 0.939 0.960 0.919 0 8.06 3.88 95.82Hough transform 0.927 0.977 0.882 0 11.79 2.09 90.30Otsu 0.651 0.766 0.567 2.09 43.28 15.22 74.03Note: For BeadNet, the median of five trained models is shown. The other methods are deterministic.BeadNet 4669Downloaded from https://academic.oup.com/bioinformatics/article/36/17/4668/5863246 by KIT Library user on 03 December 2020Braun,L. et al. (1998) The InlB protein of Listeria monocytogenes is sufficientto promote entry into mammalian cells. Mol. Microbiol., 27, 1077–1087.Hebert,K. et al. (2017) Anaplasma marginale outer membrane protein A is anadhesin that recognizes sialylated and fucosylated glycans and functionallydepends on an essential binding domain. Infect. Immun., 85, 1–16.Jung,C. et al. (2009) Involvement of CD44v6 in InlB-dependent Listeria inva-sion. Mol. Microbiol., 72, 1196–1207.Milletari,F. et al. (2016) V-Net: fully convolutional neural networks for volu-metric medical image segmentation. In: 2016 Fourth InternationalConference on 3D Vision, IEEE, Stanford, CA, USA. pp. 565–571.Mueller,F. et al. (2013) FISH-quant: automatic counting of transcripts in 3DFISH images. Nat. Methods, 10, 277–278.Otsu,N. (1979) A threshold selection method from gray-level histograms.IEEE Trans. Syst. Man Cybern., 9, 62–66.Ronneberger,O. et al. (2015) U-Net: convolutional networks for biomedicalimage segmentation. In: Navab,N. et al. (eds) Medical Image Computingand Computer-Assisted Intervention – MICCAI 2015. Springer InternationalPublishing, Cham, pp. 234–241.Stegmaier,J. et al. (2014) Fast segmentation of stained nuclei in terabyte-scale,time resolved 3D microscopy image stacks. PLoS One, 9, e90036.Weigert,M. et al. (2018) Content-aware image restoration: pushing the limitsof fluorescence microscopy. Nat. Methods, 15, 1090–1097.Yuen,H.K. et al. (1990) Comparative study of Hough transform methods forcircle finding. Image Vis. Comput., 8, 71–77.4670 T.Scherr et al.Downloaded from https://academic.oup.com/bioinformatics/article/36/17/4668/5863246 by KIT Library user on 03 December 2020

BeadNet: Deep learning-based bead detection and counting in low-resolution microscopy images

https://publikationen.bibliothek.kit.edu/1000120804/94741334

BeadNet: Deep learning-based bead detection and counting in low-resolution microscopy images

Abstract

Similar works

Full text

Available Versions

Publikationsserver der RWTH Aachen University

KITopen