Evaluation of Deep Learning Strategies for Nucleus Segmentation in Fluorescence Images

Identifying nuclei is often a critical first step in analyzing microscopy images of cells and classical image processing algorithms are most commonly used for this task. Recent developments in deep learning can yield superior accuracy, but typical evaluation metrics for nucleus segmentation do not satisfactorily capture error modes that are relevant in cellular images. We present an evaluation framework to measure accuracy, types of errors, and computational efficiency; and use it to compare deep learning strategies and classical approaches. We publicly release a set of 23,165 manually annotated nuclei and source code to reproduce experiments and run the proposed evaluation methodology. Our evaluation framework shows that deep learning improves accuracy and can reduce the number of biologically relevant errors by half. (c) 2019 The Authors. Cytometry Part A published by Wiley Periodicals, Inc. on behalf of International Society for Advancement of Cytometry

Objective assessment of stored blood quality by deep learning

Stored red blood cells (RBCs) are needed for life-saving blood transfusions, but they undergo continuous degradation. RBC storage lesions are often assessed by microscopic examination or biochemical and biophysical assays, which are complex, time-consuming, and destructive to fragile cells. Here we demonstrate the use of label-free imaging flow cytometry and deep learning to characterize RBC lesions. Using brightfield images, a trained neural network achieved 76.7% agreement with experts in classifying seven clinically relevant RBC morphologies associated with storage lesions, comparable to 82.5% agreement between different experts. Given that human observation and classification may not optimally discern RBC quality, we went further and eliminated subjective human annotation in the training step by training a weakly supervised neural network using only storage duration times. The feature space extracted by this network revealed a chronological progression of morphological changes that better predicted blood quality, as measured by physiological hemolytic assay readouts, than the conventional expert-assessed morphology classification system. With further training and clinical testing across multiple sites, protocols, and instruments, deep learning and label-free imaging flow cytometry might be used to routinely and objectively assess RBC storage lesions. This would automate a complex protocol, minimize laboratory sample handling and preparation, and reduce the impact of procedural errors and discrepancies between facilities and blood donors. The chronology-based machine-learning approach may also improve upon humans’ assessment of morphological changes in other biomedically important progressions, such as differentiation and metastasis

Learning representations for image-based profiling of perturbations

Abstract Measuring the phenotypic effect of treatments on cells through imaging assays is an efficient and powerful way of studying cell biology, and requires computational methods for transforming images into quantitative data. Here, we present an improved strategy for learning representations of treatment effects from high-throughput imaging, following a causal interpretation. We use weakly supervised learning for modeling associations between images and treatments, and show that it encodes both confounding factors and phenotypic features in the learned representation. To facilitate their separation, we constructed a large training dataset with images from five different studies to maximize experimental diversity, following insights from our causal analysis. Training a model with this dataset successfully improves downstream performance, and produces a reusable convolutional network for image-based profiling, which we call Cell Painting CNN. We evaluated our strategy on three publicly available Cell Painting datasets, and observed that the Cell Painting CNN improves performance in downstream analysis up to 30% with respect to classical features, while also being more computationally efficient

CellProfiler 3.0: Next-generation image processing for biology

<div><p>CellProfiler has enabled the scientific research community to create flexible, modular image analysis pipelines since its release in 2005. Here, we describe CellProfiler 3.0, a new version of the software supporting both whole-volume and plane-wise analysis of three-dimensional (3D) image stacks, increasingly common in biomedical research. CellProfiler’s infrastructure is greatly improved, and we provide a protocol for cloud-based, large-scale image processing. New plugins enable running pretrained deep learning models on images. Designed by and for biologists, CellProfiler equips researchers with powerful computational tools via a well-documented user interface, empowering biologists in all fields to create quantitative, reproducible image analysis workflows.</p></div

Segmentation steps for the quantification of transcripts per cell within a 3D blastocyst.

<p>Images were captured of a mouse embryo blastocyst cell membrane stained with WGA and FISH for GAPDH transcripts. (A) Original 3D image of blastocyst cell membrane prior to analysis. (B) CellProfiler 3.0 image processing modules used for membrane image processing. Figure labels: RH (“RemoveHoles”), Close (“Closing”), Erode (“Erosion”), Mask (“MaskImage”), Math (“ImageMath”), EorS Features (“EnhanceOrSuppressFeatures”). (C) Nuclei after segmentation by CellProfiler, as viewed in Fiji. (D) Segmentation of cells after setting nuclei as seeds by CellProfiler, as viewed in Fiji. (E) Segmentation of GAPDH transcript foci using CellProfiler, as viewed in Fiji. (F) Examples of analysis that can be done by CellProfiler: (top) cell volume relative nucleus volume, (middle) GAPDH transcript quantity in each cell using CellProfiler’s “RelateObjects” module, (bottom) number of GAPDH transcripts in Z-plane (bin size = 2.5 μm). The underlying measurements may be downloaded as <a href="http://www.plosbiology.org/article/info:doi/10.1371/journal.pbio.2005970#pbio.2005970.s012" target="_blank">S1 File</a>. <i>Images were provided by Javier Frias Aldeguer and Nicolas Rivron from Hubrecht Institute</i>, <i>Netherlands</i>, <i>and are available from the Broad Bioimage Benchmark Collection (<a href="https://data.broadinstitute.org/bbbc/BBBC032/" target="_blank">https://data.broadinstitute.org/bbbc/BBBC032/</a></i>). 3D, three-dimensional; FISH, fluorescent in situ hybridization; GAPDH, glyceraldehyde 3-phosphate dehydrogenase; WGA, wheat germ agglutinin.</p

Segmentation and analysis of 3D hiPSC images using CellProfiler 3.0.

<p>DNA channel showing nuclei (A), CellMaskDeepRed channel showing membrane (B), and GFP channel showing beta-actin (C) at the center (left) and edge (right) of the hiPSC colony. (D) Various measurements obtained from the samples are shown; note that cells touching the edge of each image are excluded from this analysis. The underlying measurements may be downloaded as <a href="http://www.plosbiology.org/article/info:doi/10.1371/journal.pbio.2005970#pbio.2005970.s013" target="_blank">S2 File</a>. <i>Images are from the Allen Institute for Cell Science</i>, <i>Seattle</i>, <i>and are available from the Broad Bioimage Benchmark Collection (<a href="https://data.broadinstitute.org/bbbc/BBBC034/" target="_blank">https://data.broadinstitute.org/bbbc/BBBC034/</a>)</i>. 3D, three-dimensional; GFP, green fluorescent protein; hiPSC, human induced pluripotent stem cell.</p

Examples of 3D image segmentation produced by CellProfiler 3.0, across two experimental systems and two sets of synthesized images.

<p>Three focal planes shown for each. Raw images (left) and CellProfiler outputs (right) showing nuclei of mouse embryo blastocyst (A), mouse trophoblast stem cells (B), and synthetic images of HL60 cell lines (C) and (D). More information about segmentation steps used for these images can be found in <a href="http://www.plosbiology.org/article/info:doi/10.1371/journal.pbio.2005970#pbio.2005970.s002" target="_blank">S2</a>–<a href="http://www.plosbiology.org/article/info:doi/10.1371/journal.pbio.2005970#pbio.2005970.s005" target="_blank">S5</a> Figs. (E) Comparison of the segmentation accuracy of CellProfiler 3.0 and Fiji’s plugin MorphoLibJ, based on the Rand index of the processed image and its ground truth (out of a total of 1.0). Object accuracy comparisons of these same images may be found in <a href="http://www.plosbiology.org/article/info:doi/10.1371/journal.pbio.2005970#pbio.2005970.s017" target="_blank">S6 Fig</a> and <a href="http://www.plosbiology.org/article/info:doi/10.1371/journal.pbio.2005970#pbio.2005970.s014" target="_blank">S3</a> File. 3D, three-dimensional; hiPSC, human induced pluripotent stem cell.</p