127 research outputs found
OpSeF : Open Source Python Framework for Collaborative Instance Segmentation of Bioimages
Various pre-trained deep learning models for the segmentation of bioimages have been made available as developer-to-end-user solutions. They are optimized for ease of use and usually require neither knowledge of machine learning nor coding skills. However, individually testing these tools is tedious and success is uncertain. Here, we present the Open Segmentation Framework (OpSeF), a Python framework for deep learning-based instance segmentation. OpSeF aims at facilitating the collaboration of biomedical users with experienced image analysts. It builds on the analysts' knowledge in Python, machine learning, and workflow design to solve complex analysis tasks at any scale in a reproducible, well-documented way. OpSeF defines standard inputs and outputs, thereby facilitating modular workflow design and interoperability with other software. Users play an important role in problem definition, quality control, and manual refinement of results. OpSeF semi-automates preprocessing, convolutional neural network (CNN)-based segmentation in 2D or 3D, and postprocessing. It facilitates benchmarking of multiple models in parallel. OpSeF streamlines the optimization of parameters for pre- and postprocessing such, that an available model may frequently be used without retraining. Even if sufficiently good results are not achievable with this approach, intermediate results can inform the analysts in the selection of the most promising CNN-architecture in which the biomedical user might invest the effort of manually labeling training data. We provide Jupyter notebooks that document sample workflows based on various image collections. Analysts may find these notebooks useful to illustrate common segmentation challenges, as they prepare the advanced user for gradually taking over some of their tasks and completing their projects independently. The notebooks may also be used to explore the analysis options available within OpSeF in an interactive way and to document and share final workflows. Currently, three mechanistically distinct CNN-based segmentation methods, the U-Net implementation used in Cellprofiler 3.0, StarDist, and Cellpose have been integrated within OpSeF. The addition of new networks requires little; the addition of new models requires no coding skills. Thus, OpSeF might soon become both an interactive model repository, in which pre-trained models might be shared, evaluated, and reused with ease.Peer reviewe
AnyStar: Domain randomized universal star-convex 3D instance segmentation
Star-convex shapes arise across bio-microscopy and radiology in the form of
nuclei, nodules, metastases, and other units. Existing instance segmentation
networks for such structures train on densely labeled instances for each
dataset, which requires substantial and often impractical manual annotation
effort. Further, significant reengineering or finetuning is needed when
presented with new datasets and imaging modalities due to changes in contrast,
shape, orientation, resolution, and density. We present AnyStar, a
domain-randomized generative model that simulates synthetic training data of
blob-like objects with randomized appearance, environments, and imaging physics
to train general-purpose star-convex instance segmentation networks. As a
result, networks trained using our generative model do not require annotated
images from unseen datasets. A single network trained on our synthesized data
accurately 3D segments C. elegans and P. dumerilii nuclei in fluorescence
microscopy, mouse cortical nuclei in micro-CT, zebrafish brain nuclei in EM,
and placental cotyledons in human fetal MRI, all without any retraining,
finetuning, transfer learning, or domain adaptation. Code is available at
https://github.com/neel-dey/AnyStar.Comment: Code available at https://github.com/neel-dey/AnySta
YOLO2U-Net: Detection-Guided 3D Instance Segmentation for Microscopy
Microscopy imaging techniques are instrumental for characterization and
analysis of biological structures. As these techniques typically render 3D
visualization of cells by stacking 2D projections, issues such as out-of-plane
excitation and low resolution in the -axis may pose challenges (even for
human experts) to detect individual cells in 3D volumes as these
non-overlapping cells may appear as overlapping. In this work, we introduce a
comprehensive method for accurate 3D instance segmentation of cells in the
brain tissue. The proposed method combines the 2D YOLO detection method with a
multi-view fusion algorithm to construct a 3D localization of the cells. Next,
the 3D bounding boxes along with the data volume are input to a 3D U-Net
network that is designed to segment the primary cell in each 3D bounding box,
and in turn, to carry out instance segmentation of cells in the entire volume.
The promising performance of the proposed method is shown in comparison with
some current deep learning-based 3D instance segmentation methods
Non-invasive scoring of cellular atypia in keratinocyte cancers in 3D LC-OCT images using Deep Learning
Diagnosis based on histopathology for skin cancer detection is today's gold standard and relies on the presence or absence of biomarkers and cellular atypia. However it suffers drawbacks: it requires a strong expertise and is time-consuming. Moreover the notion of atypia or dysplasia of the visible cells used for diagnosis is very subjective, with poor inter-rater agreement reported in the literature. Lastly, histology requires a biopsy which is an invasive procedure and only captures a small sample of the lesion, which is insufficient in the context of large fields of cancerization. Here we demonstrate that the notion of cellular atypia can be objectively defined and quantified with a non-invasive in-vivo approach in three dimensions (3D). A Deep Learning (DL) algorithm is trained to segment keratinocyte (KC) nuclei from Line-field Confocal Optical Coherence Tomography (LC-OCT) 3D images. Based on these segmentations, a series of quantitative, reproducible and biologically relevant metrics is derived to describe KC nuclei individually. We show that, using those metrics, simple and more complex definitions of atypia can be derived to discriminate between healthy and pathological skins, achieving Area Under the ROC Curve (AUC) scores superior than 0.965, largely outperforming medical experts on the same task with an AUC of 0.766. All together, our approach and findings open the door to a precise quantitative monitoring of skin lesions and treatments, offering a promising non-invasive tool for clinical studies to demonstrate the effects of a treatment and for clinicians to assess the severity of a lesion and follow the evolution of pre-cancerous lesions over time.© 2022. The Author(s)
Uncertainty Estimation in Instance Segmentation with Star-convex Shapes
Instance segmentation has witnessed promising advancements through deep
neural network-based algorithms. However, these models often exhibit incorrect
predictions with unwarranted confidence levels. Consequently, evaluating
prediction uncertainty becomes critical for informed decision-making. Existing
methods primarily focus on quantifying uncertainty in classification or
regression tasks, lacking emphasis on instance segmentation. Our research
addresses the challenge of estimating spatial certainty associated with the
location of instances with star-convex shapes. Two distinct clustering
approaches are evaluated which compute spatial and fractional certainty per
instance employing samples by the Monte-Carlo Dropout or Deep Ensemble
technique. Our study demonstrates that combining spatial and fractional
certainty scores yields improved calibrated estimation over individual
certainty scores. Notably, our experimental results show that the Deep Ensemble
technique alongside our novel radial clustering approach proves to be an
effective strategy. Our findings emphasize the significance of evaluating the
calibration of estimated certainties for model reliability and decision-making
Nucleus segmentation : towards automated solutions
Single nucleus segmentation is a frequent challenge of microscopy image processing, since it is the first step of many quantitative data analysis pipelines. The quality of tracking single cells, extracting features or classifying cellular phenotypes strongly depends on segmentation accuracy. Worldwide competitions have been held, aiming to improve segmentation, and recent years have definitely brought significant improvements: large annotated datasets are now freely available, several 2D segmentation strategies have been extended to 3D, and deep learning approaches have increased accuracy. However, even today, no generally accepted solution and benchmarking platform exist. We review the most recent single-cell segmentation tools, and provide an interactive method browser to select the most appropriate solution.Peer reviewe
Automated cell tracking using StarDist and TrackMate [version 1; peer review: awaiting peer review]
The ability of cells to migrate is a fundamental physiological process involved in embryonic development, tissue homeostasis, immune surveillance, and wound healing. Therefore, the mechanisms governing cellular locomotion have been under intense scrutiny over the last 50 years. One of the main tools of this scrutiny is live-cell quantitative imaging, where researchers image cells over time to study their migration and quantitatively analyze their dynamics by tracking them using the recorded images. Despite the availability of computational tools, manual tracking remains widely used among researchers due to the difficulty setting up robust automated cell tracking and large-scale analysis. Here we provide a detailed analysis pipeline illustrating how the deep learning network StarDist can be combined with the popular tracking software TrackMate to perform 2D automated cell tracking and provide fully quantitative readouts. Our proposed protocol is compatible with both fluorescent and widefield images. It only requires freely available and open-source software (ZeroCostDL4Mic and Fiji), and does not require any coding knowledge from the users, making it a versatile and powerful tool for the field. We demonstrate this pipeline's usability by automatically tracking cancer cells and T cells using fluorescent and brightfield images. Importantly, we provide, as supplementary information, a detailed step-by-step protocol to allow researchers to implement it with their images
Opportunities and challenges for deep learning in cell dynamics research
With the growth of artificial intelligence (AI), there has been an increase
in the adoption of computer vision and deep learning (DL) techniques for the
evaluation of microscopy images and movies. This adoption has not only
addressed hurdles in quantitative analysis of dynamic cell biological
processes, but it has also started supporting advances in drug development,
precision medicine and genome-phenome mapping. Here we survey existing AI-based
techniques and tools, and open-source datasets, with a specific focus on the
computational tasks of segmentation, classification, and tracking of cellular
and subcellular structures and dynamics. We summarise long-standing challenges
in microscopy video analysis from the computational perspective and review
emerging research frontiers and innovative applications for deep
learning-guided automation for cell dynamics research
- …