Particle detection and tracking in fluorescence time-lapse imaging: a contrario approach

This paper proposes a probabilistic approach for the detection and the tracking of particles in fluorescent time-lapse imaging. In the presence of a very noised and poor-quality data, particles and trajectories can be characterized by an a contrario model, that estimates the probability of observing the structures of interest in random data. This approach, first introduced in the modeling of human visual perception and then successfully applied in many image processing tasks, leads to algorithms that neither require a previous learning stage, nor a tedious parameter tuning and are very robust to noise. Comparative evaluations against a well-established baseline show that the proposed approach outperforms the state of the art.Comment: Published in Journal of Machine Vision and Application

Automatic Hotspots Detection for Intracellular Calcium Analysis in Fluorescence Microscopic Videos

In recent years, life-cell imaging techniques and their software applications have become powerful tools to investigate complex biological mechanisms such as calcium signalling. In this paper, we propose an automated framework to detect areas inside cells that show changes in their calcium concentration i.e. the regions of interests or hotspots, based on videos taken after loading living mouse cardiomyocytes with fluorescent calcium reporter dyes. The proposed system allows an objective and efficient analysis through the following four key stages: (1) Pre-processing to enhance video quality, (2) First level segmentation to detect candidate hotspots based on adaptive thresholding on the frame level, (3) Second-level segmentation to fuse and identify the best hotspots from the entire video by proposing the concept of calcium fluorescence hit-ratio, and (4) Extraction of the changes of calcium fluorescence over time per hotspot. From the extracted signals, different measurements are calculated such as maximum peak amplitude, area under the curve, peak frequency, and inter-spike interval of calcium changes. The system was tested using calcium imaging data collected from Heart muscle cells. The paper argues that the automated proposal offers biologists a tool to speed up the processing time and mitigate the consequences of inter-intra observer variability

SPOT DETECTION METHODS IN FLUORESCENCE MICROSCOPY IMAGING: A REVIEW

Fluorescence microscopy imaging has become one of the essential tools used by biologists to visualize and study intracellular particles within a cell. Studying these particles is a long-term research effort in the field of microscopy image analysis, consisting of discovering the relationship between the dynamics of particles and their functions. However, biologists are faced with challenges such as the counting and tracking of these intracellular particles. To overcome the issues faced by biologists, tools which can extract the location and motion of these particles are essential. One of the most important steps in these analyses is to accurately detect particle positions in an image, termed spot detection. The detection of spots in microscopy imaging is seen as a critical step for further quantitative analysis. However, the evaluation of these microscopic images is mainly conducted manually, with automated methods becoming popular. This work presents some advances in fluorescence microscopy image analysis, focusing on the detection methods needed for quantifying the location of these spots. We review several existing detection methods in microscopy imaging, along with existing synthetic benchmark datasets and evaluation metrics

The origin of heterogeneous nanoparticle uptake by cells

Understanding nanoparticle uptake by biological cells is fundamentally important to wide-ranging fields from nanotoxicology to drug delivery. It is now accepted that the arrival of nanoparticles at the cell is an extremely complicated process, shaped by many factors including unique nanoparticle physico-chemical characteristics, protein-particle interactions and subsequent agglomeration, diffusion and sedimentation. Sequentially, the nanoparticle internalisation process itself is also complex, and controlled by multiple aspects of a cell’s state. Despite this multitude of factors, here we demonstrate that the statistical distribution of the nanoparticle dose per endosome is independent of the initial administered dose and exposure duration. Rather, it is the number of nanoparticle containing endosomes that are dependent on these initial dosing conditions. These observations explain the heterogeneity of nanoparticle delivery at the cellular level and allow the derivation of simple, yet powerful probabilistic distributions that accurately predict the nanoparticle dose delivered to individual cells across a population.J.W.W. would like to acknowledge Girton College and the Herchel Smith Fund of Cambridge for providing him with a post-doctoral Fellowship. The authors are grateful to J.J. Powell and S. H. Doak for their critical insights. This work was supported by the Engineering and Physical Sciences Research Council (EPSRC) (grant number EP/H008683/1). P.R. and H.D.S. would also like to acknowledge the support of the Biotechnology and Biological Sciences Research Council (BBSRC) under grants BB/N005163/1 and BB/P026818/1

Biomolecule Trafficking and Network Tomography-based Simulations

International audienceDuring the past two decades many groundbreaking technologies, including Green Fluorescent Protein (GFP)-tagging and super-resolution microscopy, emerged and allowed the visualization of protein dynamics and molecular interactions at different levels of spatial and temporal resolution. In the meantime, the automated quantification of microscopy images depicting moving biomolecules has become of major importance in cell biology since it offers a better understanding of fundamental mechanisms including membrane transport, cell signaling, cell division and motility. Consequently, dedicated image analysis methods have been developed to process challenging temporal series of 2D-3D images and to estimate individual trajectories of biomolecules. Nevertheless, the current tracking methods cannot provide global information about biomolecule trafficking. This motivated the development of simulation techniques able to generate realistic fluorescence microscopy image sequences depicting trafficking of small moving particles in interaction, with variable velocities within the cell. In this chapter, we describe a simulation approach based on the concept of Network Tomography (NT) which is generally used in network communications and transport to infer the main routes of communication between origins and destinations. The trafficking model, scaled down for microscopy, is combined with real 2D-3D image sequences to generate artificial videos depicting fluorescently tagged moving proteins within cells. Simulation in bioimaging is timely since it has become essential to build ground truth datasets for image processing algorithm evaluation such as biomolecule detectors and trackers, as well as to generate training datasets for deep learning algorithms

Lifetime estimation on moving sub-cellular objects in frequency domain FLIM imaging

International audienceFluorescence lifetime is usually defined as the average nanosecond-scale delay between excitation and emission of fluorescence. It has been established that lifetime measurement yields numerous indications on cellular processes such as inter-protein and intra-protein mechanisms through fluorescent tagging and Förster resonance energy transfer (FRET). In this area, frequency domain fluorescence lifetime imaging microscopy (FD FLIM) is particularly well appropriate to probe a sample non-invasively and quantify these interactions in living cells. The aim is then to measure fluorescence lifetime in the sample at each location in space from fluorescence variations observed in a temporal sequence of images obtained by phase modulation of the detection signal. This leads to a sensitivity of lifetime determination to other sources of fluorescence variations such as intracellular motion. In this paper, we propose a robust statistical method for lifetime estimation on both background and small moving structures with a focus on intracellular vesicle trafficking

Piecewise-stationary motion modeling and iterative smoothing to track heterogeneous particle motions in dense environments

International audienceOne of the major challenges in multiple particle tracking is the capture of extremely heterogeneous movements of objects in crowded scenes. The presence of numerous assignment candidates in the expected range of particle motion makes the tracking ambiguous and induces false positives. Lowering the ambiguity by reducing the search range, on the other hand, is not an option, as this would increase the rate of false negatives. We propose here a piecewise-stationary motion model (PMM) for the particle transport along an iterative smoother that exploits recursive tracking in multiple rounds in forward and backward temporal directions. By fusing past and future information, our method, termed PMMS, can recover fast transitions from freely or confined diffusive to directed motions with linear time complexity. To avoid false positives we complemented recursive tracking with a robust inline estimator of the search radius for assignment (a.k.a. gating), where past and future information are exploited using only two frames at each optimization step. We demonstrate the improvement of our technique on simulated data – especially the impact of density, variation in frame to frame displacements, and motion switching probability. We evaluated our technique on the 2D particle tracking challenge dataset published by Chenouard et al in 2014. Using high SNR to focus on motion modeling challenges, we show superior performance at high particle density. On biological applications, our algorithm allows us to quantify the extremely small percentage of motor-driven movements of fluorescent particles along microtubules in a dense field of unbound, diffusing particles. We also show with virus imaging that our algorithm can cope with a strong reduction in recording frame rate while keeping the same performance relative to methods relying on fast sampling

Deep Learning for Detection and Segmentation in High-Content Microscopy Images

High-content microscopy led to many advances in biology and medicine. This fast emerging technology is transforming cell biology into a big data driven science. Computer vision methods are used to automate the analysis of microscopy image data. In recent years, deep learning became popular and had major success in computer vision. Most of the available methods are developed to process natural images. Compared to natural images, microscopy images pose domain specific challenges such as small training datasets, clustered objects, and class imbalance. In this thesis, new deep learning methods for object detection and cell segmentation in microscopy images are introduced. For particle detection in fluorescence microscopy images, a deep learning method based on a domain-adapted Deconvolution Network is presented. In addition, a method for mitotic cell detection in heterogeneous histopathology images is proposed, which combines a deep residual network with Hough voting. The method is used for grading of whole-slide histology images of breast carcinoma. Moreover, a method for both particle detection and cell detection based on object centroids is introduced, which is trainable end-to-end. It comprises a novel Centroid Proposal Network, a layer for ensembling detection hypotheses over image scales and anchors, an anchor regularization scheme which favours prior anchors over regressed locations, and an improved algorithm for Non-Maximum Suppression. Furthermore, a novel loss function based on Normalized Mutual Information is proposed which can cope with strong class imbalance and is derived within a Bayesian framework. For cell segmentation, a deep neural network with increased receptive field to capture rich semantic information is introduced. Moreover, a deep neural network which combines both paradigms of multi-scale feature aggregation of Convolutional Neural Networks and iterative refinement of Recurrent Neural Networks is proposed. To increase the robustness of the training and improve segmentation, a novel focal loss function is presented. In addition, a framework for black-box hyperparameter optimization for biomedical image analysis pipelines is proposed. The framework has a modular architecture that separates hyperparameter sampling and hyperparameter optimization. A visualization of the loss function based on infimum projections is suggested to obtain further insights into the optimization problem. Also, a transfer learning approach is presented, which uses only one color channel for pre-training and performs fine-tuning on more color channels. Furthermore, an approach for unsupervised domain adaptation for histopathological slides is presented. Finally, Galaxy Image Analysis is presented, a platform for web-based microscopy image analysis. Galaxy Image Analysis workflows for cell segmentation in cell cultures, particle detection in mice brain tissue, and MALDI/H&E image registration have been developed. The proposed methods were applied to challenging synthetic as well as real microscopy image data from various microscopy modalities. It turned out that the proposed methods yield state-of-the-art or improved results. The methods were benchmarked in international image analysis challenges and used in various cooperation projects with biomedical researchers

Deep Learning Methods for Detection and Tracking of Particles in Fluorescence Microscopy Images

Studying the dynamics of sub-cellular structures such as receptors, filaments, and vesicles is a prerequisite for investigating cellular processes at the molecular level. In addition, it is important to characterize the dynamic behavior of virus structures to gain a better understanding of infection mechanisms and to develop novel drugs. To investigate the dynamics of fluorescently labeled sub-cellular and viral structures, time-lapse fluorescence microscopy is the most often used imaging technique. Due to the limited spatial resolution of microscopes caused by diffraction, these very small structures appear as bright, blurred spots, denoted as particles, in microscopy images. To draw statistically meaningful biological conclusions, a large number of such particles need to be analyzed. However, since manual analysis of fluorescent particles is very time consuming, fully automated computer-based methods are indispensable. We introduce novel deep learning methods for detection and tracking of multiple particles in fluorescence microscopy images. We propose a particle detection method based on a convolutional neural network which performs image-to-image mapping by density map regression and uses the adaptive wing loss. For particle tracking, we present a recurrent neural network that exploits past and future information in both forward and backward direction. Assignment probabilities across multiple detections as well as the probabilities for missing detections are computed jointly. To resolve tracking ambiguities using future information, several track hypotheses are propagated to later time points. In addition, we developed a novel probabilistic deep learning method for particle tracking, which is based on a recurrent neural network mimicking classical Bayesian filtering. The method includes both aleatoric and epistemic uncertainty, and provides valuable information about the reliability of the computed trajectories. Short and long-term temporal dependencies of individual object dynamics are exploited for state prediction, and assigned detections are used to update the predicted states. Moreover, we developed a convolutional Long Short-Term Memory neural network for combined particle tracking and colocalization analysis in two-channel microscopy image sequences. The network determines colocalization probabilities, and colocalization information is exploited to improve tracking. Short and long-term temporal dependencies of object motion as well as image intensities are taken into account to compute assignment probabilities jointly across multiple detections. We also introduce a deep learning method for probabilistic particle detection and tracking. For particle detection, temporal information is integrated to regress a density map and determine sub-pixel particle positions. For tracking, a fully Bayesian neural network is presented that mimics classical Bayesian filtering and takes into account both aleatoric and epistemic uncertainty. Uncertainty information of individual particle detections is considered. Network training for the developed deep learning-based particle tracking methods relies only on synthetic data, avoiding the need of time-consuming manual annotation. We performed an extensive evaluation of our methods based on image data of the Particle Tracking Challenge as well as on fluorescence microscopy images displaying virus proteins of HCV and HIV, chromatin structures, and cell-surface receptors. It turned out that the methods outperform previous methods

ACME: Automatic feature extraction for cell migration examination through intravital microscopy imaging.

Cell detection and tracking applied to in vivo fluorescence microscopy has become an essential tool in biomedicine to characterize 4D (3D space plus time) biological processes at the cellular level. Traditional approaches to cell motion analysis by microscopy imaging, although based on automatic frameworks, still require manual supervision at some points of the system. Hence, when dealing with a large amount of data, the analysis becomes incredibly time-consuming and typically yields poor biological information. In this paper, we propose a fully-automated system for segmentation, tracking and feature extraction of migrating cells within blood vessels in 4D microscopy imaging. Our system consists of a robust 3D convolutional neural network (CNN) for joint blood vessel and cell segmentation, a 3D tracking module with collision handling, and a novel method for feature extraction, which takes into account the particular geometry in the cell-vessel arrangement. Experiments on a large 4D intravital microscopy dataset show that the proposed system achieves a significantly better performance than the state-of-the-art tools for cell segmentation and tracking. Furthermore, we have designed an analytical method of cell behaviors based on the automatically extracted features, which supports the hypotheses related to leukocyte migration posed by expert biologists. This is the first time that such a comprehensive automatic analysis of immune cell migration has been performed, where the total population under study reaches hundreds of neutrophils and thousands of time instances.This work has been partially supported by the National Grant TEC2017-84395-P of the Spanish Ministry of Economy and Competitiveness, Madrid Regional Government and Universidad Carlos III de Madrid through the project SHARON-CM-UC3M, RTI2018- 095497-B-I00 from Ministerio de Ciencia e Innovación (MICINN) and HR17_00527 from Fundación La Caixa to A.H. M.M-M. is supported by the Spanish Ministry of Education, Culture and Sports FPU Grant FPU18/02825. M.P-S. is supported by a Federation of European Biochemical Societies long-term fellowship. J.S. is supported by a fellowship (PRE2019-089130) from MICINN.S