Particle detection and tracking in fluorescence time-lapse imaging: a contrario approach

This paper proposes a probabilistic approach for the detection and the tracking of particles in fluorescent time-lapse imaging. In the presence of a very noised and poor-quality data, particles and trajectories can be characterized by an a contrario model, that estimates the probability of observing the structures of interest in random data. This approach, first introduced in the modeling of human visual perception and then successfully applied in many image processing tasks, leads to algorithms that neither require a previous learning stage, nor a tedious parameter tuning and are very robust to noise. Comparative evaluations against a well-established baseline show that the proposed approach outperforms the state of the art.Comment: Published in Journal of Machine Vision and Application

A beta-herpesvirus with fluorescent capsids to study transport in living cells.

Fluorescent tagging of viral particles by genetic means enables the study of virus dynamics in living cells. However, the study of beta-herpesvirus entry and morphogenesis by this method is currently limited. This is due to the lack of replication competent, capsid-tagged fluorescent viruses. Here, we report on viable recombinant MCMVs carrying ectopic insertions of the small capsid protein (SCP) fused to fluorescent proteins (FPs). The FPs were inserted into an internal position which allowed the production of viable, fluorescently labeled cytomegaloviruses, which replicated with wild type kinetics in cell culture. Fluorescent particles were readily detectable by several methods. Moreover, in a spread assay, labeled capsids accumulated around the nucleus of the newly infected cells without any detectable viral gene expression suggesting normal entry and particle trafficking. These recombinants were used to record particle dynamics by live-cell microscopy during MCMV egress with high spatial as well as temporal resolution. From the resulting tracks we obtained not only mean track velocities but also their mean square displacements and diffusion coefficients. With this key information, we were able to describe particle behavior at high detail and discriminate between particle tracks exhibiting directed movement and tracks in which particles exhibited free or anomalous diffusion

Deep Learning Methods for Detection and Tracking of Particles in Fluorescence Microscopy Images

Studying the dynamics of sub-cellular structures such as receptors, filaments, and vesicles is a prerequisite for investigating cellular processes at the molecular level. In addition, it is important to characterize the dynamic behavior of virus structures to gain a better understanding of infection mechanisms and to develop novel drugs. To investigate the dynamics of fluorescently labeled sub-cellular and viral structures, time-lapse fluorescence microscopy is the most often used imaging technique. Due to the limited spatial resolution of microscopes caused by diffraction, these very small structures appear as bright, blurred spots, denoted as particles, in microscopy images. To draw statistically meaningful biological conclusions, a large number of such particles need to be analyzed. However, since manual analysis of fluorescent particles is very time consuming, fully automated computer-based methods are indispensable. We introduce novel deep learning methods for detection and tracking of multiple particles in fluorescence microscopy images. We propose a particle detection method based on a convolutional neural network which performs image-to-image mapping by density map regression and uses the adaptive wing loss. For particle tracking, we present a recurrent neural network that exploits past and future information in both forward and backward direction. Assignment probabilities across multiple detections as well as the probabilities for missing detections are computed jointly. To resolve tracking ambiguities using future information, several track hypotheses are propagated to later time points. In addition, we developed a novel probabilistic deep learning method for particle tracking, which is based on a recurrent neural network mimicking classical Bayesian filtering. The method includes both aleatoric and epistemic uncertainty, and provides valuable information about the reliability of the computed trajectories. Short and long-term temporal dependencies of individual object dynamics are exploited for state prediction, and assigned detections are used to update the predicted states. Moreover, we developed a convolutional Long Short-Term Memory neural network for combined particle tracking and colocalization analysis in two-channel microscopy image sequences. The network determines colocalization probabilities, and colocalization information is exploited to improve tracking. Short and long-term temporal dependencies of object motion as well as image intensities are taken into account to compute assignment probabilities jointly across multiple detections. We also introduce a deep learning method for probabilistic particle detection and tracking. For particle detection, temporal information is integrated to regress a density map and determine sub-pixel particle positions. For tracking, a fully Bayesian neural network is presented that mimics classical Bayesian filtering and takes into account both aleatoric and epistemic uncertainty. Uncertainty information of individual particle detections is considered. Network training for the developed deep learning-based particle tracking methods relies only on synthetic data, avoiding the need of time-consuming manual annotation. We performed an extensive evaluation of our methods based on image data of the Particle Tracking Challenge as well as on fluorescence microscopy images displaying virus proteins of HCV and HIV, chromatin structures, and cell-surface receptors. It turned out that the methods outperform previous methods

Protein Tracking by CNN-Based Candidate Pruning and Two-Step Linking with Bayesian Network

Protein trafficking plays a vital role in understanding many biological processes and disease. Automated tracking of protein vesicles is challenging due to their erratic behaviour, changing appearance, and visual clutter. In this paper we present a novel tracking approach which utilizes a two-step linking process that exploits a probabilistic graphical model to predict tracklet linkage. The vesicles are initially detected with help of a candidate selection process, where the candidates are identified by a multi-scale spot enhancing filter. Subsequently, these candidates are pruned and selected by a light weight convolutional neural network. At the linking stage, the tracklets are formed based on the distance and the detection assignment which is implemented via combinatorial optimization algorithm. Each tracklet is described by a number of parameters used to evaluate the probability of tracklets connection by the inference over the Bayesian network. The tracking results are presented for confocal fluorescence microscopy data of protein trafficking in epithelial cells. The proposed method achieves a root mean square error (RMSE) of 1.39 for the vesicle localisation and of 0.7 representing the degree of track matching with ground truth. The presented method is also evaluated against the state-of-the-art “Trackmate“ framework

Imaging, Tracking and Computational Analyses of Virus Entry and Egress with the Cytoskeleton

Viruses have a dual nature: particles are “passive substances” lacking chemical energy transformation, whereas infected cells are “active substances” turning-over energy. How passive viral substances convert to active substances, comprising viral replication and assembly compartments has been of intense interest to virologists, cell and molecular biologists and immunologists. Infection starts with virus entry into a susceptible cell and delivers the viral genome to the replication site. This is a multi-step process, and involves the cytoskeleton and associated motor proteins. Likewise, the egress of progeny virus particles from the replication site to the extracellular space is enhanced by the cytoskeleton and associated motor proteins. This overcomes the limitation of thermal diffusion, and transports virions and virion components, often in association with cellular organelles. This review explores how the analysis of viral trajectories informs about mechanisms of infection. We discuss the methodology enabling researchers to visualize single virions in cells by fluorescence imaging and tracking. Virus visualization and tracking are increasingly enhanced by computational analyses of virus trajectories as well as in silico modeling. Combined approaches reveal previously unrecognized features of virus-infected cells. Using select examples of complementary methodology, we highlight the role of actin filaments and microtubules, and their associated motors in virus infections. In-depth studies of single virion dynamics at high temporal and spatial resolutions thereby provide deep insight into virus infection processes, and are a basis for uncovering underlying mechanisms of how cells function

Motion Analysis of Live Objects by Super-Resolution Fluorescence Microscopy

Motion analysis plays an important role in studing activities or behaviors of live objects in medicine, biotechnology, chemistry, physics, spectroscopy, nanotechnology, enzymology, and biological engineering. This paper briefly reviews the developments in this area mostly in the recent three years, especially for cellular analysis in fluorescence microscopy. The topic has received much attention with the increasing demands in biomedical applications. The tasks of motion analysis include detection and tracking of objects, as well as analysis of motion behavior, living activity, events, motion statistics, and so forth. In the last decades, hundreds of papers have been published in this research topic. They cover a wide area, such as investigation of cell, cancer, virus, sperm, microbe, karyogram, and so forth. These contributions are summarized in this review. Developed methods and practical examples are also introduced. The review is useful to people in the related field for easy referral of the state of the art