Particle detection and tracking in fluorescence time-lapse imaging: a contrario approach

This paper proposes a probabilistic approach for the detection and the tracking of particles in fluorescent time-lapse imaging. In the presence of a very noised and poor-quality data, particles and trajectories can be characterized by an a contrario model, that estimates the probability of observing the structures of interest in random data. This approach, first introduced in the modeling of human visual perception and then successfully applied in many image processing tasks, leads to algorithms that neither require a previous learning stage, nor a tedious parameter tuning and are very robust to noise. Comparative evaluations against a well-established baseline show that the proposed approach outperforms the state of the art.Comment: Published in Journal of Machine Vision and Application

Probabilistic Tracking and Behavior Identification of Fluorescent Particles

Explicit and tractable characterizations of the dynamical behavior of virus particles are pivotal for a thorough understanding of the infection mechanisms of viruses. This thesis deals with the problem of extracting symbolic representations of the dynamical behavior of fluorescent particles from fluorescence microscopy image sequences. The focus is on the behavior of virus particles such as fusion with the cell membrane. A numerical representation is obtained by tracking the particles in the image sequences. We have investigated probabilistic tracking approaches, including approaches based on the Kalman filter as well as based on particle filters. For reasons of efficiency and robustness, we developed a tracking approach based on the probabilistic data association (PDA) algorithm in combination with an ellipsoidal sampling scheme that exploits effectively the image data via parametric appearance models. To track objects in close proximity, we compute the support that each image position provides to each tracked object relative to the support provided to the object's neighbors. After tracking, the problem of mapping the trajectory information computed by the tracking approaches to symbolic representations of the behavior arises. To compute symbolic representations of behaviors related to the fusion of single virus particles with the cell membrane based on their intensity over time, we developed a layered probabilistic approach based on stochastic hybrid systems as well as hidden Markov models (HMMs). We use a maxbelief strategy to efficiently combine both representations. The layered approach describes the intensity, intensity models, and behaviors of single virus particles. We introduce models for the evolution of the intensity and the behavior. To compute estimates for the intensity, intensity models, and behaviors we use a hybrid particle filter and the Viterbi algorithm. The developed approaches have been applied to synthetic images as well as to real microscopy image sequences displaying human immunodeficiency virus (HIV-1) particles. We have performed an extensive quantitative evaluation of the performance and a comparison with several existing approaches. It turned out that our approaches outperform previous ones, thus yielding more accurate and more reliable information about the behavior of virus particles. Moreover, we have successfully applied our tracking approaches to 3D image sequences displaying herpes simplex virus (HSV) replication compartments. We also applied the tracking approaches to image data displaying microtubule tips and analyzed their motion. In addition, our tracking approaches were successfully applied to the 2D and 3D image data of a Particle Tracking Challenge

Deep Learning Methods for Detection and Tracking of Particles in Fluorescence Microscopy Images

Studying the dynamics of sub-cellular structures such as receptors, filaments, and vesicles is a prerequisite for investigating cellular processes at the molecular level. In addition, it is important to characterize the dynamic behavior of virus structures to gain a better understanding of infection mechanisms and to develop novel drugs. To investigate the dynamics of fluorescently labeled sub-cellular and viral structures, time-lapse fluorescence microscopy is the most often used imaging technique. Due to the limited spatial resolution of microscopes caused by diffraction, these very small structures appear as bright, blurred spots, denoted as particles, in microscopy images. To draw statistically meaningful biological conclusions, a large number of such particles need to be analyzed. However, since manual analysis of fluorescent particles is very time consuming, fully automated computer-based methods are indispensable. We introduce novel deep learning methods for detection and tracking of multiple particles in fluorescence microscopy images. We propose a particle detection method based on a convolutional neural network which performs image-to-image mapping by density map regression and uses the adaptive wing loss. For particle tracking, we present a recurrent neural network that exploits past and future information in both forward and backward direction. Assignment probabilities across multiple detections as well as the probabilities for missing detections are computed jointly. To resolve tracking ambiguities using future information, several track hypotheses are propagated to later time points. In addition, we developed a novel probabilistic deep learning method for particle tracking, which is based on a recurrent neural network mimicking classical Bayesian filtering. The method includes both aleatoric and epistemic uncertainty, and provides valuable information about the reliability of the computed trajectories. Short and long-term temporal dependencies of individual object dynamics are exploited for state prediction, and assigned detections are used to update the predicted states. Moreover, we developed a convolutional Long Short-Term Memory neural network for combined particle tracking and colocalization analysis in two-channel microscopy image sequences. The network determines colocalization probabilities, and colocalization information is exploited to improve tracking. Short and long-term temporal dependencies of object motion as well as image intensities are taken into account to compute assignment probabilities jointly across multiple detections. We also introduce a deep learning method for probabilistic particle detection and tracking. For particle detection, temporal information is integrated to regress a density map and determine sub-pixel particle positions. For tracking, a fully Bayesian neural network is presented that mimics classical Bayesian filtering and takes into account both aleatoric and epistemic uncertainty. Uncertainty information of individual particle detections is considered. Network training for the developed deep learning-based particle tracking methods relies only on synthetic data, avoiding the need of time-consuming manual annotation. We performed an extensive evaluation of our methods based on image data of the Particle Tracking Challenge as well as on fluorescence microscopy images displaying virus proteins of HCV and HIV, chromatin structures, and cell-surface receptors. It turned out that the methods outperform previous methods

Two-Dimensional Flow Nanometry of Biological Nanoparticles for Accurate Determination of Their Size and Emission Intensity

Biological nanoparticles (BNPs) are of high interest due to their key role in various biological processes and use as biomarkers. BNP size and molecular composition are decisive for their functions, but simultaneous determination of both properties with high accuracy remains challenging, which is a severe limitation. Surface-sensitive microscopy allows one to precisely determine fluorescence or scattering intensity, but not the size of individual BNPs. The latter is better determined by tracking their random motion in bulk, but the limited illumination volume for tracking this motion impedes reliable intensity determination. We here show that attaching BNPs (specifically, vesicles and functionalized gold NPs) to a supported lipid bilayer, subjecting them to a hydrodynamic flow, and tracking their motion via surface-sensitive imaging enable to determine their diffusion coefficients and flow-induced drift velocities and to accurately quantify both BNP size and emission intensity. For vesicles, the high accuracy is demonstrated by resolving the expected radius-squared dependence of their fluorescence intensity.Comment: 28 pages, 5 figure

Quantitative Live-Cell Imaging of Human Immunodeficiency Virus (HIV-1) Assembly

Advances in fluorescence methodologies make it possible to investigate biological systems in unprecedented detail. Over the last few years, quantitative live-cell imaging has increasingly been used to study the dynamic interactions of viruses with cells and is expected to become even more indispensable in the future. Here, we describe different fluorescence labeling strategies that have been used to label HIV-1 for live cell imaging and the fluorescence based methods used to visualize individual aspects of virus-cell interactions. This review presents an overview of experimental methods and recent experiments that have employed quantitative microscopy in order to elucidate the dynamics of late stages in the HIV-1 replication cycle. This includes cytosolic interactions of the main structural protein, Gag, with itself and the viral RNA genome, the recruitment of Gag and RNA to the plasma membrane, virion assembly at the membrane and the recruitment of cellular proteins involved in HIV-1 release to the nascent budding site

Objective comparison of particle tracking methods

Particle tracking is of key importance for quantitative analysis of intracellular dynamic processes from time-lapse microscopy image data. Because manually detecting and following large numbers of individual particles is not feasible, automated computational methods have been developed for these tasks by many groups. Aiming to perform an objective comparison of methods, we gathered the community and organized an open competition in which participating teams applied their own methods independently to a commonly defined data set including diverse scenarios. Performance was assessed using commonly defined measures. Although no single method performed best across all scenarios, the results revealed clear differences between the various approaches, leading to notable practical conclusions for users and developers

Imaging, Tracking and Computational Analyses of Virus Entry and Egress with the Cytoskeleton

Viruses have a dual nature: particles are “passive substances” lacking chemical energy transformation, whereas infected cells are “active substances” turning-over energy. How passive viral substances convert to active substances, comprising viral replication and assembly compartments has been of intense interest to virologists, cell and molecular biologists and immunologists. Infection starts with virus entry into a susceptible cell and delivers the viral genome to the replication site. This is a multi-step process, and involves the cytoskeleton and associated motor proteins. Likewise, the egress of progeny virus particles from the replication site to the extracellular space is enhanced by the cytoskeleton and associated motor proteins. This overcomes the limitation of thermal diffusion, and transports virions and virion components, often in association with cellular organelles. This review explores how the analysis of viral trajectories informs about mechanisms of infection. We discuss the methodology enabling researchers to visualize single virions in cells by fluorescence imaging and tracking. Virus visualization and tracking are increasingly enhanced by computational analyses of virus trajectories as well as in silico modeling. Combined approaches reveal previously unrecognized features of virus-infected cells. Using select examples of complementary methodology, we highlight the role of actin filaments and microtubules, and their associated motors in virus infections. In-depth studies of single virion dynamics at high temporal and spatial resolutions thereby provide deep insight into virus infection processes, and are a basis for uncovering underlying mechanisms of how cells function

Light Microscopy Combined with Computational Image Analysis Uncovers Virus-Specific Infection Phenotypes and Host Cell State Variability

Abstract: The study of virus infection phenotypes and variability plays a critical role in understanding viral pathogenesis and host response. Virus-host interactions can be investigated by light and various label-free microscopy methods, which provide a powerful tool for the spatiotemporal analysis of patterns at the cellular and subcellular levels in live or fixed cells. Analysis of microscopy images is increasingly complemented by sophisticated statistical methods and leverages artificial intelligence (AI) to address the tasks of image denoising, segmentation, classification, and tracking. Work in this thesis demonstrates that combining microscopy with AI techniques enables models that accurately detect and quantify viral infection due to the virus-induced cytopathic effect (CPE). Furthermore, it shows that statistical analysis of microscopy image data can disentangle stochastic and deterministic factors that contribute to viral infection variability, such as the cellular state. In summary, the integration of microscopy and computational image analysis offers a powerful and flexible approach for studying virus infection phenotypes and variability, ultimately contributing to a more advanced understanding of infection processes and creating possibilities for the development of more effective antiviral strategies