Bayesian multi-target tracking: application to total internal reflection fluorescence microscopy

This thesis focuses on the problem of automated tracking of tiny cellular and sub-cellular structures, known as particles, in the sequences acquired from total internal reflection fluorescence microscopy (TIRFM) imaging technique. Our primary biological motivation is to develop an automated system for tracking the sub-cellular structures involving exocytosis (an intracellular mechanism) which is helpful for studying the possible causes of the defects in diseases such as diabetes and obesity. However, all methods proposed in this thesis are generalized to be applicable for a wide range of particle tracking applications. A reliable multi-particle tracking method should be capable of tracking numerous similar objects in the presence of high levels of noise, high target density and complex motions and interactions. In this thesis, we choose the Bayesian filtering framework as our main approach to deal with this problem. We focus on the approaches that work based on detections. Therefore, in this thesis, we first propose a method that robustly detects the particles in the noisy TIRFM sequences with inhomogeneous and time-varying background. In order to evaluate our detection and tracking methods on the sequences with known and reliable ground truth, we also present a framework for generating realistic synthetic TIRFM data. To propose a reliable multi-particle tracking method for TIRFM sequences, we suggest a framework by combining two robust Bayesian filters, the interacting multiple model and joint probabilistic data association (IMM-JPDA) filters. The performance of our particle tracking method is compared against those of several popular and state-of-the art particle tracking approaches on both synthetic and real sequences. Although our approach performs well in tracking particles, it can be very computationally demanding for the applications with dense targets with poor detections. To propose a computationally cheap, but reliable, multi-particle tracking method, we investigate the performance of a recent multi-target Bayesian filter based on random finite theory, the probability hypothesis density (PHD) filter, on our application. To this end, we propose a general framework for tracking particles using this filter. Moreover, we assess the performance of our proposed PHD filter on both synthetic and real sequences with high level of noise and particle density. We compare its results from both aspects of accuracy and processing time against our IMM-JPDA filter. Finally, we suggest a framework for tracking particles in a challenging problem where the noise characteristic and the background intensity of sequences change during the acquisition process which make detection profile and clutter rate time-variant. To deal with this, we propose a bootstrap filter using another type of the random finite set based Bayesian filters, the cardinalized PHD (CPHD) filter, composed of an estimator and a tracker. The estimator adaptively estimates the required meta parameters for the tracker such as clutter rate and the detection probability while the tracker estimates the state of the targets. We evaluate the performance of our bootstrap on both synthetic and real sequences under these time-varying conditions. Moreover, its performance is compared against those of our other particle trackers as well as the state-of-the art particle tracking approaches

A multiple model probability hypothesis density tracker for time-lapse cell microscopy sequences

Abstract. Quantitative analysis of the dynamics of tiny cellular and subcellular structures in time-lapse cell microscopy sequences requires the development of a reliable multi-target tracking method capable of tracking numerous similar targets in the presence of high levels of noise, high target density, maneuvering motion patterns and intricate interactions. The linear Gaussian jump Markov system probability hypothesis density (LGJMS-PHD) filter is a recent Bayesian tracking filter that is well-suited for this task. However, the existing recursion equations for this filter do not consider a state-dependent transition probability matrix. As required in many biological applications, we propose a new closed-form recursion that incorporates this assumption and introduce a general framework for particle tracking using the proposed filter. We apply our scheme to multi-target tracking in total internal reflection fluorescence microscopy (TIRFM) sequences and evaluate the performance of our filter against the existing LGJMS-PHD and IMM-JPDA filters

Marker-Less Stage Drift Correction in Super-Resolution Microscopy Using the Single-Cluster PHD Filter

Fluorescence microscopy is a technique which allows the imaging of cellular and intracellular dynamics through the activation of fluorescent molecules attached to them. It is a very important technique because it can be used to analyze the behavior of intracellular processes in vivo in contrast to methods like electron microscopy. There are several challenges related to the extraction of meaningful information from images acquired from optical microscopes due to the low contrast between objects and background and the fact that point-like objects are observed as blurred spots due to the diffraction limit of the optical system. Another consideration is that for the study of intracellular dynamics, multiple particles must be tracked at the same time, which is a challenging task due to problems such as the presence of false positives and missed detections in the acquired data. Additionally, the objective of the microscope is not completely static with respect to the cover slip due to mechanical vibrations or thermal expansions which introduces bias in the measurements. In this paper, a Bayesian approach is used to simultaneously track the locations of objects with different motion behaviors and the stage drift using image data obtained from fluorescence microscopy experiments. Namely, detections are extracted from the acquired frames using image processing techniques, and then these detections are used to accurately estimate the particle positions and simultaneously correct the drift introduced by the motion of the sample stage. A single cluster Probability Hypothesis Density (PHD) filter with object classification is used for the estimation of the multiple target state assuming different motion behaviors. The detection and tracking methods are tested and their performance is evaluated on both simulated and real data

Automatic approach for spot detection in microscopy imaging based on image processing and statistical analysis

Abstract: In biological research, fluorescence microscopy has become one of the vital tools used for observation, allowing researchers to study, visualise and image the details of intracel-lular structures which result in better understanding of biology. However, analysis of large numbers of samples is often required to draw statistically verifiable conclusions. Automated methods for analysis of microscopy image data make it possible to handle large datasets, and at the same time reduce the risk of bias imposed by manual techniques in the image analysis pipeline. This work covers automated methods for extracting quan-titative measurements from microscopy images, enabling the detection of spots resulting from different experimental conditions. The work resulted in four main significant con-tributions developed around the microscopy image analysis pipeline. Firstly, an investiga-tion into the importance of spot detection within the automated image analysis pipeline is conducted. Experimental findings show that poor spot detection adversely affected the remainder of the processing pipeline...D.Ing. (Electrical and Electronic Engineering

Image Processing and Simulation Toolboxes of Microscopy Images of Bacterial Cells

Recent advances in microscopy imaging technology have allowed the characterization of the dynamics of cellular processes at the single-cell and single-molecule level. Particularly in bacterial cell studies, and using the E. coli as a case study, these techniques have been used to detect and track internal cell structures such as the Nucleoid and the Cell Wall and fluorescently tagged molecular aggregates such as FtsZ proteins, Min system proteins, inclusion bodies and all the different types of RNA molecules. These studies have been performed with using multi-modal, multi-process, time-lapse microscopy, producing both morphological and functional images. To facilitate the finding of relationships between cellular processes, from small-scale, such as gene expression, to large-scale, such as cell division, an image processing toolbox was implemented with several automatic and/or manual features such as, cell segmentation and tracking, intra-modal and intra-modal image registration, as well as the detection, counting and characterization of several cellular components. Two segmentation algorithms of cellular component were implemented, the first one based on the Gaussian Distribution and the second based on Thresholding and morphological structuring functions. These algorithms were used to perform the segmentation of Nucleoids and to identify the different stages of FtsZ Ring formation (allied with the use of machine learning algorithms), which allowed to understand how the temperature influences the physical properties of the Nucleoid and correlated those properties with the exclusion of protein aggregates from the center of the cell. Another study used the segmentation algorithms to study how the temperature affects the formation of the FtsZ Ring. The validation of the developed image processing methods and techniques has been based on benchmark databases manually produced and curated by experts. When dealing with thousands of cells and hundreds of images, these manually generated datasets can become the biggest cost in a research project. To expedite these studies in terms of time and lower the cost of the manual labour, an image simulation was implemented to generate realistic artificial images. The proposed image simulation toolbox can generate biologically inspired objects that mimic the spatial and temporal organization of bacterial cells and their processes, such as cell growth and division and cell motility, and cell morphology (shape, size and cluster organization). The image simulation toolbox was shown to be useful in the validation of three cell tracking algorithms: Simple Nearest-Neighbour, Nearest-Neighbour with Morphology and DBSCAN cluster identification algorithm. It was shown that the Simple Nearest-Neighbour still performed with great reliability when simulating objects with small velocities, while the other algorithms performed better for higher velocities and when there were larger clusters present

Optical dispersions through intracellular inhomogeneities

Transport of intensity equation (TIE) exhibits a non-interferometric correlation between intensity and phase variations of intermediate fields (e.g., light and electron) in biological imaging. Previous TIE formulations have generally assumed a free space propagation of monochromatic coherent field functions crossing phase distributions along a longitudinal direction. Here, we modify the TIE with fractal (or self-similar) organization models based on intracellular refractive index turbulence. We then implement the TIE simulation over a broad range of fractal dimensions and wavelengths. Simulation results show how the intensity propagation through the spatial fluctuation of intracellular refractive index interconnects fractal-dimensionality with intensity dispersion (or transmissivity) within the picometer to micrometer wavelength range. In addition, we provide a spatial-autocorrelation of phase derivatives which allows the direct measurement and reconstruction of intracellular fractal profiles from optical and electron microscopy imaging.Comment: 22 pages, 18 figures, 4 table

SPOT DETECTION METHODS IN FLUORESCENCE MICROSCOPY IMAGING: A REVIEW

Fluorescence microscopy imaging has become one of the essential tools used by biologists to visualize and study intracellular particles within a cell. Studying these particles is a long-term research effort in the field of microscopy image analysis, consisting of discovering the relationship between the dynamics of particles and their functions. However, biologists are faced with challenges such as the counting and tracking of these intracellular particles. To overcome the issues faced by biologists, tools which can extract the location and motion of these particles are essential. One of the most important steps in these analyses is to accurately detect particle positions in an image, termed spot detection. The detection of spots in microscopy imaging is seen as a critical step for further quantitative analysis. However, the evaluation of these microscopic images is mainly conducted manually, with automated methods becoming popular. This work presents some advances in fluorescence microscopy image analysis, focusing on the detection methods needed for quantifying the location of these spots. We review several existing detection methods in microscopy imaging, along with existing synthetic benchmark datasets and evaluation metrics

Simulation of biologically inspired object movement for the study of object tracking algorithms

Major advances in Cell and Molecular Biology have been associated with the advances in live-cell microscopy imaging, and these studies started to rely on temporal single cell imaging. To support these efforts, available automated image analysis methods such as cell segmentation and cell tracking during a time-series analysis should be improved. One important step is the validation of such image processing methods. Ideally, the “ground truth” should be known, which is possible only by manually labelling images or by artificially produced images. To simulate such artificial images we developed a platform that can simulate biologically inspired objects, by generating bodies with different morphologies, physical movement and that can aggregate in clusters. Using this platform, we tested and compared four tracking algorithms: Simple Nearest-Neighbour (NN), NN with Morphology and two DBSCAN based ones. In this work we showed that Simple NN work for small object velocities, while the other algorithms perform better on higher velocities and when clustered. This platform can generate new benchmark images and is openly available to test other tracking algorithms. (http://griduni.uninova.pt/Clustergen/ClusterGen_v1.0.zip