Particle detection and tracking in fluorescence time-lapse imaging: a contrario approach

This paper proposes a probabilistic approach for the detection and the tracking of particles in fluorescent time-lapse imaging. In the presence of a very noised and poor-quality data, particles and trajectories can be characterized by an a contrario model, that estimates the probability of observing the structures of interest in random data. This approach, first introduced in the modeling of human visual perception and then successfully applied in many image processing tasks, leads to algorithms that neither require a previous learning stage, nor a tedious parameter tuning and are very robust to noise. Comparative evaluations against a well-established baseline show that the proposed approach outperforms the state of the art.Comment: Published in Journal of Machine Vision and Application

Unraveling the Thousand Word Picture: An Introduction to Super-Resolution Data Analysis

Super-resolution microscopy provides direct insight into fundamental biological processes occurring at length scales smaller than light’s diffraction limit. The analysis of data at such scales has brought statistical and machine learning methods into the mainstream. Here we provide a survey of data analysis methods starting from an overview of basic statistical techniques underlying the analysis of super-resolution and, more broadly, imaging data. We subsequently break down the analysis of super-resolution data into four problems: the localization problem, the counting problem, the linking problem, and what we’ve termed the interpretation problem

Automated detection and analysis of fluorescence changes evoked by molecular signalling

Fluorescent dyes and genetically encoded fluorescence indicators (GEFI) are common tools for visualizing concentration changes of specific ions and messenger molecules during intra- as well as intercellular communication. While fluorescent dyes have to be directly loaded into target cells and function only transiently, the expression of GEFIs can be controlled in a cell and time-specific fashion, even allowing long-term analysis in living organisms. Dye and GEFI based fluorescence fluctuations, recorded using advanced imaging technologies, are the foundation for the analysis of physiological molecular signaling. Analyzing the plethora of complex fluorescence signals is a laborious and time-consuming task. An automated analysis of fluorescent signals circumvents user bias and time constraints. However, it requires to overcome several challenges, including correct estimation of fluorescence fluctuations at basal concentrations of messenger molecules, detection and extraction of events themselves, proper segmentation of neighboring events as well as tracking of propagating events. Moreover, event detection algorithms need to be sensitive enough to accurately capture localized and low amplitude events exhibiting a limited spatial extent. This thesis presents three novel algorithms, PBasE, CoRoDe and KalEve, for the automated analysis of fluorescence events, developed to overcome the aforementioned challenges. The algorithms are integrated into a graphical application called MSparkles, specifically designed for the analysis of fluorescence signals, developed in MATLAB. The capabilities of the algorithms are demonstrated by analyzing astroglial Ca2+ events, recorded in anesthetized and awake mice, visualized using genetically encoded Ca2+ indicators (GECIs) GCaMP3 as well as GCaMP5. The results were compared to those obtained by other software packages. In addition, the analysis of neuronal Na+ events recorded in acute brain slices using SBFI-AM serve to indicate the putatively broad application range of the presented algorithms. Finally, due to increasing evidence of the pivotal role of astrocytes in neurodegenerative diseases such as epilepsy, a metric to assess the synchronous occurrence of fluorescence events is introduced. In a proof-of-principle analysis, this metric is used to correlate astroglial Ca2+ events with EEG measurementsFluoreszenzfarbstoffe und genetisch kodierte Fluoreszenzindikatoren (GEFI) sind gängige Werkzeuge zur Visualisierung von Konzentrationsänderungen bestimmter Ionen und Botenmoleküle der intra- sowie interzellulären Kommunikation. Während Fluoreszenzfarbstoffe direkt in die Zielzellen eingebracht werden müssen und nur über einen begrenzten Zeitraum funktionieren, kann die Expression von GEFIs zell- und zeitspezifisch gesteuert werden, was darüber hinaus Langzeitanalysen in lebenden Organismen ermöglicht. Farbstoff- und GEFI-basierte Fluoreszenzfluktuationen, die mit Hilfe moderner bildgebender Verfahren aufgezeichnet werden, bilden die Grundlage für die Analyse physiologischer molekularer Kommunikation. Die Analyse einer großen Zahl komplexer Fluoreszenzsignale ist jedoch eine schwierige und zeitaufwändige Aufgabe. Eine automatisierte Analyse ist dagegen weniger zeitaufwändig und unabhängig von der Voreingenommenheit des Anwenders. Allerdings müssen hierzu mehrere Herausforderungen bewältigt werden. Unter anderem die korrekte Schätzung von Fluoreszenzschwankungen bei Basalkonzentrationen von Botenmolekülen, die Detektion und Extraktion von Signalen selbst, die korrekte Segmentierung benachbarter Signale sowie die Verfolgung sich ausbreitender Signale. Darüber hinaus müssen die Algorithmen zur Signalerkennung empfindlich genug sein, um lokalisierte Signale mit geringer Amplitude sowie begrenzter räumlicher Ausdehnung genau zu erfassen. In dieser Arbeit werden drei neue Algorithmen, PBasE, CoRoDe und KalEve, für die automatische Extraktion und Analyse von Fluoreszenzsignalen vorgestellt, die entwickelt wurden, um die oben genannten Herausforderungen zu bewältigen. Die Algorithmen sind in eine grafische Anwendung namens MSparkles integriert, die speziell für die Analyse von Fluoreszenzsignalen entwickelt und in MATLAB implementiert wurde. Die Fähigkeiten der Algorithmen werden anhand der Analyse astroglialer Ca2+-Signale demonstriert, die in narkotisierten sowie wachen Mäusen aufgezeichnet und mit den genetisch kodierten Ca2+-Indikatoren (GECIs) GCaMP3 und GCaMP5 visualisiert wurden. Erlangte Ergebnisse werden anschließend mit denen anderer Softwarepakete verglichen. Darüber hinaus dient die Analyse neuronaler Na+-Signale, die in akuten Hirnschnitten mit SBFI-AM aufgezeichnet wurden, dazu, den breiten Anwendungsbereich der Algorithmen aufzuzeigen. Zu guter Letzt wird aufgrund der zunehmenden Indizien auf die zentrale Rolle von Astrozyten bei neurodegenerativen Erkrankungen wie Epilepsie eine Metrik zur Bewertung des synchronen Auftretens fluoreszenter Signale eingeführt. In einer Proof-of-Principle-Analyse wird diese Metrik verwendet, um astrogliale Ca2+-Signale mit EEG-Messungen zu korrelieren

MEMS Technology for Biomedical Imaging Applications

Biomedical imaging is the key technique and process to create informative images of the human body or other organic structures for clinical purposes or medical science. Micro-electro-mechanical systems (MEMS) technology has demonstrated enormous potential in biomedical imaging applications due to its outstanding advantages of, for instance, miniaturization, high speed, higher resolution, and convenience of batch fabrication. There are many advancements and breakthroughs developing in the academic community, and there are a few challenges raised accordingly upon the designs, structures, fabrication, integration, and applications of MEMS for all kinds of biomedical imaging. This Special Issue aims to collate and showcase research papers, short commutations, perspectives, and insightful review articles from esteemed colleagues that demonstrate: (1) original works on the topic of MEMS components or devices based on various kinds of mechanisms for biomedical imaging; and (2) new developments and potentials of applying MEMS technology of any kind in biomedical imaging. The objective of this special session is to provide insightful information regarding the technological advancements for the researchers in the community

Nanoscale Magnetometry with Single Fluorescent Nanodiamonds Manipulated in an Anti-Brownian Electrokinetic Trap

Studies on single-molecule spectroscopy and nanoscale detection have been remarkably driven by an interest to reveal quantum and conformational states of single particles, the intra-molecular dynamics and their response to physical observables hidden by ensemble level measurements. A straightforward practice used in enhancing the signal from single particles is either to immobilize them on an engineered substrate or to embed them in a solid matrix. Given that the biophysical properties of the host environment introduce new perturbations and the particles will not behave as in their native environment, such approaches are inefficient to reflect the real dynamics. Therefore, recent advances in the field of single-molecule have led to a renewed interest in novel trapping methods, increased efforts into the development of promising tools for extended investigation, and the manipulation of solution-phase bio-molecules in real time. Despite the variety of successful passive trapping techniques, precise manipulation through non-perturbative forces is a big challenge for nano-sized particles. Such techniques either exert high power to the sample or compel special operating conditions disturbing the native environment. Therefore, an active trapping scheme guiding non-perturbative forces can break the trade-off between the particle size and the excreted power. This dissertation presents the development of an active trapping set-up using non-perturbative electrokinetic feedback and demonstrates its performance on nano-sized single particles for aims in biophysics. The essential theme is the engineering aspect of the technique, including the feedback configurations for various fluidic devices, the corresponding particle tracking schemes and the integration of the trapping platform to an integrated circuit pattern for advanced manipulation aims. The second theme is on specialized single fluorescence nanodiamonds (FNDs) as scanning magnetometer in fluidics. The implemented active trapping tool is employed for the manipulation of a rotationally free single FND to detect the localized magnetic field through an optically detected magnetic resonance (ODMR) spectrum. While the laser beam used in particle tracking can serve in optical excitation, an external radio frequency (RF) source is not sufficient to achieve microwave manipulation. Therefore, an RF antenna is designed to transmit the microwave signal to the proximity of the trapping chamber for electron spin resonance (ESR) spectroscopy. A nanostage positioning controller introduces scanning ability to the sample plane, in relative position of the trapped particle, in order to map the distribution of the detected fields over a fluidic volume. As FNDs are also sensitive to many other physical quantities, nanoscale single particle trapping and diamond photonics linkages are realized in this work, which provide an outstanding alternative for detection and imaging in complex fluidic environments that are closed to AFM-like physically supported probes

Functional Effects of Calcium Regulation of Thin Filaments at Single Particle Resolution

Heart disease is the leading cause of death in the United States. Understanding heart function at the molecular level is critical for developing of more effective treatments. In the cardiac muscle, the thin filament is composed by troponin (Tn), tropomyosin (Tm), and F-actin. It provides Ca2+-dependent regulation of contraction by modulating myosin attachment and force generation in a cooperative scheme. However, this mechanism remains unclear. To understand thin filament activation, we studied the binding and functional properties of Tn and Tm to F-actin at single particle resolution by employing fluorescence image colocalization, in vitro motility assays, and Förster resonance energy transfer based on fluorescence lifetime imaging (FLIM-FRET). Our results suggest that under physiologically relevant conditions, Tn and Tm binding to Factin is not cooperative and it is not affected by Ca2+. This suggests that one single type of interaction is involved in fully regulated thin filaments. Thin filament activation has been confirmed by in vitro motility assays, where phosphorylation of serine 23 and 24 in TnI, truncation of TnT C-terminal region, and incorporation of a Ca2+ desensitizer altered the Ca2+ response to filament sliding. FLIM-FRET measurements revealed an allosteric dependence on thin filament activation as a function of myosin and Ca2+. Our results provide evidence of multiple allosteric elements within thin filaments responsible for the molecular modulation of cardiac muscle activation

Quantitative analysis of microscopy

Particle tracking is an essential tool for the study of dynamics of biological processes. The dynamics of these processes happens in three-dimensional (3D) space as the biological structures themselves are 3D. The focus of this thesis is on the development of single particle tracking methods for analysis of the dynamics of biological processes through the use of image processing techniques. Firstly, introduced is a novel particle tracking method that works with two-dimensional (2D) image data. This method uses the theory of Haar-like features for particle detection and trajectory linking is achieved using a combination of three Kalman filters within an interacting multiple models framework. The trajectory linking process utilises an extended state space variable which better describe the morphology and intensity profiles of the particles under investigation at their current position. This tracking method is validated using both 2D synthetically generated images as well as 2D experimentally collected images. It is shown that this method outperforms 14 other stateof-the-art methods. Next this method is used to analyse the dynamics of fluorescently labelled particles using a live-cell fluorescence microscopy technique, specifically a variant of the super-resolution (SR) method PALM, spt-PALM. From this application, conclusions about the organisation of the proteins under investigation at the cell membrane are drawn. Introduced next is a second particle tracking method which is highly efficient and capable of working with both 2D and 3D image data. This method uses a novel Haar-inspired feature for particle detection, drawing inspiration from the type of particles to be detected which are typically circular in 2D space and spherical in 3D image space. Trajectory linking in this method utilises a global nearest neighbour methodology incorporating both motion models to describe the motion of the particles under investigation and a further extended state space variable describing many more aspects of the particles to be linked. This method is validated using a variety of both 2D and 3D synthetic image data. The methods performance is compared with 14 other state-of-the-art methods showing it to be one of the best overall performing methods. Finally, analysis tools to study a SR image restoration method developed by our research group, referred to as Translation Microscopy (TRAM) are investigated [1]. TRAM can be implemented on any standardised microscope and deliver an improvement in resolution of up to 7-fold. However, the results from TRAM and other SR imaging methods require specialised tools to validate and analyse them. Tools have been developed to validate that TRAM performs correctly using a specially designed ground truth. Furthermore, through analysis of results on a biological sample corroborate other published results based on the size of biological structures, showing again that TRAM performs as expected.EPSC

Piecewise-stationary motion modeling and iterative smoothing to track heterogeneous particle motions in dense environments

International audienceOne of the major challenges in multiple particle tracking is the capture of extremely heterogeneous movements of objects in crowded scenes. The presence of numerous assignment candidates in the expected range of particle motion makes the tracking ambiguous and induces false positives. Lowering the ambiguity by reducing the search range, on the other hand, is not an option, as this would increase the rate of false negatives. We propose here a piecewise-stationary motion model (PMM) for the particle transport along an iterative smoother that exploits recursive tracking in multiple rounds in forward and backward temporal directions. By fusing past and future information, our method, termed PMMS, can recover fast transitions from freely or confined diffusive to directed motions with linear time complexity. To avoid false positives we complemented recursive tracking with a robust inline estimator of the search radius for assignment (a.k.a. gating), where past and future information are exploited using only two frames at each optimization step. We demonstrate the improvement of our technique on simulated data – especially the impact of density, variation in frame to frame displacements, and motion switching probability. We evaluated our technique on the 2D particle tracking challenge dataset published by Chenouard et al in 2014. Using high SNR to focus on motion modeling challenges, we show superior performance at high particle density. On biological applications, our algorithm allows us to quantify the extremely small percentage of motor-driven movements of fluorescent particles along microtubules in a dense field of unbound, diffusing particles. We also show with virus imaging that our algorithm can cope with a strong reduction in recording frame rate while keeping the same performance relative to methods relying on fast sampling

Deep Learning Methods for Detection and Tracking of Particles in Fluorescence Microscopy Images

Studying the dynamics of sub-cellular structures such as receptors, filaments, and vesicles is a prerequisite for investigating cellular processes at the molecular level. In addition, it is important to characterize the dynamic behavior of virus structures to gain a better understanding of infection mechanisms and to develop novel drugs. To investigate the dynamics of fluorescently labeled sub-cellular and viral structures, time-lapse fluorescence microscopy is the most often used imaging technique. Due to the limited spatial resolution of microscopes caused by diffraction, these very small structures appear as bright, blurred spots, denoted as particles, in microscopy images. To draw statistically meaningful biological conclusions, a large number of such particles need to be analyzed. However, since manual analysis of fluorescent particles is very time consuming, fully automated computer-based methods are indispensable. We introduce novel deep learning methods for detection and tracking of multiple particles in fluorescence microscopy images. We propose a particle detection method based on a convolutional neural network which performs image-to-image mapping by density map regression and uses the adaptive wing loss. For particle tracking, we present a recurrent neural network that exploits past and future information in both forward and backward direction. Assignment probabilities across multiple detections as well as the probabilities for missing detections are computed jointly. To resolve tracking ambiguities using future information, several track hypotheses are propagated to later time points. In addition, we developed a novel probabilistic deep learning method for particle tracking, which is based on a recurrent neural network mimicking classical Bayesian filtering. The method includes both aleatoric and epistemic uncertainty, and provides valuable information about the reliability of the computed trajectories. Short and long-term temporal dependencies of individual object dynamics are exploited for state prediction, and assigned detections are used to update the predicted states. Moreover, we developed a convolutional Long Short-Term Memory neural network for combined particle tracking and colocalization analysis in two-channel microscopy image sequences. The network determines colocalization probabilities, and colocalization information is exploited to improve tracking. Short and long-term temporal dependencies of object motion as well as image intensities are taken into account to compute assignment probabilities jointly across multiple detections. We also introduce a deep learning method for probabilistic particle detection and tracking. For particle detection, temporal information is integrated to regress a density map and determine sub-pixel particle positions. For tracking, a fully Bayesian neural network is presented that mimics classical Bayesian filtering and takes into account both aleatoric and epistemic uncertainty. Uncertainty information of individual particle detections is considered. Network training for the developed deep learning-based particle tracking methods relies only on synthetic data, avoiding the need of time-consuming manual annotation. We performed an extensive evaluation of our methods based on image data of the Particle Tracking Challenge as well as on fluorescence microscopy images displaying virus proteins of HCV and HIV, chromatin structures, and cell-surface receptors. It turned out that the methods outperform previous methods