Optical Flow on Evolving Surfaces with an Application to the Analysis of 4D Microscopy Data

We extend the concept of optical flow to a dynamic non-Euclidean setting. Optical flow is traditionally computed from a sequence of flat images. It is the purpose of this paper to introduce variational motion estimation for images that are defined on an evolving surface. Volumetric microscopy images depicting a live zebrafish embryo serve as both biological motivation and test data.Comment: The final publication is available at link.springer.co

3D Flow Field Estimation and Assessment for Live Cell Fluorescence Microscopy

International audienceMotivation: The revolution in light sheet microscopy enables the concurrent observation of thousands of dynamic processes, from single molecules to cellular organelles, with high spatiotemporal resolution. However, challenges in the interpretation of multidimensional data requires the fully automaticmeasurement of those motions to link local processes to cellular functions. This includes the design and the implementation of image processing pipelines able to deal with diverse motion types, and 3D visualization tools adapted to the human visual system.Results: Here, we describe a new method for 3D motion estimation that addresses the aforementioned issues. We integrate 3D matching and variational approach to handle a diverse range of motion without any prior on the shape of moving objects. We compare dierent similarity measures to cope with intensity ambiguities and demonstrate the eectiveness of the Census signature for both stages. Additionally, wepresent two intuitive visualization approaches to adapt complex 3D measures into an interpretable 2D view, and a novel way to assess the quality of flow estimates in absence of ground truth

Filter-Based Probabilistic Markov Random Field Image Priors: Learning, Evaluation, and Image Analysis

Markov random fields (MRF) based on linear filter responses are one of the most popular forms for modeling image priors due to their rigorous probabilistic interpretations and versatility in various applications. In this dissertation, we propose an application-independent method to quantitatively evaluate MRF image priors using model samples. To this end, we developed an efficient auxiliary-variable Gibbs samplers for a general class of MRFs with flexible potentials. We found that the popular pairwise and high-order MRF priors capture image statistics quite roughly and exhibit poor generative properties. We further developed new learning strategies and obtained high-order MRFs that well capture the statistics of the inbuilt features, thus being real maximum-entropy models, and other important statistical properties of natural images, outlining the capabilities of MRFs. We suggest a multi-modal extension of MRF potentials which not only allows to train more expressive priors, but also helps to reveal more insights of MRF variants, based on which we are able to train compact, fully-convolutional restricted Boltzmann machines (RBM) that can model visual repetitive textures even better than more complex and deep models. The learned high-order MRFs allow us to develop new methods for various real-world image analysis problems. For denoising of natural images and deconvolution of microscopy images, the MRF priors are employed in a pure generative setting. We propose efficient sampling-based methods to infer Bayesian minimum mean squared error (MMSE) estimates, which substantially outperform maximum a-posteriori (MAP) estimates and can compete with state-of-the-art discriminative methods. For non-rigid registration of live cell nuclei in time-lapse microscopy images, we propose a global optical flow-based method. The statistics of noise in fluorescence microscopy images are studied to derive an adaptive weighting scheme for increasing model robustness. High-order MRFs are also employed to train image filters for extracting important features of cell nuclei and the deformation of nuclei are then estimated in the learned feature spaces. The developed method outperforms previous approaches in terms of both registration accuracy and computational efficiency

Deep-learning-based 2.5D flow field estimation for maximum intensity projections of 4D optical coherence tomography

In microsurgery, lasers have emerged as precise tools for bone ablation. A challenge is automatic control of laser bone ablation with 4D optical coherence tomography (OCT). OCT as high resolution imaging modality provides volumetric images of tissue and foresees information of bone position and orientation (pose) as well as thickness. However, existing approaches for OCT based laser ablation control rely on external tracking systems or invasively ablated artificial landmarks for tracking the pose of the OCT probe relative to the tissue. This can be superseded by estimating the scene flow caused by relative movement between OCT-based laser ablation system and patient. Therefore, this paper deals with 2.5D scene flow estimation of volumetric OCT images for application in laser ablation. We present a semi-supervised convolutional neural network based tracking scheme for subsequent 3D OCT volumes and apply it to a realistic semi-synthetic data set of ex vivo human temporal bone specimen. The scene flow is estimated in a two-stage approach. In the first stage, 2D lateral scene flow is computed on census-transformed en-face arguments-of-maximum intensity projections. Subsequent to this, the projections are warped by predicted lateral flow and 1D depth flow is estimated. The neural network is trained semi-supervised by combining error to ground truth and the reconstruction error of warped images with assumptions of spatial flow smoothness. Quantitative evaluation reveals a mean endpoint error of (4.7 ± 3.5) voxel or (27.5 ± 20.5) μm for scene flow estimation caused by simulated relative movement between the OCT probe and bone. The scene flow estimation for 4D OCT enables its use for markerless tracking of mastoid bone structures for image guidance in general, and automated laser ablation control. © 2019 SPIE

IMPROVING THE SPEED AND OPTICAL SECTIONING OF FLUORESCENCE MICROSCOPY TECHNIQUES FOR BIOPHYSICAL ANALYSIS OF SUBCELLULAR PROCESSES

This dissertation focuses on novel fluorescence microscopy techniques and the biophysical analysis of cell biology enabled by such techniques. Modern cell biology research benefits greatly from the ability to accurately visualize the inner workings of cells. Fluorescence microscopy is particularly well suited to imaging live cells, as it is gentle enough to avoid damaging cells, provides sufficient spatial resolution to image small cellular features, and targets and visualizes specific cell structures and processes with high contrast. An additional feature that is often desirable in fluorescence microscopy is the ability to image rapidly enough to freeze the motion of dynamic cell processes, yet technical limitations make imaging with both high spatial and temporal resolution challenging. In this thesis I address methods for improving the speed, spatial resolution, and optical sectioning of fluorescence microscopy techniques. I then apply some of these innovations to study actin structures and dynamics in epithelial cells. Because of its role in driving cellular motion, targeted studies of the actin cytoskeleton using fluorescence microscopy can be used to examine cell migration dynamics. In both in vivo and in vitro experiments, I use high spatiotemporal resolution fluorescence microscopy techniques to provide insight into the role of the actin cytoskeleton in responding to external structural stimuli

Fast, Three-Dimensional Fluorescence Imaging of Living Cells

This thesis focuses on multi-plane fluorescence microscopy for fast live-cell imaging. To improve the performance of multi-plane microscopy, I developed new image analysis methods. I used these methods to measure and analyze the movements of cardiomyocytesand Dictyostelium discoideum cells.The multi-plane setup is based on a conventional wide-field microscope using a custom multiple beam-splitter in the detection path. This prism creates separate images of eight distinct focal planes in the sample. Since 3D volume is imaged without scanning, three-dimensional imaging at a very high speed becomes possible. However, as in conventional wide-field microscopy, the "missing cone" of spatial frequencies along the optical axis in the optical transfer function (OTF) prevents optical sectioning in such a microscope. This is in stark contrast to other truly three-dimensional imaging modalities like confocal and light-sheet microscopy. In order to overcome the lack of optical sectioning, I developed a new deconvolution method. Deconvolution describes methods that restore or sharpen an image based on physical assumptions and knowledge of the imaging process. Deconvolution methods have been widely used to sharpen images of microscopes and telescopes. The recently developed SUPPOSe algorithm is a deconvolution algorithm that uses a set of numerous virtual point sources. It tries to reconstruct an image by distributing these point sources in space and optimizing their positions so that the resulting image reproduces as good as possible the measured data. SUPPOSe has never been used for 3D images. Compared to other algorithms, this method has superior performance when the number of pixels is increased by interpolation. In this work, I extended the method to work also with 3D image data. The 3D-SUPPOSe program is suitable for analyzing data of our multi-plane setup. The multi-plane setup has only eight vertically aligned image planes. Furthermore, for accurate reconstruction of 3D images, I studied a method of correcting each image plane's relative brightness constituting an image, and I also developed a method of measuring the movement of point emitters in 3D space. Using these methods, I measured and analyzed the beating motion of cardiomyocytes and the chemotaxis of Dicyosteilium discoidem. Cardiomyocytes are the cells of the heart muscle and consist of repetitive sarcomeres. These cells are characterized by fast and periodic movements, and so far the dynamics of these cells was studied only with two-dimensional imaging. In this thesis, the beating motion was analyzed by tracing the spatial distribution of the so-called z-discs, one of the constituent components of cardiomyocytes. I found that the vertical distribution of $\alpha$-actinine-2 in a single z-disc changed very rapidly, which may serve as a starting point for a better understanding the motion of cardiomyocytes. \textit{Dictyostelium discoideum} is a well established single cell model organism that migrates along the gradient of a chemoattractant. One has conducted much research to understand the mechanism of chemotaxis, and many efforts have been made to understand the role of actin in the chemotactic motion. By suppressing the motor protein, myosin, a cell line was created that prevented the formation of normal actin filaments. In these myosin null cells, F-actin moves in a flow-like behaviour and induces cell movement. In this study, I imaged the actin dynamics, and I analyzed the flow using the newly created deconvolution and flow estimation methods. As a result of the analysis, the spatio-temporal correlation between pseudo-pod formation and dynamics and actin flow was investigated.2022-01-2