Point-Spread-Function-Aware Slice-to-Volume Registration: Application to Upper Abdominal MRI Super-Resolution

MR image acquisition of moving organs remains challenging despite the advances in ultra-fast 2D MRI sequences. Post-acquisition techniques have been proposed to increase spatial resolution a posteriori by combining acquired orthogonal stacks into a single, high-resolution (HR) volume. Current super-resolution techniques classically rely on a two-step procedure. The volumetric reconstruction step leverages a physical slice acquisition model. However, the motion correction step typically neglects the point spread function (PSF) information. In this paper, we propose a PSF-aware slice-to-volume registration approach and, for the first time, demonstrate the potential benefit of Super-Resolution for upper abdominal imaging. Our novel reconstruction pipeline takes advantage of different MR acquisitions clinically used in routine MR cholangiopancreatography studies to guide the registration. On evaluation of clinically relevant image information, our approach outperforms state-of-the-art reconstruction toolkits in terms of visual clarity and preservation of raw data information. Overall, we achieve promising results towards replacing currently required CT scans

Computational Imaging Methods for Improving Resolution in Biological Microscopy

Optical microscopy is an essential tool for biological research, as it allows for non-invasive imaging of small animals. However, optical microscopy has its limits. Due to the low light level, fluorescence microscopy prohibits high speed imaging, making it difficult to study fast dynamic biological processes. In addition, optical blur due to the diffraction of light results in limited spatial resolution, particularly when using objective lenses with low numerical apertures. In this thesis, we propose computational imaging methods to overcome these limitations using a combination of novel image acquisition procedures and reconstruction algorithms.The first part of this thesis deals with improving temporal resolution in fluorescence microscopy to image rapid, repeating processes. We take advantage of multiple acquisitions, each taken with different time delays or temporally modulated illumination patterns, to recover high frequency information that is lost with traditional imaging. We demonstrate our method to image the beating heart in live embryonic zebrafish with reduced motion blur and high resolution in time.The second part of this thesis deals with reducing spatial blur in optical projection tomography, a form of optical microscopy that uses multiple 2D projections to reconstruct a 3D image of an object. We propose a method to reduce the optical distortion (as characterized by the system's optical point spread function) that can be implemented with a scanning acquisition approach combined with a modified filtered backprojection algorithm for reconstruction. We demonstrate our method to image blood vessels in larval zebrafish with high spatial resolution and reduced out-of-focus blur.The final part of this thesis deals with the dimensional limitation of 2D sensors for measuring 3D motion in microscopy. We propose a method to combine two-dimensional motion estimates from multiple views to recover out-of-plane velocity and reconstruct a divergence-free, three-dimensional velocity field. We demonstrate our method to measure, for the first time, dynamic blood flow in 3D inside the beating heart of a live zebrafish using optical microscopy.This thesis provides new tools that integrate custom image acquisition procedures and image reconstruction algorithms to overcome the resolution limitations -- temporal, spatial, and out-of-plane velocity resolution -- in optical microscopy. The methods presented in this thesis, in particular the single camera, active illumination method for temporal superresolution in fluorescence microscopy, will be directly applicable to a broad range of biological studies and will open up new perspectives for imaging small organisms in 3D (and time) with high spatio-temporal resolution

Filter-Based Probabilistic Markov Random Field Image Priors: Learning, Evaluation, and Image Analysis

Markov random fields (MRF) based on linear filter responses are one of the most popular forms for modeling image priors due to their rigorous probabilistic interpretations and versatility in various applications. In this dissertation, we propose an application-independent method to quantitatively evaluate MRF image priors using model samples. To this end, we developed an efficient auxiliary-variable Gibbs samplers for a general class of MRFs with flexible potentials. We found that the popular pairwise and high-order MRF priors capture image statistics quite roughly and exhibit poor generative properties. We further developed new learning strategies and obtained high-order MRFs that well capture the statistics of the inbuilt features, thus being real maximum-entropy models, and other important statistical properties of natural images, outlining the capabilities of MRFs. We suggest a multi-modal extension of MRF potentials which not only allows to train more expressive priors, but also helps to reveal more insights of MRF variants, based on which we are able to train compact, fully-convolutional restricted Boltzmann machines (RBM) that can model visual repetitive textures even better than more complex and deep models. The learned high-order MRFs allow us to develop new methods for various real-world image analysis problems. For denoising of natural images and deconvolution of microscopy images, the MRF priors are employed in a pure generative setting. We propose efficient sampling-based methods to infer Bayesian minimum mean squared error (MMSE) estimates, which substantially outperform maximum a-posteriori (MAP) estimates and can compete with state-of-the-art discriminative methods. For non-rigid registration of live cell nuclei in time-lapse microscopy images, we propose a global optical flow-based method. The statistics of noise in fluorescence microscopy images are studied to derive an adaptive weighting scheme for increasing model robustness. High-order MRFs are also employed to train image filters for extracting important features of cell nuclei and the deformation of nuclei are then estimated in the learned feature spaces. The developed method outperforms previous approaches in terms of both registration accuracy and computational efficiency

Spatio-Temporal Reconstruction Techniques for Optical Microscopy

Optical microscopy offers the unique possibility to study living samples under conditions akin to their native state. However, the technique is not void of inherent problems such as optical blur due to light diffraction, contamination with out-of-focus light from adjacent focal planes, and spherical aberrations. Furthermore, with a dearth of techniques that are capable of imaging multiple focal sections in quick succession, the multi-dimensional capture of dynamically changing samples remains a challenge of its own. Computational techniques that use auxiliary knowledge about the imaging system and the sample to mitigate these problems are hence of great interest in optical microscopy.The first part of this thesis deals with the design of a discrete model to characterize light propagation. Following the scalar diffraction theory in optics, we propose a discrete algorithm, based on generalized sampling theory, to reverse the coherent diffraction process via back propagation. The algorithm consists of a wavelet-based model for the spherical waves emanating from the object of interest and an optimized multi-rate filtering protocol for reconstruction from the diffraction data recorded by non-ideal detectors. The second part of this thesis describes a spatial registration tool designed for multi-view microscopy. Here, the imaged sample is rotated about a lateral axis for the acquisition of multiple 3D datasets from different views in order to subsequently alleviate the severe axial blur found in each such dataset. Automatic algorithms that only rely on maximizing pixel-based similarity provide poor results in such applications owing to the anisotropic point-spread-function (PSF) of optical microscopes. We propose a pyramid-based spatial registration algorithm that re-blurs the multi-view datasets with transformed forms of the PSF in order to make them comparable, before maximizing their pixel-based similarity for registration.The third part of this thesis describes a fast converging iterative multi-view deconvolution technique that can be applied to the spatially registered forms of the 3D datasets acquired using multi-view microscopy. Our sparsity based algorithm solves a non-linear objective function to jointly deconvolve and fuse the multi-view datasets to finally produce a single deblurred 3D result that has nearly isotropic spatial resolution.The fourth part of this thesis addresses problems due to spherical aberrations encountered during the imaging of thick samples in optical microscopy. The depth-varying nature of the optical blur found in such cases renders fast and efficient shift-invariant deconvolution techniques to be inapplicable. Here, we propose a fast iterative-shrinkage-thresholding shift-variant 3D deconvolution method that uses depth-dependent PSFs to reconstruct a 3D deblurred form of the imaged thick specimen. The final part of this thesis describes a non-rigid temporal registration tool that aids in the multi-dimensional imaging of quasi-periodic processes such as cardiac cycles. We propose a variant of dynamic time warping that is capable of both temporally warping and wrapping an input sequence by allowing for jump discontinuities in the non-linear temporal alignment function akin to those found in wrapped phase functions. This work provides a new set of tools for spatio-temporal reconstruction in optical microscopy and we anticipate them to be useful for a wide range of problems in practice

Motion compensated micro-CT reconstruction for in-situ analysis of dynamic processes

This work presents a framework to exploit the synergy between Digital Volume Correlation ( DVC) and iterative CT reconstruction to enhance the quality of high-resolution dynamic X-ray CT (4D-mu CT) and obtain quantitative results from the acquired dataset in the form of 3D strain maps which can be directly correlated to the material properties. Furthermore, we show that the developed framework is capable of strongly reducing motion artifacts even in a dataset containing a single 360 degrees rotation

Development and application of fluorescence lifetime imaging and super-resolution microscopy

This PhD thesis reports the development and application of fluorescence imaging technologies for studying biological processes on spatial scales below the diffraction limit. Two strategies were addressed: firstly fluorescence lifetime imaging (FLIM) to study molecular processes, e.g. using Förster resonance energy transfer (FRET) to read out protein interactions, and secondly direct imaging of nanostructure using super-resolution microscopy (SRM). For quantitative FRET readouts, the development and characterisation of an automated multiwell plate FLIM microscope for high content analysis (HCA) is described. Open source software was developed for the data acquisition and analysis, and approaches to improve the performance of time-gated imaging for FLIM were evaluated including different methods to despeckle the laser illumination and testing of an enhanced detector. This instrument was evaluated using standard fluorescent dye samples and cells expressing fluorescent protein-based FRET constructs. It was applied to an assay of live cells expressing a FRET biosensor and to FRET readouts of aggregation of a membrane receptor (DDR1) in fixed cells. A novel instrument, combining structured illumination microscopy (SIM) with FLIM, was developed to explore the combination of SRM and FLIM-FRET readouts. This enabled the simultaneous mapping of molecular readouts with FLIM and super-resolved imaging. The SIM+FLIM system was applied to image collagen-stimulated DDR1 aggregation in cells, to image DNA structures during the cell cycle and to explore interactions between cell organelles. A novel SRM approach based on a stimulated emission of depletion (STED) microscope incorporating a spatial light modulator (SLM) was developed to provide straightforward robust alignment with collinear excitation/depletion beams, aberration correction, an extended field of view and multiple beam scanning for faster STED image acquisition. The performance of easySLM-STED was evaluated by imaging bead samples, labelled vimentin in Vero cells and the synaptonemal complex in homologs of C. elegans germlines.Open Acces

Development of a test setup for the characterization of an optical microscope for high precision length metrology applications

A test setup to qualify the performance of optical microscopes has been designed and optimized using FEM calculations to exhibit a minimal susceptibility to thermal and mechanical influences of the ambient environment. The alignment is performed using an alignment autocollimator and alignment targets. The data acquisition of the camera and the position sensors of the stage is synchronized. The short-term repeatability (1s) of the line position and -width measurement obtained with the integrated UV microscope are 1 nm and 0.2 nm respectively. In long-term measurements the maximum lateral and focus drift rate observed were 30- and 20 nm / hour respectively. The measured point spread function contained only radial symmetric optical aberrations. Using the Zernike-Nijboer theory including only the defocus and spherical aberrations, fit residuals were obtained that contain systematic deviations in the order of the noise level

Multi-scale data fusion for surface metrology

The major trends in manufacturing are miniaturization, convergence of the traditional research fields and creation of interdisciplinary research areas. These trends have resulted in the development of multi-scale models and multi-scale surfaces to optimize the performance. Multi-scale surfaces that exhibit specific properties at different scales for a specific purpose require multi-scale measurement and characterization. Researchers and instrument developers have developed instruments that are able to perform measurements at multiple scales but lack the much required multi- scale characterization capability. The primary focus of this research was to explore possible multi-scale data fusion strategies and options for surface metrology domain and to develop enabling software tools in order to obtain effective multi-scale surface characterization, maximizing fidelity while minimizing measurement cost and time. This research effort explored the fusion strategies for surface metrology domain and narrowed the focus on Discrete Wavelet Frame (DWF) based multi-scale decomposition. An optimized multi-scale data fusion strategy ‘FWR method’ was developed and was successfully demonstrated on both high aspect ratio surfaces and non-planar surfaces. It was demonstrated that the datum features can be effectively characterized at a lower resolution using one system (Vision CMM) and the actual features of interest could be characterized at a higher resolution using another system (Coherence Scanning Interferometer) with higher capability while minimizing the measurement time