Computational Imaging Methods for Improving Resolution in Biological Microscopy

Optical microscopy is an essential tool for biological research, as it allows for non-invasive imaging of small animals. However, optical microscopy has its limits. Due to the low light level, fluorescence microscopy prohibits high speed imaging, making it difficult to study fast dynamic biological processes. In addition, optical blur due to the diffraction of light results in limited spatial resolution, particularly when using objective lenses with low numerical apertures. In this thesis, we propose computational imaging methods to overcome these limitations using a combination of novel image acquisition procedures and reconstruction algorithms.The first part of this thesis deals with improving temporal resolution in fluorescence microscopy to image rapid, repeating processes. We take advantage of multiple acquisitions, each taken with different time delays or temporally modulated illumination patterns, to recover high frequency information that is lost with traditional imaging. We demonstrate our method to image the beating heart in live embryonic zebrafish with reduced motion blur and high resolution in time.The second part of this thesis deals with reducing spatial blur in optical projection tomography, a form of optical microscopy that uses multiple 2D projections to reconstruct a 3D image of an object. We propose a method to reduce the optical distortion (as characterized by the system's optical point spread function) that can be implemented with a scanning acquisition approach combined with a modified filtered backprojection algorithm for reconstruction. We demonstrate our method to image blood vessels in larval zebrafish with high spatial resolution and reduced out-of-focus blur.The final part of this thesis deals with the dimensional limitation of 2D sensors for measuring 3D motion in microscopy. We propose a method to combine two-dimensional motion estimates from multiple views to recover out-of-plane velocity and reconstruct a divergence-free, three-dimensional velocity field. We demonstrate our method to measure, for the first time, dynamic blood flow in 3D inside the beating heart of a live zebrafish using optical microscopy.This thesis provides new tools that integrate custom image acquisition procedures and image reconstruction algorithms to overcome the resolution limitations -- temporal, spatial, and out-of-plane velocity resolution -- in optical microscopy. The methods presented in this thesis, in particular the single camera, active illumination method for temporal superresolution in fluorescence microscopy, will be directly applicable to a broad range of biological studies and will open up new perspectives for imaging small organisms in 3D (and time) with high spatio-temporal resolution

U-Net and its variants for medical image segmentation: theory and applications

U-net is an image segmentation technique developed primarily for medical image analysis that can precisely segment images using a scarce amount of training data. These traits provide U-net with a very high utility within the medical imaging community and have resulted in extensive adoption of U-net as the primary tool for segmentation tasks in medical imaging. The success of U-net is evident in its widespread use in all major image modalities from CT scans and MRI to X-rays and microscopy. Furthermore, while U-net is largely a segmentation tool, there have been instances of the use of U-net in other applications. As the potential of U-net is still increasing, in this review we look at the various developments that have been made in the U-net architecture and provide observations on recent trends. We examine the various innovations that have been made in deep learning and discuss how these tools facilitate U-net. Furthermore, we look at image modalities and application areas where U-net has been applied.Comment: 42 pages, in IEEE Acces

Automated analysis of 3D echocardiography

In this thesis we aim at automating the analysis of 3D echocardiography, mainly targeting the functional analysis of the left ventricle. Manual analysis of these data is cumbersome, time-consuming and is associated with inter-observer and inter-institutional variability. Methods for reconstruction of 3D echocardiographic images from fast rotating ultrasound transducers is presented and methods for analysis of 3D echocardiography in general, using tracking, detection and model-based segmentation techniques to ultimately fully automatically segment the left ventricle for functional analysis. We show that reliable quantification of left ventricular volume and mitral valve displacement can be achieved using the presented techniques.SenterNovem (IOP Beeldverwerking, grant IBVC02003), Dutch Technology Foundation STW (grant 06666)UBL - phd migration 201

Production and characterization of genetically modified magnetized cells for future targeted treatment of cardiovascular diseases

Gene therapy is a promising tool for cardiovascular disease treatment. Its wide clinical application is, however, limited by the lack of suitable targeted gene delivery systems. In this work, a non-viral vector was tested - composed by polyethyleneimine and magnetic nanoparticles (PEI/MNP) - with primary endothelial cells as in vitro model for angiogenesis and CD133+ hematopoietic stem cells. The obtained results indicate that transient genetic modification of cells with miRNA delivered by PEI/MNP is feasible and results in their magnetization sufficient for targeting and MRI detection

U-net and its variants for medical image segmentation: A review of theory and applications

U-net is an image segmentation technique developed primarily for image segmentation tasks. These traits provide U-net with a high utility within the medical imaging community and have resulted in extensive adoption of U-net as the primary tool for segmentation tasks in medical imaging. The success of U-net is evident in its widespread use in nearly all major image modalities, from CT scans and MRI to Xrays and microscopy. Furthermore, while U-net is largely a segmentation tool, there have been instances of the use of U-net in other applications. Given that U-net’s potential is still increasing, this narrative literature review examines the numerous developments and breakthroughs in the U-net architecture and provides observations on recent trends. We also discuss the many innovations that have advanced in deep learning and discuss how these tools facilitate U-net. In addition, we review the different image modalities and application areas that have been enhanced by U-net

Cardiac motion estimation in ultrasound images using a sparse representation and dictionary learning

Les maladies cardiovasculaires sont de nos jours un problème de santé majeur. L'amélioration des méthodes liées au diagnostic de ces maladies représente donc un réel enjeu en cardiologie. Le coeur étant un organe en perpétuel mouvement, l'analyse du mouvement cardiaque est un élément clé pour le diagnostic. Par conséquent, les méthodes dédiées à l'estimation du mouvement cardiaque à partir d'images médicales, plus particulièrement en échocardiographie, font l'objet de nombreux travaux de recherches. Cependant, plusieurs difficultés liées à la complexité du mouvement du coeur ainsi qu'à la qualité des images échographiques restent à surmonter afin d'améliorer la qualité et la précision des estimations. Dans le domaine du traitement d'images, les méthodes basées sur l'apprentissage suscitent de plus en plus d'intérêt. Plus particulièrement, les représentations parcimonieuses et l'apprentissage de dictionnaires ont démontré leur efficacité pour la régularisation de divers problèmes inverses. Cette thèse a ainsi pour but d'explorer l'apport de ces méthodes, qui allient parcimonie et apprentissage, pour l'estimation du mouvement cardiaque. Trois principales contributions sont présentées, chacune traitant différents aspects et problématiques rencontrées dans le cadre de l'estimation du mouvement en échocardiographie. Dans un premier temps, une méthode d'estimation du mouvement cardiaque se basant sur une régularisation parcimonieuse est proposée. Le problème d'estimation du mouvement est formulé dans le cadre d'une minimisation d'énergie, dont le terme d'attache aux données est construit avec l'hypothèse d'un bruit de Rayleigh multiplicatif. Une étape d'apprentissage de dictionnaire permet une régularisation exploitant les propriétés parcimonieuses du mouvement cardiaque, combinée à un terme classique de lissage spatial. Dans un second temps, une méthode robuste de flux optique est présentée. L'objectif de cette approche est de robustifier la méthode d'estimation développée au premier chapitre de manière à la rendre moins sensible aux éléments aberrants. Deux régularisations sont mises en oeuvre, imposant d'une part un lissage spatial et de l'autre la parcimonie des champs de mouvements dans un dictionnaire approprié. Afin d'assurer la robustesse de la méthode vis-à-vis des anomalies, une stratégie de minimisation récursivement pondérée est proposée. Plus précisément, les fonctions employées pour cette pondération sont basées sur la théorie des M-estimateurs. Le dernier travail présenté dans cette thèse, explore une méthode d'estimation du mouvement cardiaque exploitant une régularisation parcimonieuse combinée à un lissage à la fois dans les domaines spatial et temporel. Le problème est formulé dans un cadre général d'estimation de flux optique. La régularisation temporelle proposée impose des trajectoires de mouvement lisses entre images consécutives. De plus, une méthode itérative d'estimation permet d'incorporer les trois termes de régularisations, tout en rendant possible le traitement simultané d'un ensemble d'images. Dans cette thèse, les contributions proposées sont validées en employant des images synthétiques et des simulations réalistes d'images ultrasonores. Ces données avec vérité terrain permettent d'évaluer la précision des approches considérées, et de souligner leur compétitivité par rapport à des méthodes de l'état-del'art. Pour démontrer la faisabilité clinique, des images in vivo de patients sains ou atteints de pathologies sont également considérées pour les deux premières méthodes. Pour la dernière contribution de cette thèse, i.e., exploitant un lissage temporel, une étude préliminaire est menée en utilisant des données de simulation.Cardiovascular diseases have become a major healthcare issue. Improving the diagnosis and analysis of these diseases have thus become a primary concern in cardiology. The heart is a moving organ that undergoes complex deformations. Therefore, the quantification of cardiac motion from medical images, particularly ultrasound, is a key part of the techniques used for diagnosis in clinical practice. Thus, significant research efforts have been directed toward developing new cardiac motion estimation methods. These methods aim at improving the quality and accuracy of the estimated motions. However, they are still facing many challenges due to the complexity of cardiac motion and the quality of ultrasound images. Recently, learning-based techniques have received a growing interest in the field of image processing. More specifically, sparse representations and dictionary learning strategies have shown their efficiency in regularizing different ill-posed inverse problems. This thesis investigates the benefits that such sparsity and learning-based techniques can bring to cardiac motion estimation. Three main contributions are presented, investigating different aspects and challenges that arise in echocardiography. Firstly, a method for cardiac motion estimation using a sparsity-based regularization is introduced. The motion estimation problem is formulated as an energy minimization, whose data fidelity term is built using the assumption that the images are corrupted by multiplicative Rayleigh noise. In addition to a classical spatial smoothness constraint, the proposed method exploits the sparse properties of the cardiac motion to regularize the solution via an appropriate dictionary learning step. Secondly, a fully robust optical flow method is proposed. The aim of this work is to take into account the limitations of ultrasound imaging and the violations of the regularization constraints. In this work, two regularization terms imposing spatial smoothness and sparsity of the motion field in an appropriate cardiac motion dictionary are also exploited. In order to ensure robustness to outliers, an iteratively re-weighted minimization strategy is proposed using weighting functions based on M-estimators. As a last contribution, we investigate a cardiac motion estimation method using a combination of sparse, spatial and temporal regularizations. The problem is formulated within a general optical flow framework. The proposed temporal regularization enforces smoothness of the motion trajectories between consecutive images. Furthermore, an iterative groupewise motion estimation allows us to incorporate the three regularization terms, while enabling the processing of the image sequence as a whole. Throughout this thesis, the proposed contributions are validated using synthetic and realistic simulated cardiac ultrasound images. These datasets with available groundtruth are used to evaluate the accuracy of the proposed approaches and show their competitiveness with state-of-the-art algorithms. In order to demonstrate clinical feasibility, in vivo sequences of healthy and pathological subjects are considered for the first two methods. A preliminary investigation is conducted for the last contribution, i.e., exploiting temporal smoothness, using simulated data

Novel Algorithms for Merging Computational Fluid Dynamics and 4D Flow MRI

Time-resolved three-dimensional spatial encoding combined with three-directional velocity-encoded phase contrast magnetic resonance imaging (termed as 4D flow MRI), can provide valuable information for diagnosis, treatment, and monitoring of vascular diseases. The accuracy of this technique, however, is limited by errors in flow estimation due to acquisition noise as well as systematic errors. Furthermore, available spatial resolution is limited to 1.5mm - 3mm and temporal resolution is limited to 30-40ms. This is often grossly inadequate to resolve flow details in small arteries, such as those in cerebral circulation. Recently, there have been efforts to address the limitations of the spatial and temporal resolution of MR flow imaging through the use of computational fluid dynamics (CFD). While CFD is capable of providing essentially unlimited spatial and temporal resolution, numerical results are very sensitive to errors in estimation of the flow boundary conditions. In this work, we present three novel techniques that combine CFD with 4D flow MRI measurements in order to address the resolution and noise issues. The first technique is a variant of the Kalman Filter state estimator called the Ensemble Kalman Filter (EnKF). In this technique, an ensemble of patient-specific CFD solutions are used to compute filter gains. These gains are then used in a predictor-corrector scheme to not only denoise the data but also increase its temporal and spatial resolution. The second technique is based on proper orthogonal decomposition and ridge regression (POD-rr). The POD method is typically used to generate reduced order models (ROMs) in closed control applications of large degree of freedom systems that result from discretization of governing partial differential equations (PDE). The POD-rr process results in a set of basis functions (vectors), that capture the local space of solutions of the PDE in question. In our application, the basis functions are generated from an ensemble of patient-specific CFD solutions whose boundary conditions are estimated from 4D flow MRI data. The CFD solution that should be most closely representing the actual flow is generated by projecting 4D flow MRI data onto the basis vectors followed by reconstruction in both MRI and CFD resolution. The rr algorithm was used for between resolution mapping. Despite the accuracy of using rr as the mapping step, due to manual adjustment of a coefficient in the algorithm we developed the third algorithm. In this step, the rr algorithm was substituded with a dynamic mode decomposition algorithm to preserve the robustness. These algorithms have been implemented and tested using a numerical model of the flow in a cerebral aneurysm. Solutions at time intervals corresponding to the 4D flow MRI temporal resolution were collected and downsampled to the spatial resolution of the imaging data. A simulated acquisition noise was then added in k-space. Finally, the simulated data affected by noise were used as an input to the merging algorithms. Rigorous comparison to state-of-the-art techniques were conducted to assess the accuracy and performance of the proposed method. The results provided denoised flow fields with less than 1\% overall error for different signal-to-noise ratios. At the end, a small cohort of three patients were corrected and the data were reconstructed using different methods, the wall shear stress (WSS) was calculated using different reconstructed data and the results were compared. As it has been shown in chapter 5, the calculated WSS using different methods results in mutual high and low shear stress regions, however, the exact value and patterns are significantly different

Volumetric MRI Reconstruction from 2D Slices in the Presence of Motion

Despite recent advances in acquisition techniques and reconstruction algorithms, magnetic resonance imaging (MRI) remains challenging in the presence of motion. To mitigate this, ultra-fast two-dimensional (2D) MRI sequences are often used in clinical practice to acquire thick, low-resolution (LR) 2D slices to reduce in-plane motion. The resulting stacks of thick 2D slices typically provide high-quality visualizations when viewed in the in-plane direction. However, the low spatial resolution in the through-plane direction in combination with motion commonly occurring between individual slice acquisitions gives rise to stacks with overall limited geometric integrity. In further consequence, an accurate and reliable diagnosis may be compromised when using such motion-corrupted, thick-slice MRI data. This thesis presents methods to volumetrically reconstruct geometrically consistent, high-resolution (HR) three-dimensional (3D) images from motion-corrupted, possibly sparse, low-resolution 2D MR slices. It focuses on volumetric reconstructions techniques using inverse problem formulations applicable to a broad field of clinical applications in which associated motion patterns are inherently different, but the use of thick-slice MR data is current clinical practice. In particular, volumetric reconstruction frameworks are developed based on slice-to-volume registration with inter-slice transformation regularization and robust, complete-outlier rejection for the reconstruction step that can either avoid or efficiently deal with potential slice-misregistrations. Additionally, this thesis describes efficient Forward-Backward Splitting schemes for image registration for any combination of differentiable (not necessarily convex) similarity measure and convex (not necessarily smooth) regularization with a tractable proximal operator. Experiments are performed on fetal and upper abdominal MRI, and on historical, printed brain MR films associated with a uniquely long-term study dating back to the 1980s. The results demonstrate the broad applicability of the presented frameworks to achieve robust reconstructions with the potential to improve disease diagnosis and patient management in clinical practice

Ultrasound Imaging Innovations for Visualization and Quantification of Vascular Biomarkers

The existence of plaque in the carotid arteries, which provide circulation to the brain, is a known risk for stroke and dementia. Alas, this risk factor is present in 25% of the adult population. Proper assessment of carotid plaque may play a significant role in preventing and managing stroke and dementia. However, current plaque assessment routines have known limitations in assessing individual risk for future cardiovascular events. There is a practical need to derive new vascular biomarkers that are indicative of cardiovascular risk based on hemodynamic information. Nonetheless, the derivation of these biomarkers is not a trivial technical task because none of the existing clinical imaging modalities have adequate time resolution to track the spatiotemporal dynamics of arterial blood flow that is pulsatile in nature. The goal of this dissertation is to devise a new ultrasound imaging framework to measure vascular biomarkers related to turbulent flow, intra-plaque microvasculature, and blood flow rate. Central to the proposed framework is the use of high frame rate ultrasound (HiFRUS) imaging principles to track hemodynamic events at fine temporal resolution (through using frame rates of greater than 1000 frames per second). The existence of turbulent flow and intra-plaque microvessels, as well as anomalous blood flow rate, are all closely related to the formation and progression of carotid plaque. Therefore, quantifying these biomarkers can improve the identification of individuals with carotid plaque who are at risk for future cardiovascular events. To facilitate the testing and the implementation of the proposed imaging algorithms, this dissertation has included the development of new experimental models (in the form of flow phantoms) and a new HiFRUS imaging platform with live scanning and on-demand playback functionalities. Pilot studies were also carried out on rats and human volunteers. Results generally demonstrated the real-time performance and the practical efficacy of the proposed algorithms. The proposed ultrasound imaging framework is expected to improve carotid plaque risk classification and, in turn, facilitate timely identification of at-risk individuals. It may also be used to derive new insights on carotid plaque formation and progression to aid disease management and the development of personalized treatment strategies