Rapid three-dimensional multiparametric MRI with quantitative transient-state imaging

Novel methods for quantitative, transient-state multiparametric imaging are increasingly being demonstrated for assessment of disease and treatment efficacy. Here, we build on these by assessing the most common Non-Cartesian readout trajectories (2D/3D radials and spirals), demonstrating efficient anti-aliasing with a k-space view-sharing technique, and proposing novel methods for parameter inference with neural networks that incorporate the estimation of proton density. Our results show good agreement with gold standard and phantom references for all readout trajectories at 1.5T and 3T. Parameters inferred with the neural network were within 6.58% difference from the parameters inferred with a high-resolution dictionary. Concordance correlation coefficients were above 0.92 and the normalized root mean squared error ranged between 4.2% - 12.7% with respect to gold-standard phantom references for T1 and T2. In vivo acquisitions demonstrate sub-millimetric isotropic resolution in under five minutes with reconstruction and inference times < 7 minutes. Our 3D quantitative transient-state imaging approach could enable high-resolution multiparametric tissue quantification within clinically acceptable acquisition and reconstruction times.Comment: 43 pages, 12 Figures, 5 Table

Magnetic Resonance Fingerprinting Using a Residual Convolutional Neural Network

Conventional dictionary matching based MR Fingerprinting (MRF) reconstruction approaches suffer from time-consuming operations that map temporal MRF signals to quantitative tissue parameters. In this paper, we design a 1-D residual convolutional neural network to perform the signature-to-parameter mapping in order to improve inference speed and accuracy. In particular, a 1-D convolutional neural network with shortcuts, a.k.a skip connections, for residual learning is developed using a TensorFlow platform. To avoid the requirement for a large amount of MRF data, the designed network is trained on synthesized MRF data simulated with the Bloch equations and fast imaging with steady state precession (FISP) sequences. The proposed approach was validated on both synthetic data and phantom data generated from a healthy subject. The reconstruction performance demonstrates a significantly improved speed - only 1.6s for reconstructing a pair of T1/T2 maps of size 128 × 128 - 50× faster than the original dictionary matching based method. The better performance was also confirmed by improved signal to noise ratio (SNR) and reduced root mean square error (RMSE). Furthermore, it is more compact to store a network instead of a large dictionary

Is attention all you need in medical image analysis? A review

Medical imaging is a key component in clinical diagnosis, treatment planning and clinical trial design, accounting for almost 90% of all healthcare data. CNNs achieved performance gains in medical image analysis (MIA) over the last years. CNNs can efficiently model local pixel interactions and be trained on small-scale MI data. The main disadvantage of typical CNN models is that they ignore global pixel relationships within images, which limits their generalisation ability to understand out-of-distribution data with different 'global' information. The recent progress of Artificial Intelligence gave rise to Transformers, which can learn global relationships from data. However, full Transformer models need to be trained on large-scale data and involve tremendous computational complexity. Attention and Transformer compartments (Transf/Attention) which can well maintain properties for modelling global relationships, have been proposed as lighter alternatives of full Transformers. Recently, there is an increasing trend to co-pollinate complementary local-global properties from CNN and Transf/Attention architectures, which led to a new era of hybrid models. The past years have witnessed substantial growth in hybrid CNN-Transf/Attention models across diverse MIA problems. In this systematic review, we survey existing hybrid CNN-Transf/Attention models, review and unravel key architectural designs, analyse breakthroughs, and evaluate current and future opportunities as well as challenges. We also introduced a comprehensive analysis framework on generalisation opportunities of scientific and clinical impact, based on which new data-driven domain generalisation and adaptation methods can be stimulated

Reasoning with Uncertainty in Deep Learning for Safer Medical Image Computing

Deep learning is now ubiquitous in the research field of medical image computing. As such technologies progress towards clinical translation, the question of safety becomes critical. Once deployed, machine learning systems unavoidably face situations where the correct decision or prediction is ambiguous. However, the current methods disproportionately rely on deterministic algorithms, lacking a mechanism to represent and manipulate uncertainty. In safety-critical applications such as medical imaging, reasoning under uncertainty is crucial for developing a reliable decision making system. Probabilistic machine learning provides a natural framework to quantify the degree of uncertainty over different variables of interest, be it the prediction, the model parameters and structures, or the underlying data (images and labels). Probability distributions are used to represent all the uncertain unobserved quantities in a model and how they relate to the data, and probability theory is used as a language to compute and manipulate these distributions. In this thesis, we explore probabilistic modelling as a framework to integrate uncertainty information into deep learning models, and demonstrate its utility in various high-dimensional medical imaging applications. In the process, we make several fundamental enhancements to current methods. We categorise our contributions into three groups according to the types of uncertainties being modelled: (i) predictive; (ii) structural and (iii) human uncertainty. Firstly, we discuss the importance of quantifying predictive uncertainty and understanding its sources for developing a risk-averse and transparent medical image enhancement application. We demonstrate how a measure of predictive uncertainty can be used as a proxy for the predictive accuracy in the absence of ground-truths. Furthermore, assuming the structure of the model is flexible enough for the task, we introduce a way to decompose the predictive uncertainty into its orthogonal sources i.e. aleatoric and parameter uncertainty. We show the potential utility of such decoupling in providing a quantitative “explanations” into the model performance. Secondly, we introduce our recent attempts at learning model structures directly from data. One work proposes a method based on variational inference to learn a posterior distribution over connectivity structures within a neural network architecture for multi-task learning, and share some preliminary results in the MR-only radiotherapy planning application. Another work explores how the training algorithm of decision trees could be extended to grow the architecture of a neural network to adapt to the given availability of data and the complexity of the task. Lastly, we develop methods to model the “measurement noise” (e.g., biases and skill levels) of human annotators, and integrate this information into the learning process of the neural network classifier. In particular, we show that explicitly modelling the uncertainty involved in the annotation process not only leads to an improvement in robustness to label noise, but also yields useful insights into the patterns of errors that characterise individual experts

Anatomical Modeling of Cerebral Microvascular Structures: Application to Identify Biomarkers of Microstrokes

Les réseaux microvasculaires corticaux sont responsables du transport de l’oxygène et des substrats énergétiques vers les neurones. Ces réseaux réagissent dynamiquement aux demandes énergétiques lors d’une activation neuronale par le biais du couplage neurovasculaire. Afin d’élucider le rôle de la composante microvasculaire dans ce processus de couplage, l’utilisation de la modélisation in-formatique pourrait se révéler un élément clé. Cependant, la manque de méthodologies de calcul appropriées et entièrement automatisées pour modéliser et caractériser les réseaux microvasculaires reste l’un des principaux obstacles. Le développement d’une solution entièrement automatisée est donc important pour des explorations plus avancées, notamment pour quantifier l’impact des mal-formations vasculaires associées à de nombreuses maladies cérébrovasculaires. Une observation courante dans l’ensemble des troubles neurovasculaires est la formation de micro-blocages vascu-laires cérébraux (mAVC) dans les artérioles pénétrantes de la surface piale. De récents travaux ont démontré l’impact de ces événements microscopiques sur la fonction cérébrale. Par conséquent, il est d’une importance vitale de développer une approche non invasive et comparative pour identifier leur présence dans un cadre clinique. Dans cette thèse,un pipeline de traitement entièrement automatisé est proposé pour aborder le prob-lème de la modélisation anatomique microvasculaire. La méthode de modélisation consiste en un réseau de neurones entièrement convolutif pour segmenter les capillaires sanguins, un générateur de modèle de surface 3D et un algorithme de contraction de la géométrie pour produire des mod-èles graphiques vasculaires ne comportant pas de connections multiples. Une amélioration de ce pipeline est développée plus tard pour alléger l’exigence de maillage lors de la phase de représen-tation graphique. Un nouveau schéma permettant de générer un modèle de graphe est développé avec des exigences d’entrée assouplies et permettant de retenir les informations sur les rayons des vaisseaux. Il est inspiré de graphes géométriques déformants construits en respectant les morpholo-gies vasculaires au lieu de maillages de surface. Un mécanisme pour supprimer la structure initiale du graphe à chaque exécution est implémenté avec un critère de convergence pour arrêter le pro-cessus. Une phase de raffinement est introduite pour obtenir des modèles vasculaires finaux. La modélisation informatique développée est ensuite appliquée pour simuler les signatures IRM po-tentielles de mAVC, combinant le marquage de spin artériel (ASL) et l’imagerie multidirectionnelle pondérée en diffusion (DWI). L’hypothèse est basée sur des observations récentes démontrant une réorientation radiale de la microvascularisation dans la périphérie du mAVC lors de la récupéra-tion chez la souris. Des lits capillaires synthétiques, orientés aléatoirement et radialement, et des angiogrammes de tomographie par cohérence optique (OCT), acquis dans le cortex de souris (n = 5) avant et après l’induction d’une photothrombose ciblée, sont analysés. Les graphes vasculaires informatiques sont exploités dans un simulateur 3D Monte-Carlo pour caractériser la réponse par résonance magnétique (MR), tout en considérant les effets des perturbations du champ magnétique causées par la désoxyhémoglobine, et l’advection et la diffusion des spins nucléaires. Le pipeline graphique proposé est validé sur des angiographies synthétiques et réelles acquises avec différentes modalités d’imagerie. Comparé à d’autres méthodes effectuées dans le milieu de la recherche, les expériences indiquent que le schéma proposé produit des taux d’erreur géométriques et topologiques amoindris sur divers angiogrammes. L’évaluation confirme également l’efficacité de la méthode proposée en fournissant des modèles représentatifs qui capturent tous les aspects anatomiques des structures vasculaires. Ensuite, afin de trouver des signatures de mAVC basées sur le signal IRM, la modélisation vasculaire proposée est exploitée pour quantifier le rapport de perte de signal intravoxel minimal lors de l’application de plusieurs directions de gradient, à des paramètres de séquence variables avec et sans ASL. Avec l’ASL, les résultats démontrent une dif-férence significative (p <0,05) entre le signal calculé avant et 3 semaines après la photothrombose. La puissance statistique a encore augmenté (p <0,005) en utilisant des angiogrammes capturés à la semaine suivante. Sans ASL, aucun changement de signal significatif n’est trouvé. Des rapports plus élevés sont obtenus à des intensités de champ magnétique plus faibles (par exemple, B0 = 3) et une lecture TE plus courte (<16 ms). Cette étude suggère que les mAVC pourraient être carac-térisés par des séquences ASL-DWI, et fournirait les informations nécessaires pour les validations expérimentales postérieures et les futurs essais comparatifs.----------ABSTRACT Cortical microvascular networks are responsible for carrying the necessary oxygen and energy substrates to our neurons. These networks react to the dynamic energy demands during neuronal activation through the process of neurovascular coupling. A key element in elucidating the role of the microvascular component in the brain is through computational modeling. However, the lack of fully-automated computational frameworks to model and characterize these microvascular net-works remains one of the main obstacles. Developing a fully-automated solution is thus substantial for further explorations, especially to quantify the impact of cerebrovascular malformations associ-ated with many cerebrovascular diseases. A common pathogenic outcome in a set of neurovascular disorders is the formation of microstrokes, i.e., micro occlusions in penetrating arterioles descend-ing from the pial surface. Recent experiments have demonstrated the impact of these microscopic events on brain function. Hence, it is of vital importance to develop a non-invasive and translatable approach to identify their presence in a clinical setting. In this thesis, a fully automatic processing pipeline to address the problem of microvascular anatom-ical modeling is proposed. The modeling scheme consists of a fully-convolutional neural network to segment microvessels, a 3D surface model generator and a geometry contraction algorithm to produce vascular graphical models with a single connected component. An improvement on this pipeline is developed later to alleviate the requirement of water-tight surface meshes as inputs to the graphing phase. The novel graphing scheme works with relaxed input requirements and intrin-sically captures vessel radii information, based on deforming geometric graphs constructed within vascular boundaries instead of surface meshes. A mechanism to decimate the initial graph struc-ture at each run is formulated with a convergence criterion to stop the process. A refinement phase is introduced to obtain final vascular models. The developed computational modeling is then ap-plied to simulate potential MRI signatures of microstrokes, combining arterial spin labeling (ASL) and multi-directional diffusion-weighted imaging (DWI). The hypothesis is driven based on recent observations demonstrating a radial reorientation of microvasculature around the micro-infarction locus during recovery in mice. Synthetic capillary beds, randomly- and radially oriented, and op-tical coherence tomography (OCT) angiograms, acquired in the barrel cortex of mice (n=5) before and after inducing targeted photothrombosis, are analyzed. The computational vascular graphs are exploited within a 3D Monte-Carlo simulator to characterize the magnetic resonance (MR) re-sponse, encompassing the effects of magnetic field perturbations caused by deoxyhemoglobin, and the advection and diffusion of the nuclear spins. The proposed graphing pipeline is validated on both synthetic and real angiograms acquired with different imaging modalities. Compared to other efficient and state-of-the-art graphing schemes, the experiments indicate that the proposed scheme produces the lowest geometric and topological error rates on various angiograms. The evaluation also confirms the efficiency of the proposed scheme in providing representative models that capture all anatomical aspects of vascular struc-tures. Next, searching for MRI-based signatures of microstokes, the proposed vascular modeling is exploited to quantify the minimal intravoxel signal loss ratio when applying multiple gradient di-rections, at varying sequence parameters with and without ASL. With ASL, the results demonstrate a significant difference (p<0.05) between the signal-ratios computed at baseline and 3 weeks after photothrombosis. The statistical power further increased (p<0.005) using angiograms captured at week 4. Without ASL, no reliable signal change is found. Higher ratios with improved significance are achieved at low magnetic field strengths (e.g., at 3 Tesla) and shorter readout TE (<16 ms). This study suggests that microstrokes might be characterized through ASL-DWI sequences, and provides necessary insights for posterior experimental validations, and ultimately, future transla-tional trials

Intelligent Imaging of Perfusion Using Arterial Spin Labelling

Arterial spin labelling (ASL) is a powerful magnetic resonance imaging technique, which can be used to noninvasively measure perfusion in the brain and other organs of the body. Promising research results show how ASL might be used in stroke, tumours, dementia and paediatric medicine, in addition to many other areas. However, significant obstacles remain to prevent widespread use: ASL images have an inherently low signal to noise ratio, and are susceptible to corrupting artifacts from motion and other sources. The objective of the work in this thesis is to move towards an "intelligent imaging" paradigm: one in which the image acquisition, reconstruction and processing are mutually coupled, and tailored to the individual patient. This thesis explores how ASL images may be improved at several stages of the imaging pipeline. We review the relevant ASL literature, exploring details of ASL acquisitions, parameter inference and artifact post-processing. We subsequently present original work: we use the framework of Bayesian experimental design to generate optimised ASL acquisitions, we present original methods to improve parameter inference through anatomically-driven modelling of spatial correlation, and we describe a novel deep learning approach for simultaneous denoising and artifact filtering. Using a mixture of theoretical derivation, simulation results and imaging experiments, the work in this thesis presents several new approaches for ASL, and hopefully will shape future research and future ASL usage

Learning-based Algorithms for Inverse Problems in MR Image Reconstruction and Quantitative Perfusion Imaging

Medical imaging has become an integral part of the clinical pipeline through its widespread use in the diagnosis, prognosis and treatment planning of several diseases. Magnetic Resonance Imaging (MRI) is particularly useful because it is free from ionizing radiation and is able to provide excellent soft tissue contrast. However, MRI suffers from drawbacks like long scanning durations that increase the cost of imaging and render the acquired images vulnerable to artifacts like motion. In modalities like Arterial Spin Labeling (ASL), which is used for non-invasive and quantitative perfusion imaging, low signal-to-noise ratio and lack of precision in parameter estimates also present significant problems. In this thesis, we develop and present algorithms whose focus can be divided into two broad categories. First, we investigate the reconstruction of MR images from fewer measurements, using data-driven machine learning to fill in the gaps in acquisition, thereby reducing the scan duration. Specifically, we first combine a supervised and an unsupervised (blind) learned dictionary in a residual fashion as a spatial prior in MR image reconstruction, and then extend this framework to include deep supervised learning. The latter, called blind primed supervised (BLIPS) learning, proposes that there exists synergy between features learned using shallower dictionary-based methods or traditional prior-based image reconstruction and those learned using newer deep supervised learning-based approaches. We show that this synergy can be exploited to yield reconstructions that are approx. 0.5-1 dB better in PSNR (in avg. across undersampling patterns). We also observe that the BLIPS algorithm is more robust to a scarcity of available training data, yielding reconstructions that are approx. 0.8 dB better (in terms of avg. PSNR) compared to strict supervised learning reconstruction when training data is very limited. Secondly, we aim to provide more precise estimates for multiple physiological parameters and tissue properties from ASL scans by estimation-theory-based optimization of ASL scan design, and combination with MR Fingerprinting. For this purpose, we use the Cramer-Rao Lower Bound (CRLB) for optimizing the scan design, and deep learning for regression-based estimation. We also show that regardless of the estimator used, optimization improves the precision in parameter estimates, and enables us to increase the available ‘useful’ information obtained in a fixed scanning duration. Specifically, we successfully improve the theoretical precision of perfusion estimates by 4.6% compared to a scan design where the repetition times are randomly chosen (a popular choice in literature) thereby yielding a 35.2% improvement in the corresponding RMSE in our in-silico experiments. This improvement is also visually evident in our in-vivo studies on healthy human subjects.PHDElectrical and Computer EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/169819/1/anishl_1.pd

Development of Chemokine Receptor 4 Targeted MRI Contrast Agent for the Precision MRI Imaging of Cancer and Metastasis

Liver metastases are often observed with primary cancers, including uveal melanoma (UM), breast, ovarian, and colon cancer. Chemokine receptor 4 (CXCR4) is one of the major chemokine receptors that overexpressed by cancers due to its role in tumor growth, migration, and progression. Organs such as the liver, bone, and brain have an intrinsically high concentration of CXCR4 ligand, C-X-C motif chemokine ligand 12 (CXCL12). As a result, CXCR4 is also often expressed at metastases in these organs. There is an unmet medical need for noninvasively early detecting, staging, and monitoring the prognosis of liver cancers and metastases as well as probing tumor microenvironment for target therapy and drug. In this dissertation, we first report the elevated expression of CXCR4 in the UM patients’ liver metastases and metastatic UM mouse models and validate its role as an imaging biomarker for liver metastases. We then report our achievement in the development of a CXCR4-targeted MRI contrast agent, ProCA32.CXCR4, for sensitive detection of UM liver metastases using MRI. ProCA32.CXCR4 exhibits high relaxivities (r1 = 30.9 mM-1 s-1, r2 = 43.2 mM-1 s-1, 1.5 T; r1 = 23.5 mM-1 s-1, r2 = 98.6 mM-1 s-1, 7.0 T), strong CXCR4 binding affinity (Kd = 1.1 ± 0.2 µM), and the capability of CXCR4 molecular imaging in both metastatic and intrahepatic xenotransplantation UM mouse models. ProCA32.CXCR4 enables robust detection of UM metastases as small as 0.1 mm2 in the liver. CXCR4 receptor blockage experiment proved the specific binding of ProCA32.CXCR4 to CXCR4. In addition, the application of ProCA32.CXCR4 in primary liver cancer detection and treatment effect monitoring are tested in hepatocellular carcinoma (HCC) mouse model. Except for gadolinium, we explored the possibility of manganese-based protein contrast agent and proved manganese-based ProCA exhibits high relaxivity and strong gadolinium binding affinity. We further designed and characterized ProCA32.RGD for integrin avb3 molecular imaging to reveal the tumor microenvironment. ProCA32.RGD targeted imaging was achieved in a 4T1 breast cancer mouse model. Further development of the biomarker targeted imaging agents is expected to have strong translation potential for early detection, surveillance, and treatment stratification of cancer and related diseases