Feature based estimation of myocardial motion from tagged MR images

In the past few years we witnessed an increase in mortality due to cancer relative to mortality due to cardiovascular diseases. In 2008, the Netherlands Statistics Agency reports that 33.900 people died of cancer against 33.100 deaths due to cardiovascular diseases, making cancer the number one cause of death in the Netherlands [33]. Even if the rate of people affected by heart diseases is continually rising, they "simply don’t die of it", according to the research director Prof. Mat Daemen of research institute CARIM of the University of Maastricht [50]. The reason for this is the early diagnosis, and the treatment of people with identified risk factors for diseases like ischemic heart disease, hypertrophic cardiomyopathy, thoracic aortic disease, pericardial (sac around the heart) disease, cardiac tumors, pulmonary artery disease, valvular disease, and congenital heart disease before and after surgical repair. Cardiac imaging plays a crucial role in the early diagnosis, since it allows the accurate investigation of a large amount of imaging data in a small amount of time. Moreover, cardiac imaging reduces costs of inpatient care, as has been shown in recent studies [77]. With this in mind, in this work we have provided several tools with the aim to help the investigation of the cardiac motion. In chapters 2 and 3 we have explored a novel variational optic flow methodology based on multi-scale feature points to extract cardiac motion from tagged MR images. Compared to constant brightness methods, this new approach exhibits several advantages. Although the intensity of critical points is also influenced by fading, critical points do retain their characteristic even in the presence of intensity changes, such as in MR imaging. In an experiment in section 5.4 we have applied this optic flow approach directly on tagged MR images. A visual inspection confirmed that the extracted motion fields realistically depicted the cardiac wall motion. The method exploits also the advantages from the multiscale framework. Because sparse velocity formulas 2.9, 3.7, 6.21, and 7.5 provide a number of equations equal to the number of unknowns, the method does not suffer from the aperture problem in retrieving velocities associated to the critical points. In chapters 2 and 3 we have moreover introduced a smoothness component of the optic flow equation described by means of covariant derivatives. This is a novelty in the optic flow literature. Many variational optic flow methods present a smoothness component that penalizes for changes from global assumptions such as isotropic or anisotropic smoothness. In the smoothness term proposed deviations from a predefined motion model are penalized. Moreover, the proposed optic flow equation has been decomposed in rotation-free and divergence-free components. This decomposition allows independent tuning of the two components during the vector field reconstruction. The experiments and the Table of errors provided in 3.8 showed that the combination of the smoothness term, influenced by a predefined motion model, and the Helmholtz decomposition in the optic flow equation reduces the average angular error substantially (20%-25%) with respect to a similar technique that employs only standard derivatives in the smoothness term. In section 5.3 we extracted the motion field of a phantom of which we know the ground truth of and compared the performance of this optic flow method with the performance of other optic flow methods well known in the literature, such as the Horn and Schunck [76] approach, the Lucas and Kanade [111] technique and the tuple image multi-scale optic flow constraint equation of Van Assen et al. [163]. Tests showed that the proposed optic flow methodology provides the smallest average angular error (AAE = 3.84 degrees) and L2 norm = 0.1. In this work we employed the Helmholtz decomposition also to study the cardiac behavior, since the vector field decomposition allows to investigate cardiac contraction and cardiac rotation independently. In chapter 4 we carried out an analysis of cardiac motion of ten volunteers and one patient where we estimated the kinetic energy for the different components. This decomposition is useful since it allows to visualize and quantify the contributions of each single vector field component to the heart beat. Local measurements of the kinetic energy have also been used to detect areas of the cardiac walls with little movement. Experiments on a patient and a comparison between a late enhancement cardiac image and an illustration of the cardiac kinetic energy on a bull’s eye plot illustrated that a correspondence between an infarcted area and an area with very small kinetic energy exists. With the aim to extend in the future the proposed optic flow equation to a 3D approach, in chapter 6 we investigated the 3D winding number approach as a tool to locate critical points in volume images. We simplified the mathematics involved with respect to a previous work [150] and we provided several examples and applications such as cardiac motion estimation from 3-dimensional tagged images, follicle and neuronal cell counting. Finally in chapter 7 we continued our investigation on volume tagged MR images, by retrieving the cardiac motion field using a 3-dimensional and simple version of the proposed optic flow equation based on standard derivatives. We showed that the retrieved motion fields display the contracting and rotating behavior of the cardiac muscle. We moreover extracted the through-plane component, which provides a realistic illustration of the vector field and is missed by 2-dimensional approaches

MR imaging of left-ventricular function : novel image acquisition and analysis techniques.

Many cardiac diseases, such as myocardial ischemia, secondary to coronary artery disease, may be identified and localized through the analysis of cardiac deformations. Early efforts for quantifying ventricular wall motion used surgical implantation and tracking of radiopaque markers with X-ray imaging in canine hearts [1]. Such techniques are invasive and affect the regional motion pattern of the ventricular wall during the marker tracking process and, clearly are not feasible clinically. Noninvasive imaging techniques are vital and have been widely applied to the clinic. MRI is a noninvasive imaging technique with the capability to monitor and assess the progression of cardiovascular diseases (CVD) so that effective procedures for the care and treatment of patients can be developed by physicians and researchers. It is capable of providing 3D analysis of global and regional cardiac function with great accuracy and reproducibility. In the past few years, numerous efforts have been devoted to cardiac motion recovery and deformation analysis from MR imaging sequences. In order to assess cardiac function, there are two categories of indices that are used: global and regional indices. Global indices include ejection fraction, cavity volume, and myocardial mass [2]. They are important indices for cardiac disease diagnosis. However, these global indices are not specific for regional analysis. A quantitative assessment of regional parameters may prove beneficial for the diagnosis of disease and evaluation of severity and the quantification of treatment [3]. Local measures, such as wall deformation and strain in all regions of the heart, can provide objective regional quantification of ventricular wall function and relate to the location and extent of ischemic injury. This dissertation is concerned with the development of novel MR imaging techniques and image postprocessing algorithms to analyze left ventricular deformations. A novel pulse sequence, termed Orthogonal CSPAMM (OCSPAMM), has been proposed which results in the same acquisition time as SPAMM for 2D deformation estimation while keeping the main advantages of CSPAMM [4,5]: i.e., maintaining tag contrast through-out the ECG cycle. Different from CSPAMM, in OCSPAMM the second tagging pulse orientation is rotated 90 degrees relative to the first one so that motion information can be obtained simultaneously in two directions. This reduces the acquisition time by a factor of two as compared to the traditional CSPAMM, in which two separate imaging sequences are applied per acquisition. With the application of OCSPAMM, the effect of tag fading encountered in SPAMM tagging due to Tl relaxation is mitigated and tag deformations can be visualized for the entire cardiac cycle, including diastolic phases. A multilevel B-spline fitting method (MBS) has been proposed which incorporates phase-based displacement information for accurate calculation of 2D motion and strain from tagged MRI [6, 7]. The proposed method combines the advantages of continuity and smoothness of MBS, and makes use of phase information derived from tagged MR images. Compared to previous 2D B-spline-based deformation analysis methods, MBS has the following advantages: 1) It can simultaneously achieve a smooth deformation while accurately approximating the given data set; 2) Computationally, it is very fast; and 3) It can produce more accurate deformation results. Since the tag intersections (intersections between two tag lines) can be extracted accurately and are more or less distributed evenly over the myocardium, MBS has proven effective for 2D cardiac motion tracking. To derive phase-based displacements, 2D HARP and SinMod analysis techniques [8,9] were employed. By producing virtual tags from HARP /SinMod and calculating intersections of virtual tag lines, more data points are obtained. In the reference frame, virtual tag lines are the isoparametric curves of an undeformed 2D B-spline model. In subsequent frames, the locations of intersections of virtual tag lines over the myocardium are updated with phase-based displacement. The advantage of the technique is that in acquiring denser myocardial displacements, it uses both real and virtual tag line intersections. It is fast and more accurate than 2D HARP and SinMod tracking. A novel 3D sine wave modeling (3D SinMod) approach for automatic analysis of 3D cardiac deformations has been proposed [10]. An accelerated 3D complementary spatial modulation of magnetization (CSPAMM) tagging technique [11] was used to acquire complete 3D+t tagged MR data sets of the whole heart (3 dynamic CSPAMM tagged MRI volume with tags in different orientations), in-vivo, in 54 heart beats and within 3 breath-holds. In 3D SinMod, the intensity distribution around each pixel is modeled as a cosine wave front. The principle behind 3D SinMod tracking is that both phase and frequency for each voxel are determined directly from the frequency analysis and the displacement is calculated from the quotient of phase difference and local frequency. The deformation fields clearly demonstrate longitudinal shortening during systole. The contraction of the LV base towards the apex as well as the torsional motion between basal and apical slices is clearly observable from the displacements. 3D SinMod can automatically process the image data to derive measures of motion, deformations, and strains between consecutive pair of tagged volumes in 17 seconds. Therefore, comprehensive 4D imaging and postprocessing for determination of ventricular function is now possible in under 10 minutes. For validation of 3D SinMod, 7 3D+t CSPAMM data sets of healthy subjects have been processed. Comparison of mid-wall contour deformations and circumferential shortening results by 3D SinMod showed good agreement with those by 3D HARP. Tag lines tracked by the proposed technique were also compared with manually delineated ones. The average errors calculated for the systolic phase of the cardiac cycles were in the sub-pixel range

Motion tracking tMRI datasets to quantify abnormal left ventricle motion using finite element modelling

According to `The Atlas of Heart Disease and Stroke'[MMMG04] published by the World Health Organization, heart disease accounts for nearly half the deaths in both the developed and developing countries and is the world's single biggest killer. However, early detection of a diseased heart condition can prevent many of these fatalities. Regional wall motion abnormalities of the heart precede both ECG abnormalities and chest pain as an indicator of myocardial ischaemia and are an excellent indicator of coronary stenosis [GZM97]. These motion abnormalities of the heart muscle are difficult to observe and track, because the heart is a relatively smooth organ with few landmarks and non-rigid motion with a twisting motion or tangential component. The MRI tissue-tagging technique gives researchers the first glimpse into how the heart actually beats. This research uses the tagged MRI images of the heart to create a three dimensional model of a beating heart indicating the stress of a region. Tagged MRI techniques are still developing and vary vastly, meaning that there needs to be a methodology that can adapt to these changes rapidly and effectively, to meet the needs of the evolving technology. The focus of this research is to develop and test such a methodology by the means of a Strain Estimation Pipeline along with an effective way of validating any changes made to the individual processes that it comprises of

Hierarchical template matching for 3D myocardial tracking and cardiac strain estimation

Myocardial tracking and strain estimation can non-invasively assess cardiac functioning using subject-specific MRI. As the left-ventricle does not have a uniform shape and functioning from base to apex, the development of 3D MRI has provided opportunities for simultaneous 3D tracking, and 3D strain estimation. We have extended a Local Weighted Mean (LWM) transformation function for 3D, and incorporated in a Hierarchical Template Matching model to solve 3D myocardial tracking and strain estimation problem. The LWM does not need to solve a large system of equations, provides smooth displacement of myocardial points, and adapt local geometric differences in images. Hence, 3D myocardial tracking can be performed with 1.49 mm median error, and without large error outliers. The maximum error of tracking is up to 24% reduced compared to benchmark methods. Moreover, the estimated strain can be insightful to improve 3D imaging protocols, and the computer code of LWM could also be useful for geo-spatial and manufacturing image analysis researchers

Three Dimensional Tissue Motion Analysis from Tagged Magnetic Resonance Imaging

Motion estimation of soft tissues during organ deformation has been an important topic in medical imaging studies. Its application involves a variety of internal and external organs including the heart, the lung, the brain, and the tongue. Tagged magnetic resonance imaging has been used for decades to observe and quantify motion and strain of deforming tissues. It places temporary noninvasive markers—so called "tags"—in the tissue of interest that deform together with the tissue during motion, producing images that carry motion information in the deformed tagged patterns. These images can later be processed using phase-extraction algorithms to achieve motion estimation and strain computation. In this dissertation, we study three-dimensional (3D) motion estimation and analysis using tagged magnetic resonance images with applications focused on speech studies and traumatic brain injury modeling. Novel algorithms are developed to assist tagged motion analysis. Firstly, a pipeline of methods—TMAP—is proposed to compute 3D motion from tagged and cine images of the tongue during speech. TMAP produces an estimation of motion along with a multi-subject analysis of motion pattern differences between healthy control subjects and post-glossectomy patients. Secondly, an enhanced 3D motion estimation algorithm—E-IDEA—is proposed. E-IDEA tackles the incompressible motion both on the internal tissue region and the tissue boundaries, reducing the boundary errors and yielding a motion estimate that is more accurate overall. Thirdly, a novel 3D motion estimation algorithm—PVIRA—is developed. Based on image registration and tracking, PVIRA is a faster and more robust method that performs phase extraction in a novel way. Lastly, a method to reveal muscles' activity using strain in the line of action of muscle fiber directions is presented. It is a first step toward relating motion production with individual muscles and provides a new tool for future clinical and scientific use

Role of deep learning techniques in non-invasive diagnosis of human diseases.

Machine learning, a sub-discipline in the domain of artificial intelligence, concentrates on algorithms able to learn and/or adapt their structure (e.g., parameters) based on a set of observed data. The adaptation is performed by optimizing over a cost function. Machine learning obtained a great attention in the biomedical community because it offers a promise for improving sensitivity and/or specificity of detection and diagnosis of diseases. It also can increase objectivity of the decision making, decrease the time and effort on health care professionals during the process of disease detection and diagnosis. The potential impact of machine learning is greater than ever due to the increase in medical data being acquired, the presence of novel modalities being developed and the complexity of medical data. In all of these scenarios, machine learning can come up with new tools for interpreting the complex datasets that confront clinicians. Much of the excitement for the application of machine learning to biomedical research comes from the development of deep learning which is modeled after computation in the brain. Deep learning can help in attaining insights that would be impossible to obtain through manual analysis. Deep learning algorithms and in particular convolutional neural networks are different from traditional machine learning approaches. Deep learning algorithms are known by their ability to learn complex representations to enhance pattern recognition from raw data. On the other hand, traditional machine learning requires human engineering and domain expertise to design feature extractors and structure data. With increasing demands upon current radiologists, there are growing needs for automating the diagnosis. This is a concern that deep learning is able to address. In this dissertation, we present four different successful applications of deep learning for diseases diagnosis. All the work presented in the dissertation utilizes medical images. In the first application, we introduce a deep-learning based computer-aided diagnostic system for the early detection of acute renal transplant rejection. The system is based on the fusion of both imaging markers (apparent diffusion coefficients derived from diffusion-weighted magnetic resonance imaging) and clinical biomarkers (creatinine clearance and serum plasma creatinine). The fused data is then used as an input to train and test a convolutional neural network based classifier. The proposed system is tested on scans collected from 56 subjects from geographically diverse populations and different scanner types/image collection protocols. The overall accuracy of the proposed system is 92.9% with 93.3% sensitivity and 92.3% specificity in distinguishing non-rejected kidney transplants from rejected ones. In the second application, we propose a novel deep learning approach for the automated segmentation and quantification of the LV from cardiac cine MR images. We aimed at achieving lower errors for the estimated heart parameters compared to the previous studies by proposing a novel deep learning segmentation method. Using fully convolutional neural networks, we proposed novel methods for the extraction of a region of interest that contains the left ventricle, and the segmentation of the left ventricle. Following myocardial segmentation, functional and mass parameters of the left ventricle are estimated. Automated Cardiac Diagnosis Challenge dataset was used to validate our framework, which gave better segmentation, accurate estimation of cardiac parameters, and produced less error compared to other methods applied on the same dataset. Furthermore, we showed that our segmentation approach generalizes well across different datasets by testing its performance on a locally acquired dataset. In the third application, we propose a novel deep learning approach for automated quantification of strain from cardiac cine MR images of mice. For strain analysis, we developed a Laplace-based approach to track the LV wall points by solving the Laplace equation between the LV contours of each two successive image frames over the cardiac cycle. Following tracking, the strain estimation is performed using the Lagrangian-based approach. This new automated system for strain analysis was validated by comparing the outcome of these analysis with the tagged MR images from the same mice. There were no significant differences between the strain data obtained from our algorithm using cine compared to tagged MR imaging. In the fourth application, we demonstrate how a deep learning approach can be utilized for the automated classification of kidney histopathological images. Our approach can classify four classes: the fat, the parenchyma, the clear cell renal cell carcinoma, and the unusual cancer which has been discovered recently, called clear cell papillary renal cell carcinoma. Our framework consists of three convolutional neural networks and the whole-slide kidney images were divided into patches with three different sizes to be inputted to the networks. Our approach can provide patch-wise and pixel-wise classification. Our approach classified the four classes accurately and surpassed other state-of-the-art methods such as ResNet (pixel accuracy: 0.89 Resnet18, 0.93 proposed). In conclusion, the results of our proposed systems demonstrate the potential of deep learning for the efficient, reproducible, fast, and affordable disease diagnosis

An image segmentation and registration approach to cardiac function analysis using MRI

Cardiovascular diseases (CVDs) are one of the major causes of death in the world. In recent years, significant progress has been made in the care and treatment of patients with such diseases. A crucial factor for this progress has been the development of magnetic resonance (MR) imaging which makes it possible to diagnose and assess the cardiovascular function of the patient. The ability to obtain high-resolution, cine volume images easily and safely has made it the preferred method for diagnosis of CVDs. MRI is also unique in its ability to introduce noninvasive markers directly into the tissue being imaged(MR tagging) during the image acquisition process. With the development of advanced MR imaging acquisition technologies, 3D MR imaging is more and more clinically feasible. This recent development has allowed new potentially 3D image analysis technologies to be deployed. However, quantitative analysis of cardiovascular system from the images remains a challenging topic. The work presented in this thesis describes the development of segmentation and motion analysis techniques for the study of the cardiac anatomy and function in cardiac magnetic resonance (CMR) images. The first main contribution of the thesis is the development of a fully automatic cardiac segmentation technique that integrates and combines a series of state-of-the-art techniques. The proposed segmentation technique is capable of generating an accurate 3D segmentation from multiple image sequences. The proposed segmentation technique is robust even in the presence of pathological changes, large anatomical shape variations and locally varying contrast in the images. Another main contribution of this thesis is the development of motion tracking techniques that can integrate motion information from different sources. For example, the radial motion of the myocardium can be tracked easily in untagged MR imaging since the epi- and endocardial surfaces are clearly visible. On the other hand, tagged MR imaging allows easy tracking of both longitudinal and circumferential motion. We propose a novel technique based on non-rigid image registration for the myocardial motion estimation using both untagged and 3D tagged MR images. The novel aspect of our technique is its simultaneous use of complementary information from both untagged and 3D tagged MR imaging. The similarity measure is spatially weighted to maximise the utility of information from both images. The thesis also proposes a sparse representation for free-form deformations (FFDs) using the principles of compressed sensing. The sparse free-form deformation (SFFD) model can capture fine local details such as motion discontinuities without sacrificing robustness. We demonstrate the capabilities of the proposed framework to accurately estimate smooth as well as discontinuous deformations in 2D and 3D CMR image sequences. Compared to the standard FFD approach, a significant increase in registration accuracy can be observed in datasets with discontinuous motion patterns. Both the segmentation and motion tracking techniques presented in this thesis have been applied to clinical studies. We focus on two important clinical applications that can be addressed by the techniques proposed in this thesis. The first clinical application aims at measuring longitudinal changes in cardiac morphology and function during the cardiac remodelling process. The second clinical application aims at selecting patients that positively respond to cardiac resynchronization therapy (CRT). The final chapter of this thesis summarises the main conclusions that can be drawn from the work presented here and also discusses possible avenues for future research

Anatomical Image Series Analysis in the Computational Anatomy Random Orbit Model

Serially acquired medical imagery plays an important role in the computational study of human anatomy. In this work, we describe the development of novel algorithms set in the large deformation diffeomorphic metric mapping framework for analyzing serially acquired imagery of two general types: spatial image series and temporal image series. In the former case, a critical step in the analysis of neural connectivity from serially-sectioned brain histology data is the reconstruction of spatially distorted image volumes and registration into a common coordinate space. In the latter case, computational methods are required for building low dimensional representations of the infinite dimensional shape space standard to computational anatomy. Here, we review the vast body of work related to volume reconstruction and atlas-mapping of serially-sectioned data as well as diffeomorphic methods for longitudinal data and we position our work relative to these in the context of the computational anatomy random orbit model. We show how these two problems are embedded as extensions to the classic random orbit model and use it to both enforce diffeomorphic conditions and analyze the distance metric associated to diffeomorphisms. We apply our new algorithms to histology and MRI datasets to study the structure, connectivity, and pathological degeneration of the brain

A Survey on Deep Learning in Medical Image Registration: New Technologies, Uncertainty, Evaluation Metrics, and Beyond

Over the past decade, deep learning technologies have greatly advanced the field of medical image registration. The initial developments, such as ResNet-based and U-Net-based networks, laid the groundwork for deep learning-driven image registration. Subsequent progress has been made in various aspects of deep learning-based registration, including similarity measures, deformation regularizations, and uncertainty estimation. These advancements have not only enriched the field of deformable image registration but have also facilitated its application in a wide range of tasks, including atlas construction, multi-atlas segmentation, motion estimation, and 2D-3D registration. In this paper, we present a comprehensive overview of the most recent advancements in deep learning-based image registration. We begin with a concise introduction to the core concepts of deep learning-based image registration. Then, we delve into innovative network architectures, loss functions specific to registration, and methods for estimating registration uncertainty. Additionally, this paper explores appropriate evaluation metrics for assessing the performance of deep learning models in registration tasks. Finally, we highlight the practical applications of these novel techniques in medical imaging and discuss the future prospects of deep learning-based image registration