Pulmonary Image Segmentation and Registration Algorithms: Towards Regional Evaluation of Obstructive Lung Disease

Pulmonary imaging, including pulmonary magnetic resonance imaging (MRI) and computed tomography (CT), provides a way to sensitively and regionally measure spatially heterogeneous lung structural-functional abnormalities. These unique imaging biomarkers offer the potential for better understanding pulmonary disease mechanisms, monitoring disease progression and response to therapy, and developing novel treatments for improved patient care. To generate these regional lung structure-function measurements and enable broad clinical applications of quantitative pulmonary MRI and CT biomarkers, as a first step, accurate, reproducible and rapid lung segmentation and registration methods are required. In this regard, we first developed a 1H MRI lung segmentation algorithm that employs complementary hyperpolarized 3He MRI functional information for improved lung segmentation. The 1H-3He MRI joint segmentation algorithm was formulated as a coupled continuous min-cut model and solved through convex relaxation, for which a dual coupled continuous max-flow model was proposed and a max-flow-based efficient numerical solver was developed. Experimental results on a clinical dataset of 25 chronic obstructive pulmonary disease (COPD) patients ranging in disease severity demonstrated that the algorithm provided rapid lung segmentation with high accuracy, reproducibility and diminished user interaction. We then developed a general 1H MRI left-right lung segmentation approach by exploring the left-to-right lung volume proportion prior. The challenging volume proportion-constrained multi-region segmentation problem was approximated through convex relaxation and equivalently represented by a max-flow model with bounded flow conservation conditions. This gave rise to a multiplier-based high performance numerical implementation based on convex optimization theories. In 20 patients with mild- to-moderate and severe asthma, the approach demonstrated high agreement with manual segmentation, excellent reproducibility and computational efficiency. Finally, we developed a CT-3He MRI deformable registration approach that coupled the complementary CT-1H MRI registration. The joint registration problem was solved by exploring optical-flow techniques, primal-dual analyses and convex optimization theories. In a diverse group of patients with asthma and COPD, the registration approach demonstrated lower target registration error than single registration and provided fast regional lung structure-function measurements that were strongly correlated with a reference method. Collectively, these lung segmentation and registration algorithms demonstrated accuracy, reproducibility and workflow efficiency that all may be clinically-acceptable. All of this is consistent with the need for broad and large-scale clinical applications of pulmonary MRI and CT

Innovations in thoracic imaging:CT, radiomics, AI and x-ray velocimetry

In recent years, pulmonary imaging has seen enormous progress, with the introduction, validation and implementation of new hardware and software. There is a general trend from mere visual evaluation of radiological images to quantification of abnormalities and biomarkers, and assessment of 'non visual' markers that contribute to establishing diagnosis or prognosis. Important catalysts to these developments in thoracic imaging include new indications (like computed tomography [CT] lung cancer screening) and the COVID-19 pandemic. This review focuses on developments in CT, radiomics, artificial intelligence (AI) and x-ray velocimetry for imaging of the lungs. Recent developments in CT include the potential for ultra-low-dose CT imaging for lung nodules, and the advent of a new generation of CT systems based on photon-counting detector technology. Radiomics has demonstrated potential towards predictive and prognostic tasks particularly in lung cancer, previously not achievable by visual inspection by radiologists, exploiting high dimensional patterns (mostly texture related) on medical imaging data. Deep learning technology has revolutionized the field of AI and as a result, performance of AI algorithms is approaching human performance for an increasing number of specific tasks. X-ray velocimetry integrates x-ray (fluoroscopic) imaging with unique image processing to produce quantitative four dimensional measurement of lung tissue motion, and accurate calculations of lung ventilation

Texture Analysis and Machine Learning to Predict Pulmonary Ventilation from Thoracic Computed Tomography

Chronic obstructive pulmonary disease (COPD) leads to persistent airflow limitation, causing a large burden to patients and the health care system. Thoracic CT provides an opportunity to observe the structural pathophysiology of COPD, whereas hyperpolarized gas MRI provides images of the consequential ventilation heterogeneity. However, hyperpolarized gas MRI is currently limited to research centres, due to the high cost of gas and polarization equipment. Therefore, I developed a pipeline using texture analysis and machine learning methods to create predicted ventilation maps based on non-contrast enhanced, single-volume thoracic CT. In a COPD cohort, predicted ventilation maps were qualitatively and quantitatively related to ground-truth MRI ventilation, and both maps were related to important patient lung function and quality-of-life measures. This study is the first to demonstrate the feasibility of predicting hyperpolarized MRI-based ventilation from single-volume, breath-hold thoracic CT, which has potential to translate pulmonary ventilation information to widely available thoracic CT imaging

3-D lung deformation and function from respiratory-gated 4-D x-ray CT images : application to radiation treatment planning.

Many lung diseases or injuries can cause biomechanical or material property changes that can alter lung function. While the mechanical changes associated with the change of the material properties originate at a regional level, they remain largely asymptomatic and are invisible to global measures of lung function until they have advanced significantly and have aggregated. In the realm of external beam radiation therapy of patients suffering from lung cancer, determination of patterns of pre- and post-treatment motion, and measures of regional and global lung elasticity and function are clinically relevant. In this dissertation, we demonstrate that 4-D CT derived ventilation images, including mechanical strain, provide an accurate and physiologically relevant assessment of regional pulmonary function which may be incorporated into the treatment planning process. Our contributions are as follows: (i) A new volumetric deformable image registration technique based on 3-D optical flow (MOFID) has been designed and implemented which permits the possibility of enforcing physical constraints on the numerical solutions for computing motion field from respiratory-gated 4-D CT thoracic images. The proposed optical flow framework is an accurate motion model for the thoracic CT registration problem. (ii) A large displacement landmark-base elastic registration method has been devised for thoracic CT volumetric image sets containing large deformations or changes, as encountered for example in registration of pre-treatment and post-treatment images or multi-modality registration. (iii) Based on deformation maps from MOFIO, a novel framework for regional quantification of mechanical strain as an index of lung functionality has been formulated for measurement of regional pulmonary function. (iv) In a cohort consisting of seven patients with non-small cell lung cancer, validation of physiologic accuracy of the 4-0 CT derived quantitative images including Jacobian metric of ventilation, Vjac, and principal strains, (V?1, V?2, V?3, has been performed through correlation of the derived measures with SPECT ventilation and perfusion scans. The statistical correlations with SPECT have shown that the maximum principal strain pulmonary function map derived from MOFIO, outperforms all previously established ventilation metrics from 40-CT. It is hypothesized that use of CT -derived ventilation images in the treatment planning process will help predict and prevent pulmonary toxicity due to radiation treatment. It is also hypothesized that measures of regional and global lung elasticity and function obtained during the course of treatment may be used to adapt radiation treatment. Having objective methods with which to assess pre-treatment global and regional lung function and biomechanical properties, the radiation treatment dose can potentially be escalated to improve tumor response and local control

Validated respiratory drug deposition predictions from 2D and 3D medical images with statistical shape models and convolutional neural networks

For the one billion sufferers of respiratory disease, managing their disease with inhalers crucially influences their quality of life. Generic treatment plans could be improved with the aid of computational models that account for patient-specific features such as breathing pattern, lung pathology and morphology. Therefore, we aim to develop and validate an automated computational framework for patient-specific deposition modelling. To that end, an image processing approach is proposed that could produce 3D patient respiratory geometries from 2D chest X-rays and 3D CT images. We evaluated the airway and lung morphology produced by our image processing framework, and assessed deposition compared to in vivo data. The 2D-to-3D image processing reproduces airway diameter to 9% median error compared to ground truth segmentations, but is sensitive to outliers of up to 33% due to lung outline noise. Predicted regional deposition gave 5% median error compared to in vivo measurements. The proposed framework is capable of providing patient-specific deposition measurements for varying treatments, to determine which treatment would best satisfy the needs imposed by each patient (such as disease and lung/airway morphology). Integration of patient-specific modelling into clinical practice as an additional decision-making tool could optimise treatment plans and lower the burden of respiratory diseases.</p

Investigation of Neonatal Pulmonary Structure and Function via Proton and Hyperpolarized Gas Magnetic Resonance Imaging

Magnetic resonance imaging (MRI) is a modality that utilizes the phenomenon of nuclear magnetic resonance (NMR) to yield tomographic images of the body. Proton (1H) MRI has historically been successful in soft tissues but has suffered in the lung due to a variety of technical challenges, such as the low proton-density, rapid T2* relaxation time of the lung parenchymal tissue, and inherent physiological motion in the chest. Recent developments in radial ultrashort echo time (UTE) MRI have in part overcome these issues. In addition, there has been much progress in techniques for hyperpolarization of noble gases (3He and 129Xe) out of thermal equilibrium via spin exchange optical pumping, which can greatly enhance the gas NMR signal such that it is detectable within the airspaces of the lung on MRI. The lung is a unique organ due to its complex structural and functional dynamics, and its early development through the neonatal (newborn) period is not yet well understood in normal or abnormal conditions. Pulmonary morbidities are relatively common in infants and are present in a majority of patients admitted to the neonatal intensive care unit, often stemming from preterm birth and/or congenital defects. Current clinical lung imaging in these patients is typically limited to chest x-ray radiography, which does not provide tomographic information and so has lowered sensitivity. More rarely, x-ray computed tomography (CT) is used but exposes infants to ionizing radiation and typically requires sedation, both of which pose increased risks to pediatric patients. Thus the opportunity is ripe for application of novel pulmonary MRI techniques to the infant population. However, MR imaging of very small pulmonary structure and microstructure requires fundamental changes in the imaging theory of both 1H UTE MRI and hyperpolarized gas diffusion MRI. Furthermore, such young patients are often non-compliant, yielding a need for new and innovative techniques for monitoring respiratory and bulk motion. This dissertation describes methodology development and provides experimental results in both 1H UTE MRI and hyperpolarized 3He and 129Xe gas diffusion MRI, with investigation into the structure and function of infant lungs at both the macrostructural and microstructural level. In particular, anisotropically restricted gas diffusion within infant alveolar microstructure is investigated as a measurement of airspace size and geometry. Additionally, the phenomenon of respiratory and bulk motion-tracking via modulation of the k-space center\u27s magnitude and phase is explored and applied via UTE MRI in various neonatal pulmonary conditions to extract imaging-based metrics of diagnostic value. Further, the proton-density regime of pulmonary UTE MRI is validated in translational applications. These techniques are applied in infants with various pulmonary conditions, including patients diagnosed with bronchopulmonary dysplasia, congenital diaphragmatic hernia, esophageal atresia/tracheoesophageal fistula, tracheomalacia, and no suspected lung disease. In addition, explanted lung specimens from both infants with and without lung disease are examined. Development and implementation of these techniques involves a strong understanding of the physics-based theory of NMR, hyperpolarization, and MR imaging, in addition to foundations in hardware, software, and image analysis techniques. This thesis first outlines the theory and background of NMR, MRI, and pulmonary physiology and development (Part I), then proceeds into the theory, equipment, and imaging experiments for hyperpolarized gas diffusion MRI in infant lung airspaces (Part II), and finally details the theory, data processing methods, and applications of pulmonary UTE MRI in infant patients (Part III). The potential for clinical translation of the neonatal pulmonary MRI methods presented in this dissertation is very high, with the foundations of these techniques firmly rooted in the laws of physics

Statistical Shape Modelling and Segmentation of the Respiratory Airway

The human respiratory airway consists of the upper (nasal cavity, pharynx) and the lower (trachea, bronchi) respiratory tracts. Accurate segmentation of these two airway tracts can lead to better diagnosis and interpretation of airway-specific diseases, and lead to improvement in the localization of abnormal metabolic or pathological sites found within and/or surrounding the respiratory regions. Due to the complexity and the variability displayed in the anatomical structure of the upper respiratory airway along with the challenges in distinguishing the nasal cavity from non-respiratory regions such as the paranasal sinuses, it is difficult for existing algorithms to accurately segment the upper airway without manual intervention. This thesis presents an implicit non-parametric framework for constructing a statistical shape model (SSM) of the upper and lower respiratory tract, capable of distinct shape generation and be adapted for segmentation. An SSM of the nasal cavity was successfully constructed using 50 nasal CT scans. The performance of the SSM was evaluated for compactness, specificity and generality. An averaged distance error of 1.47 mm was measured for the generality assessment. The constructed SSM was further adapted with a modified locally constrained random walk algorithm to segment the nasal cavity. The proposed algorithm was evaluated on 30 CT images and outperformed comparative state-of-the-art and conventional algorithms. For the lower airway, a separate algorithm was proposed to automatically segment the trachea and bronchi, and was designed to tolerate the image characteristics inherent in low-contrast CT images. The algorithm was evaluated on 20 clinical low-contrast CT from PET-CT patient studies and demonstrated better performance (87.1±2.8 DSC and distance error of 0.37±0.08 mm) in segmentation results against comparative state-of-the-art algorithms

Human Treelike Tubular Structure Segmentation: A Comprehensive Review and Future Perspectives

Various structures in human physiology follow a treelike morphology, which often expresses complexity at very fine scales. Examples of such structures are intrathoracic airways, retinal blood vessels, and hepatic blood vessels. Large collections of 2D and 3D images have been made available by medical imaging modalities such as magnetic resonance imaging (MRI), computed tomography (CT), Optical coherence tomography (OCT) and ultrasound in which the spatial arrangement can be observed. Segmentation of these structures in medical imaging is of great importance since the analysis of the structure provides insights into disease diagnosis, treatment planning, and prognosis. Manually labelling extensive data by radiologists is often time-consuming and error-prone. As a result, automated or semi-automated computational models have become a popular research field of medical imaging in the past two decades, and many have been developed to date. In this survey, we aim to provide a comprehensive review of currently publicly available datasets, segmentation algorithms, and evaluation metrics. In addition, current challenges and future research directions are discussed.Comment: 30 pages, 19 figures, submitted to CBM journa