Semantic Segmentation of Pathological Lung Tissue with Dilated Fully Convolutional Networks

Early and accurate diagnosis of interstitial lung diseases (ILDs) is crucial for making treatment decisions, but can be challenging even for experienced radiologists. The diagnostic procedure is based on the detection and recognition of the different ILD pathologies in thoracic CT scans, yet their manifestation often appears similar. In this study, we propose the use of a deep purely convolutional neural network for the semantic segmentation of ILD patterns, as the basic component of a computer aided diagnosis (CAD) system for ILDs. The proposed CNN, which consists of convolutional layers with dilated filters, takes as input a lung CT image of arbitrary size and outputs the corresponding label map. We trained and tested the network on a dataset of 172 sparsely annotated CT scans, within a cross-validation scheme. The training was performed in an end-to-end and semi-supervised fashion, utilizing both labeled and non-labeled image regions. The experimental results show significant performance improvement with respect to the state of the art

Unsupervised and Weakly-Supervised Learning of Localized Texture Patterns of Lung Diseases on Computed Tomography

Computed tomography (CT) imaging enables in vivo assessment of lung parenchyma and several lung diseases. CT scans are key in particular for the diagnosis of 1) chronic obstructive pulmonary disease (COPD), which is the fourth leading cause of death worldwide, and largely overlaps with pulmonary emphysema; and 2) lung cancer, which is the first leading cause of cancer-related death, and manifests in its early stage with the presence of lung nodules. Most lung CT image analysis methods to-date have relied on supervised learning requiring manually annotated local regions of interest (ROIs), which are slow and labor-intensive to obtain. Machine learning models requiring less or no manual annotations are important for a sustainable development of computer-aided diagnosis (CAD) systems. This thesis focused on exploiting CT scans for lung disease characterization via two learning strategies: 1) fully unsupervised learning on a very large amount of unannotated image patches to discover novel lung texture patterns for pulmonary emphysema; and 2) weakly-supervised learning to generate voxel-level localization of lung nodules from CT whole-slice labels. In the first part of this thesis, we proposed an original unsupervised approach to learn emphysema-specific radiological texture patterns. We have designed dedicated spatial and texture features and a two-stage learning strategy incorporating clustering and graph partitioning. Learning was performed on a cohort of 2,922 high-resolution full-lung CT scans, which included a high prevalence of smokers and COPD subjects. Experiments lead to discovering 10 highly-reproducible spatially-informed lung texture patterns and 6 quantitative emphysema subtypes (QES). Our discovered QES were associated independently with distinct risk of symptoms, physiological changes, exacerbations and mortality. Genome-wide association studies identified loci associated with four subtypes. Then we designed a deep-learning approach, using unsupervised domain adaptation with adversarial training, to label the QES on cardiac CT scans, which included approximately 70% of the lung. Our proposed method accounted for the differences in CT image qualities, and enabled us to study the progression of QES on a cohort of 17,039 longitudinal cardiac and full-lung CT scans. Overall, the discovered QES provide novel emphysema sub-phenotyping that may facilitate future study of emphysema development, understanding the stages of COPD and the design of personalized therapies. In the second part of the thesis, we have designed a deep-learning method for lung nodule detection with weak labels, using classification convolutional neural networks (CNNs) with skip-connections to generate high-quality discriminative class activation maps, and a novel candidate screening framework to reduce the number of false positives. Given that the vast majority of annotated nodules are benign, we further exploited a data augmentation framework with a generative adversarial network (GAN) to address the issue of data imbalance for lung cancer prediction. Our weakly-supervised lung nodule detection on 1,000s CT scans achieved competitive performance compared to a fully-supervised method, while requiring 100 times less annotations. Our data augmentation framework enabled synthesizing nodules with high fidelity in specified categories, and is beneficial for predicting nodule malignancy scores and hence improving the accuracy / reliability of lung cancer screening

Adaptive Quantification and Subtyping of Pulmonary Emphysema on Computed Tomography

Pulmonary emphysema contributes to the chronic airflow limitation characteristic of chronic obstructive pulmonary disease (COPD), which is a leading cause of morbidity and mortality worldwide. Computed tomography (CT) has enabled in vivo assessment of pulmonary emphysema at the macroscopic level, and is commonly used to identify and assess the extent of the disease. During the past decade, the availability of CT imaging data has increased rapidly, while the image quality has continued to improve. High-resolution CT is extremely valuable both for patient diagnosis and for studying diseases at the population level. However, visual assessment of these large data sets is subjective, inefficient, and expensive. This has increased the demand for objective, automatic, and reproducible image analysis methods. For the assessment of pulmonary emphysema on CT, computational models usually aim either to give a measure of the extent of the disease, or to categorize the emphysema subtypes apparent in a scan. The standard methods for quantitating emphysema extent are widely used, but they remain sensitive to changes in imaging protocols and patient inspiration level. For computational subtyping of emphysema, the methods remain at a developmental stage, and one of the main challenges is the lack of reliable label data. Furthermore, the classic emphysema subtypes were defined on autopsy before the availability of CT and could be considered outdated. There is also no consensus on how to match the subtypes on autopsy to the varying emphysema patterns present on CT. This work presents two methodological improvements for analyzing emphysema on CT. For the assessment of emphysema extent, a novel probabilistic approach is introduced and evaluated on a longitudinal data set with varying imaging protocols. The presented model is shown to improve significantly compared to standard methods, particularly at the presence of differing noise levels. The approach is also applied on quantifying emphysema on a large data set of cardiac CT scans, and is shown to improve the prediction of emphysema extent on subsequent full-lung CT scans. The second major contribution of this work applies unsupervised learning to recognizing patterns of emphysema on CT. Instead of trying to reproduce the classic subtypes, the novel approach aims to capture the most dominant variations of lung structure pertaining to emphysema. While removing the reliance on visually assigned labels, the learned patterns are shown to represent different manifestations of emphysema with distinct appearances and regular spatial distributions. The clinical significance of the patterns is also demonstrated, along with high-level performance in the application of content-based image retrieval. The contributions of this work advance the analysis of emphysema on CT by applying novel machine learning approaches to increase the value of the available imaging data. Probabilistic methods improve from the crude standard methods that are currently used to quantitate emphysema, and the value of learning disease patterns directly from image data is demonstrated. The common framework relying on replicating visually assigned labels of outdated subtypes has not achieved widespread acceptance. The methodology presented in this work may have a substantial impact on how emphysema subtypes on CT are recognized and defined in the future

Image Analysis Applications of the Maximum Mean Discrepancy Distance Measure

The need to quantify distance between two groups of objects is prevalent throughout the signal processing world. The difference of group means computed using the Euclidean, or L2 distance, is one of the predominant distance measures used to compare feature vectors and groups of vectors, but many problems arise with it when high data dimensionality is present. Maximum mean discrepancy (MMD) is a recent unsupervised kernel-based pattern recognition method which may improve differentiation between two distinct populations over many commonly used methods such as the difference of means, when paired with the proper feature representations and kernels. MMD-based distance computation combines many powerful concepts from the machine learning literature, such as data distribution-leveraging similarity measures and kernel methods for machine learning. Due to this heritage, we posit that dissimilarity-based classification and changepoint detection using MMD can lead to enhanced separation between different populations. To test this hypothesis, we conduct studies comparing MMD and the difference of means in two subareas of image analysis and understanding: first, to detect scene changes in video in an unsupervised manner, and secondly, in the biomedical imaging field, using clinical ultrasound to assess tumor response to treatment. We leverage effective computer vision data descriptors, such as the bag-of-visual-words and sparse combinations of SIFT descriptors, and choose from an assessment of several similarity kernels (e.g. Histogram Intersection, Radial Basis Function) in order to engineer useful systems using MMD. Promising improvements over the difference of means, measured primarily using precision/recall for scene change detection, and k-nearest neighbour classification accuracy for tumor response assessment, are obtained in both applications.1 yea

Multifractal techniques for analysis and classification of emphysema images

This thesis proposes, develops and evaluates different multifractal methods for detection, segmentation and classification of medical images. This is achieved by studying the structures of the image and extracting the statistical self-similarity measures characterized by the Holder exponent, and using them to develop texture features for segmentation and classification. The theoretical framework for fulfilling these goals is based on the efficient computation of fractal dimension, which has been explored and extended in this work. This thesis investigates different ways of computing the fractal dimension of digital images and validates the accuracy of each method with fractal images with predefined fractal dimension. The box counting and the Higuchi methods are used for the estimation of fractal dimensions. A prototype system of the Higuchi fractal dimension of the computed tomography (CT) image is used to identify and detect some of the regions of the image with the presence of emphysema. The box counting method is also used for the development of the multifractal spectrum and applied to detect and identify the emphysema patterns. We propose a multifractal based approach for the classification of emphysema patterns by calculating the local singularity coefficients of an image using four multifractal intensity measures. One of the primary statistical measures of self-similarity used in the processing of tissue images is the Holder exponent (α-value) that represents the power law, which the intensity distribution satisfies in the local pixel neighbourhoods. The fractal dimension corresponding to each α-value gives a multifractal spectrum f(α) that was used as a feature descriptor for classification. A feature selection technique is introduced and implemented to extract some of the important features that could increase the discriminating capability of the descriptors and generate the maximum classification accuracy of the emphysema patterns. We propose to further improve the classification accuracy of emphysema CT patterns by combining the features extracted from the alpha-histograms and the multifractal descriptors to generate a new descriptor. The performances of the classifiers are measured by using the error matrix and the area under the receiver operating characteristic curve (AUC). The results at this stage demonstrated the proposed cascaded approach significantly improves the classification accuracy. Another multifractal based approach using a direct determination approach is investigated to demonstrate how multifractal characteristic parameters could be used for the identification of emphysema patterns in HRCT images. This further analysis reveals the multi-scale structures and characteristic properties of the emphysema images through the generalized dimensions. The results obtained confirm that this approach can also be effectively used for detecting and identifying emphysema patterns in CT images. Two new descriptors are proposed for accurate classification of emphysema patterns by hybrid concatenation of the local features extracted from the local binary patterns (LBP) and the global features obtained from the multifractal images. The proposed combined feature descriptors of the LBP and f(α) produced a very good performance with an overall classification accuracy of 98%. These performances outperform other state-of-the-art methods for emphysema pattern classification and demonstrate the discriminating power and robustness of the combined features for accurate classification of emphysema CT images. Overall, experimental results have shown that the multifractal could be effectively used for the classifications and detections of emphysema patterns in HRCT images

Artificial Intelligence for Detection, Characterization, and Classification of Complex Visual Patterns in Medical Imaging; Applications in Pulmonary and Neuro-imaging

Medical imaging is widely used in current healthcare and research settings for various purposes such as diagnosis, treatment options, patient monitoring, longitudinal studies, etc. The two most commonly used imaging modalities in the United States are Computed Tomography (CT) and Magnetic Resonance Imaging (MRI). Raw images acquired via CT or MRI need to undergo a variety of processing steps prior to being used for the purposes explained above. These processing steps include quality control, noise reduction, anatomical segmentation, tissue classification, etc. However, since medical images often include millions of voxels (smallest 3D units in the image containing information) it is extremely challenging to process them manually by relying on visual inspection and the experience of trained clinicians. In light of this, the field of medical imaging is seeking ways to automate data processing. With the impressive performance of Artificial Intelligence (AI) in the field of Computer Vision, researchers in the medical imaging community have shown increasing interest in utilizing this powerful tool to automate the task of processing medical imaging data. Despite AI’s significant contributions to the medical imaging field, large cohorts of data still remain without optimized and robust AI-based tools to process images efficiently and accurately. This thesis focuses on exploiting large cohorts of CT and MRI data to design AI-based methods for processing medical images using weakly-supervised and supervised learning strategies, as well as mathematical (and/or statistical) modeling and signal processing methods. In particular, we address four image processing problems in this thesis. Namely: 1) We propose a weakly-supervised deep learning method to automate binary quality control of diffusion MRI scans into ‘poor’ and ‘good’ quality classes; 2) We design a weakly-supervised deep learning framework to learn and detect visual patterns related to a set of different artifact categories considered in this work, in order to identify major artifact types present in dMRI volumes; 3) We develop a supervised deep learning method to classify multiple lung texture patterns with association to Emphysema disease on human lung CT scans; 4) We investigate and characterize the properties of two types of negative BOLD response elicited in human brain fMRI scans during visual stimulation using mathematical modeling and signal processing tools. Our results demonstrate that through the use of artificial intelligence and signal processing algorithms: 1) dMRI scans can be automatically categorized into two quality groups (i.e., ‘poor’ vs ‘good’) with a high classification accuracy, enabling rapid sifting of large cohorts of dMRI scans to be utilized in research or clinical settings; 2) Type of the major artifact present in ‘poor’ quality dMRI volumes can be identified robustly and automatically with high precision enabling exclusion/correction of corrupt volumes according to the artifact type contaminating them; 3) Multiple lung texture patterns related to Emphysema disease can be automatically and robustly classified across various large cohorts of CT scans enabling investigation of the disease through longitudinal studies on multiple cohorts; 4) Negative BOLD responses of different categories can be fully characterized on fMRI data collected from visual stimulation of human brain enabling researchers to better understand the human brain functionality through studying cohorts of fMRI scans

Quantitative lung CT analysis for the study and diagnosis of Chronic Obstructive Pulmonary Disease

The importance of medical imaging in the research of Chronic Obstructive Pulmonary Dis- ease (COPD) has risen over the last decades. COPD affects the pulmonary system through two competing mechanisms; emphysema and small airways disease. The relative contribu- tion of each component varies widely across patients whilst they can also evolve regionally in the lung. Patients can also be susceptible to exacerbations, which can dramatically ac- celerate lung function decline. Diagnosis of COPD is based on lung function tests, which measure airflow limitation. There is a growing consensus that this is inadequate in view of the complexities of COPD. Computed Tomography (CT) facilitates direct quantification of the pathological changes that lead to airflow limitation and can add to our understanding of the disease progression of COPD. There is a need to better capture lung pathophysiology whilst understanding regional aspects of disease progression. This has motivated the work presented in this thesis. Two novel methods are proposed to quantify the severity of COPD from CT by analysing the global distribution of features sampled locally in the lung. They can be exploited in the classification of lung CT images or to uncover potential trajectories of disease progression. A novel lobe segmentation algorithm is presented that is based on a probabilistic segmen- tation of the fissures whilst also constructing a groupwise fissure prior. In combination with the local sampling methods, a pipeline of analysis was developed that permits a re- gional analysis of lung disease. This was applied to study exacerbation susceptible COPD. Lastly, the applicability of performing disease progression modelling to study COPD has been shown. Two main subgroups of COPD were found, which are consistent with current clinical knowledge of COPD subtypes. This research may facilitate precise phenotypic characterisation of COPD from CT, which will increase our understanding of its natural history and associated heterogeneities. This will be instrumental in the precision medicine of COPD