Optimization Strategies for Interactive Classification of Interstitial Lung Disease Textures

For computerized analysis of textures in interstitial lung disease, manual annotations of lung tissue are necessary. Since making these annotations is labor intensive, we previously proposed an interactive annotation framework. In this framework, observers iteratively trained a classifier to distinguish the different texture types by correcting its classification errors. In this work, we investigated three ways to extend this approach, in order to decrease the amount of user interaction required to annotate all lung tissue in a computed tomography scan. First, we conducted automatic classification experiments to test how data from previously annotated scans can be used for classification of the scan under consideration. We compared the performance of a classifier trained on data from one observer, a classifier trained on data from multiple observers, a classifier trained on consensus training data, and an ensemble of classifiers, each trained on data from different sources. Experiments were conducted without and with texture selection (ts). In the former case, training data from all eight textures was used. In the latter, only training data from the texture types present in the scan were used, and the observer would have to indicate textures contained in the scan to be analyzed. Second, we simulated interactive annotation to test the effects of (1) asking observers to perform ts before the start of annotation, (2) the use of a classifier trained on data from previously annotated scans at the start of annotation, when the interactive classifier is untrained, and (3) allowing observers to choose which interactive or automatic classification results they wanted to correct. Finally, various strategies for selecting the classification results that were presented to the observer were considered. Classification accuracies for all possible interactive annotation scenarios were compared. Using the best-performing protocol, in which observers select the textures that should be distinguished in the scan and in which they can choose which classification results to use for correction, a median accuracy of 88% was reached. The results obtained using this protocol were significantly better than results obtained with other interactive or automatic classification protocols

Prototypes for Content-Based Image Retrieval in Clinical Practice

Content-based image retrieval (CBIR) has been proposed as key technology for computer-aided diagnostics (CAD). This paper reviews the state of the art and future challenges in CBIR for CAD applied to clinical practice

Lung Pattern Analysis using Artificial Intelligence for the Diagnosis Support of Interstitial Lung Diseases

Interstitial lung diseases (ILDs) is a group of more than 200 chronic lung disorders characterized by inflammation and scarring of the lung tissue that leads to respiratory failure. Although ILD is a heterogeneous group of histologically distinct diseases, most of them exhibit similar clinical presentations and their diagnosis often presents a diagnostic dilemma. Early diagnosis is crucial for making treatment decisions, while misdiagnosis may lead to life-threatening complications. If a final diagnosis cannot be reached with the high resolution computed tomography scan, additional invasive procedures are required (e.g. bronchoalveolar lavage, surgical biopsy). The aim of this PhD thesis was to investigate the components of a computational system that will assist radiologists with the diagnosis of ILDs, while avoiding the dangerous, expensive and time-consuming invasive biopsies. The appropriate interpretation of the available radiological data combined with clinical/biochemical information can provide a reliable diagnosis, able to improve the diagnostic accuracy of the radiologists. In this thesis, we introduce two convolutional neural networks particularly designed for ILDs and a training scheme that employs knowledge transfer from the similar domain of general texture classification for performance enhancement. Moreover, we investigate the clinical relevance of breathing information for disease classification. The breathing information is quantified as a deformation field between inhale-exhale lung images using a novel 3D convolutional neural network for medical image registration. Finally, we design and evaluate the final end-to-end computational system for ILD classification using lung anatomy segmentation algorithms from the literature and the proposed ILD quantification neural networks. Deep learning approaches have been mostly investigated for all the aforementioned steps, while the results demonstrated their potential in analyzing lung images

Computer-aided diagnosis in chest radiography

Chest radiographs account for more than half of all radiological examinations; the chest is the mirror of health and disease. This thesis is about techniques for computer analysis of chest radiographs. It describes methods for texture analysis and segmenting the lung fields and rib cage in a chest film. It includes a description of an automatic system for detecting regions with abnormal texture, that is applied to a database of images from a tuberculosis screening program

Texture Analysis of Late Gadolinium Enhanced Cardiac Magnetic Resonance Images for Characterizing Myocardial Fibrosis and Infarction

Le tiers de la population aux États-Unis est affecté par des cardiomyopathies. Lorsque le muscle du coeur, le myocarde, est altéré par la maladie, la santé du patient est détériorée et peut même entrainer la mort. Les maladies ischémiques sont le résultat d’artères coronariennes bloquées (sténose), limitant l’apport sanguin vers le myocarde. Les cardiomyopathies non-ischémiques sont les maladies dues à d’autres causes que des sténoses. Les fibres de collagène (fibrose) s’infiltrent dans le muscle cardiaque dans le but de maintenir la forme et les fonctions cardiaques lorsque la structure du myocarde est affectée par des cardiomyopathies. Ce principe, nécessaire au fonctionnement du coeur en présence de maladies, devient mal adapté et mène à des altérations du myocarde aux conséquences négatives, par exemple l’augmentation de la rigidité du myocarde. Une partie du diagnostic clinique lors de cardiomyopathies consiste à évaluer la fibrose dans le coeur avec différentes modalités d’imagerie. Les fibres de collagène s’infiltrent et s’accumulent dans la zone extracellulaire du myocarde ou peuvent remplacer progressivement les cardiomyocytes compromises. L’infiltration de fibrose dans le myocarde peut possiblement être réversible, ce qui rend sa détection particulièrement importante pour le clinicien. Différents tests diagnostiques existent pour aider le clinicien à établir l’état du patient en présence de cardiomyopathies. L’imagerie par résonance magnétique (IRM) est une modalité d’imagerie qui offre une haute résolution pour la visualisation du myocarde. Parmi les séquences disponibles avec cette modalité, l’imagerie par rehaussement tardif (RT) augmente le contraste du signal existant entre les tissus sains et les tissues malades du myocarde. Il s’agit d’images en pondération T1 avec administration d’agent de contraste qui se propage dans la matrice extracellulaire et résulte en un rehaussement du signal à cet endroit. Les images IRM RT permettent d’évaluer la présence et l’étendue des dommages au myocarde. Le clinicien peut évaluer la sévérité des cardiomyopathies et poser un pronostique à l’aide de ces images. La détection de fibrose diffuse dans ces images peut informer le clinicien sur l’état du patient et est un important marqueur de cardiomyopathies. Il est important d’établir l’occurrence de l’infarctus en présence de maladies ischémiques. En effet, l’approche interventionnelle varie selon que le clinicien fait face à une ischémie aigue ou chronique. Lors du diagnostic, Il serait donc bénéfique de différencier les infarctus du myocarde aigu de ceux chronique. Ceci s’est avéré difficile à l’aide des images IRM RT où l’intensité du signal ou la taille des régions sont similaires dans les deux types d’ischémie. Le but de la présente thèse est donc d’appliquer les méthodes d’analyse de texture à des images IRM RT afin de détecter la présence de fibrose diffuse dans le myocarde et de plus de déterminer l’âge de l’infarctus du myocarde. La première étude portait sur la détection de fibrose diffuse dans le myocarde à l’aide de l’analyse de texture appliquée à des images IRM RT afin d’établir si un lien existe entre la variation du signal d’intensité et la structure sous-jacente du myocarde. La présence de collagène dans le myocarde augmente avec l’âge et nous avons utilisé un modèle animal de rats jeunes et âgés. Nous avons fait une étude ex-vivo afin d’obtenir des images IRM RT de haute résolution avec absence de mouvement et ainsi permettre une comparaison des images avec des coupes histologiques des coeurs imagés. Des images IRM RT ont été acquises sur vingt-quatre animaux. Les coupes histologiques ont été traitées avec la méthode utilisant un marqueur ‘picrosirius red’ qui donne une teinte rouge au collagène. La quantification de la fibrose obtenue avec les images IRM RT a été comparée à la quantification obtenue sur les coupes histologiques. Ces quantifications ont de plus été comparées à l’analyse de texture appliquée aux images IRM RT. La méthode de texture a été appliquée en créant des cartes de texture basées sur la valeur de Contraste, cette mesure étant obtenue par des calculs statistiques sur la matrice de cooccurrence. Les régions montrant une plus grande complexité de signal d’intensité sur les images IRM RT ont été rehaussées avec les cartes de textures. Un calcul de régression linéaire a permis d’étudier le lien entre les différentes méthodes de quantification. Nous avons trouvés que la quantification de fibrose dans le myocarde à l’aide de l’analyse de texture appliquée sur des images IRM RT concordait avec le niveau de collagène identifié avec les images IRM et avec les coupes histologiques. De plus, nous avons trouvés que l’analyse de texture rehausse la présence de fibrose diffuse dans le myocarde. La seconde étude a pour but de discriminer les infarctus aigus du myocarde de ceux qui sont chroniques sur des images IRM RT de patients souffrant de cardiomyopathies ischémiques. Vingt-deux patients ont subi l’imagerie IRM (12 avec infarctus aigu du myocarde et 12 avec infarctus chronique). Une segmentation des images a permis d’isoler les différentes zones du myocarde, soit la zone d’infarctus, la zone grise au rebord de l’infarctus et la zone du myocarde sain, dans les deux groupes de patients. L’analyse de texture s’est faite dans ces régions en comparant les valeurs obtenues dans les deux groupes. Nous avons obtenu plus de valeurs de texture discriminantes dans la zone grise, en comparaison avec la région du myocarde sain, où aucune valeur de texture n’était significativement différente, et à la zone d’infarctus, où seule la valeur de texture statistique Moyenne était différentes dans les deux groupes. La zone grise a déjà fait l’objet d’études ayant établis cette région comme composée de cardiomyocytes sains entremêlés avec des fibres de collagène. Notre étude montre que cette région peut exhiber des différences structurelles entre les infarctus aigus du myocarde et ceux qui sont chroniques et que l’analyse de texture a réussi à les détecter. L’étude de la présence de collagène dans le myocarde est importante pour le clinicien afin qu’il puisse faire un diagnostic adéquat du patient et pour qu’il puisse faire un choix de traitement approprié. Nous avons montrés que l’analyse de texture sur des images IRM RT de patients peut différencier et même permettre la classification des ischémies aigues des ischémies chroniques, ce qui n’était pas possible avec uniquement ce type d’images. Nous avons de plus démontrés que l’analyse de texture d’images IRM RT permettait d’évaluer le contenu de fibrose diffuse dans un modèle animal de haute résolution avec validation histologique. Une telle relation entre les résultats d’analyse de texture d’images IRM RT et la structure sous-jacente du myocarde n’avait pas été étudiée dans la littérature. Notre méthode pourra être améliorée en effectuant d’autres calculs statistiques sur la matrice de cooccurrence, en testant d’autres méthodes d’analyse de texture et en appliquant notre méthode à de nouvelles séquences d’acquisition IRM, tel les images en pondération T1. D’autres améliorations possibles pourraient porter sur une évaluation de matrice de cooccurrence avec voisinage circulaire suivant la forme du myocarde sur les tranches d’images IRM RT. Plusieurs matrice de cooccurrence pourraient aussi être évaluées en fonction de la position dans l’espace du voisinage afin d’intégrer une composante directionnelle dans les calculs de texture. D’autres études sont nécessaires afin d’établir si une analyse de texture des images IRM RT pourrait différencier le stade de la fibrose pour un même patient lors d’une étude de suivi. De même, d’autres études sont nécessaires afin de valider l’utilisation de texture sur des scanners IRM différents. Établir l’âge de l’infarctus du myocarde permettra de planifier les interventions thérapeutiques et d’évaluer le pronostique pour le patient.----------ABSTRACT A third of the United States population is affected by cardiomyopathies. Impairment of the heart muscle, the myocardium, puts the patient’s health at risk and could ultimately lead to death. Ischemic cardiomyopathies result from lack of blood (ischemia) reaching the myocardium from blocked coronary arteries. Non-ischemic cardiomyopathies are diseases from other etiology than ischemia. Often collagen fibers infiltrate the heart (fibrosis), as a means to maintain its shape and function in the presence of disease that affects the myocardial cellular structure. This necessary phenomenon ultimately becomes maladaptive and results in the heart’s impairment. Part of the heart’s involvement in disease can be assessed through the analysis of myocardial fibrosis. Cardiomyopathy diagnosis involves the investigation of the presence of myocardial fibrosis, either infiltrative, defined as the increased presence of collagen protein in the extracellular space, or replacement fibrosis, when collagen fibers progressively replace diseased cardiomyocytes. The infiltrative fibrosis is believed to be reversible in some instances and consequently, myocardial fibrosis analysis has decisional impact on the interventional procedure that would benefit the health of the patient. The heart contracts and relaxes as it pumps blood to the rest of the body, an action directly impaired by myocardial damage. Any myocardial involvement should be assessed by the clinician to identify the severity of the myocardial damage, establish a prognosis and plan therapeutic intervention. Different diagnostic tests are required to image the myocardium and help the clinician in the diagnostic process. Cardiac magnetic resonance (CMR) imaging has emerged as a high resolution imaging modality that offers precise structural analysis of the heart. Among the different imaging sequences available with CMR, late gadolinium enhancement (LGE) shows the myocardium and enhances any impairments that may exist with the use of a contrast agent. It is a T1-weighted image with extracellular contrast agent (CA) administration. Increased signal intensity in the infarct scar is created from the CA dynamics. LGE CMR imaging offers information on the scar size and its location. The clinician can estimate the severity of the disease and establish prognosis with LGE CMR images. In ischemic cardiomyopathy, it is important to establish the occurrence of the infarction and know the age of the infarct to plan surgical intervention. Differentiation of acute from chronic MI is therefore important in the diagnostic process. In LGE CMR the level of signal intensity or the size of infarction are both similar in acute or in chronic MI. It has therefore been challenging to distinguish acute MI from chronic MI scars with LGE CMR images alone. The aim of this thesis was to investigate texture analysis of LGE CMR images to determine if acute MI could be distinguished from chronic MI and to detect increased presence of diffuse myocardial fibrosis in the myocardium. The first study was performed to investigate if texture analysis of LGE CMR images could detect variations in the presence of diffuse myocardial fibrosis and if the underlying myocardial structure could be related to the texture measures. Collagen content increased with aging and we used an animal model of young versus old rat. An ex-vivo animal model was necessary to allow for higher image resolution in LGE CMR images and to perform validation of our texture measures with histology images. Twenty four animals were scanned for LGE CMR images and texture analysis was applied to the heart images. Histology slices were stained with picrosirius red and collagen fibers were isolated based on their color content. LGE CMR quantification was compared to histological slices of the heart stained with the picrosirius red method. Texture analysis of LGE CMR images was also compared to the original LGE CMR image quantification and to histology. Texture analysis was done by creating contrast texture maps extracted from Haralick’s gray level co-occurrence matrix (GLCM). Regions of complex signal intensity combination were enhanced in LGE CMR images and in contrast texture maps. Regression analysis was performed to assess the level of agreement between the different analysis methods. We found that LGE CMR images could assess the different levels of collagen content in the different aged animal model, and that moreover texture analysis enhanced those differences. The location of enhancement from texture analysis images corresponded to location of increased collagen content in the old compared to the young rat hearts. Histological validation was shown for texture analysis applied to LGE CMR images to assess myocardial fibrosis. Our second study aimed at discriminating acute versus chronic MI from LGE CMR patient images alone through the use of texture analysis. Twenty two patients who had LGE CMR images were included in our study (12 acute and 12 chronic MI). Regional segmentation was performed and texture features were compared in those regions between both groups of patient. Texture analysis resulted in significantly different values between the two groups. More specifically the peri-infarct zone had the most number of discriminative features compared to the remote myocardium which had none and to the infarct core where only the mean features was significantly different. The border zone has been shown to be composed of healthy cardiomyocytes intermingled with the scar’s collagen fibers. Our study indicates this region might exhibit structural differences in the myocardium in acute from chronic MI patients that texture analysis of LGE CMR images can detect. Characterization of myocardial collagen content is important while clinicians analyze the state of the patient since it influences the course of action required to treat cardiomyopathies. LGE CMR images have been thoroughly used and validated to characterize focal myocardial scar, however it was limited in characterizing the age of infarction or quantifying diffuse collagen content. We have shown texture analysis of LGE CMR images alone can differentiate and even classify, acute from chronic MI patients, which was not previously possible. Characterization of myocardial infarction according to age will prove important in planning therapeutic interventions in clinical practice. Moreover, we have established texture analysis as a means to characterize the myocardium and detect variation in fibrosis content from high resolution LGE CMR images with histology validation. To our knowledge, such a relation between texture analysis of LGE CMR images and the underlying myocardial structure had not been done previously. Improvements could be done to our method, as we can increase the number of texture features that were analyzed from the GLCM, include other texture analysis methods such as the run-length matrix, and apply our method to other CMR imaging sequences such as T1 mapping. Adapting the GLCM to the heart could also be investigated, such as considering circular GLCM computation to consider the round shape of the myocardium in the short axis LGE CMR image slices. Directional GLCM could also be computed individually and analyzed for any myocardial or collagen fiber orientation indication. Further analysis is also required to establish if texture analysis could differentiate the age of MI in the same individual through a follow-up study. The measures of texture analysis from LGE CMR images obtained through different CMR scanners remains to be investigated as well. Knowing the age of infarct and evaluating the presence of diffuse myocardial fibrosis will help the clinician plan therapeutic interventions and establish a prognosis for the patient

Advanced machine learning methods for oncological image analysis

Cancer is a major public health problem, accounting for an estimated 10 million deaths worldwide in 2020 alone. Rapid advances in the field of image acquisition and hardware development over the past three decades have resulted in the development of modern medical imaging modalities that can capture high-resolution anatomical, physiological, functional, and metabolic quantitative information from cancerous organs. Therefore, the applications of medical imaging have become increasingly crucial in the clinical routines of oncology, providing screening, diagnosis, treatment monitoring, and non/minimally- invasive evaluation of disease prognosis. The essential need for medical images, however, has resulted in the acquisition of a tremendous number of imaging scans. Considering the growing role of medical imaging data on one side and the challenges of manually examining such an abundance of data on the other side, the development of computerized tools to automatically or semi-automatically examine the image data has attracted considerable interest. Hence, a variety of machine learning tools have been developed for oncological image analysis, aiming to assist clinicians with repetitive tasks in their workflow. This thesis aims to contribute to the field of oncological image analysis by proposing new ways of quantifying tumor characteristics from medical image data. Specifically, this thesis consists of six studies, the first two of which focus on introducing novel methods for tumor segmentation. The last four studies aim to develop quantitative imaging biomarkers for cancer diagnosis and prognosis. The main objective of Study I is to develop a deep learning pipeline capable of capturing the appearance of lung pathologies, including lung tumors, and integrating this pipeline into the segmentation networks to leverage the segmentation accuracy. The proposed pipeline was tested on several comprehensive datasets, and the numerical quantifications show the superiority of the proposed prior-aware DL framework compared to the state of the art. Study II aims to address a crucial challenge faced by supervised segmentation models: dependency on the large-scale labeled dataset. In this study, an unsupervised segmentation approach is proposed based on the concept of image inpainting to segment lung and head- neck tumors in images from single and multiple modalities. The proposed autoinpainting pipeline shows great potential in synthesizing high-quality tumor-free images and outperforms a family of well-established unsupervised models in terms of segmentation accuracy. Studies III and IV aim to automatically discriminate the benign from the malignant pulmonary nodules by analyzing the low-dose computed tomography (LDCT) scans. In Study III, a dual-pathway deep classification framework is proposed to simultaneously take into account the local intra-nodule heterogeneities and the global contextual information. Study IV seeks to compare the discriminative power of a series of carefully selected conventional radiomics methods, end-to-end Deep Learning (DL) models, and deep features-based radiomics analysis on the same dataset. The numerical analyses show the potential of fusing the learned deep features into radiomic features for boosting the classification power. Study V focuses on the early assessment of lung tumor response to the applied treatments by proposing a novel feature set that can be interpreted physiologically. This feature set was employed to quantify the changes in the tumor characteristics from longitudinal PET-CT scans in order to predict the overall survival status of the patients two years after the last session of treatments. The discriminative power of the introduced imaging biomarkers was compared against the conventional radiomics, and the quantitative evaluations verified the superiority of the proposed feature set. Whereas Study V focuses on a binary survival prediction task, Study VI addresses the prediction of survival rate in patients diagnosed with lung and head-neck cancer by investigating the potential of spherical convolutional neural networks and comparing their performance against other types of features, including radiomics. While comparable results were achieved in intra- dataset analyses, the proposed spherical-based features show more predictive power in inter-dataset analyses. In summary, the six studies incorporate different imaging modalities and a wide range of image processing and machine-learning techniques in the methods developed for the quantitative assessment of tumor characteristics and contribute to the essential procedures of cancer diagnosis and prognosis

Quantitative Analysis of Radiation-Associated Parenchymal Lung Change

Radiation-induced lung damage (RILD) is a common consequence of thoracic radiotherapy (RT). We present here a novel classification of the parenchymal features of RILD. We developed a deep learning algorithm (DLA) to automate the delineation of 5 classes of parenchymal texture of increasing density. 200 scans were used to train and validate the network and the remaining 30 scans were used as a hold-out test set. The DLA automatically labelled the data with Dice Scores of 0.98, 0.43, 0.26, 0.47 and 0.92 for the 5 respective classes. Qualitative evaluation showed that the automated labels were acceptable in over 80% of cases for all tissue classes, and achieved similar ratings to the manual labels. Lung registration was performed and the effect of radiation dose on each tissue class and correlation with respiratory outcomes was assessed. The change in volume of each tissue class over time generated by manual and automated segmentation was calculated. The 5 parenchymal classes showed distinct temporal patterns We quantified the volumetric change in textures after radiotherapy and correlate these with radiotherapy dose and respiratory outcomes. The effect of local dose on tissue class revealed a strong dose-dependent relationship We have developed a novel classification of parenchymal changes associated with RILD that show a convincing dose relationship. The tissue classes are related to both global and local dose metrics, and have a distinct evolution over time. Although less strong, there is a relationship between the radiological texture changes we can measure and respiratory outcomes, particularly the MRC score which directly represents a patient’s functional status. We have demonstrated the potential of using our approach to analyse and understand the morphological and functional evolution of RILD in greater detail than previously possible