Advanced machine learning methods for oncological image analysis

Cancer is a major public health problem, accounting for an estimated 10 million deaths worldwide in 2020 alone. Rapid advances in the field of image acquisition and hardware development over the past three decades have resulted in the development of modern medical imaging modalities that can capture high-resolution anatomical, physiological, functional, and metabolic quantitative information from cancerous organs. Therefore, the applications of medical imaging have become increasingly crucial in the clinical routines of oncology, providing screening, diagnosis, treatment monitoring, and non/minimally- invasive evaluation of disease prognosis. The essential need for medical images, however, has resulted in the acquisition of a tremendous number of imaging scans. Considering the growing role of medical imaging data on one side and the challenges of manually examining such an abundance of data on the other side, the development of computerized tools to automatically or semi-automatically examine the image data has attracted considerable interest. Hence, a variety of machine learning tools have been developed for oncological image analysis, aiming to assist clinicians with repetitive tasks in their workflow. This thesis aims to contribute to the field of oncological image analysis by proposing new ways of quantifying tumor characteristics from medical image data. Specifically, this thesis consists of six studies, the first two of which focus on introducing novel methods for tumor segmentation. The last four studies aim to develop quantitative imaging biomarkers for cancer diagnosis and prognosis. The main objective of Study I is to develop a deep learning pipeline capable of capturing the appearance of lung pathologies, including lung tumors, and integrating this pipeline into the segmentation networks to leverage the segmentation accuracy. The proposed pipeline was tested on several comprehensive datasets, and the numerical quantifications show the superiority of the proposed prior-aware DL framework compared to the state of the art. Study II aims to address a crucial challenge faced by supervised segmentation models: dependency on the large-scale labeled dataset. In this study, an unsupervised segmentation approach is proposed based on the concept of image inpainting to segment lung and head- neck tumors in images from single and multiple modalities. The proposed autoinpainting pipeline shows great potential in synthesizing high-quality tumor-free images and outperforms a family of well-established unsupervised models in terms of segmentation accuracy. Studies III and IV aim to automatically discriminate the benign from the malignant pulmonary nodules by analyzing the low-dose computed tomography (LDCT) scans. In Study III, a dual-pathway deep classification framework is proposed to simultaneously take into account the local intra-nodule heterogeneities and the global contextual information. Study IV seeks to compare the discriminative power of a series of carefully selected conventional radiomics methods, end-to-end Deep Learning (DL) models, and deep features-based radiomics analysis on the same dataset. The numerical analyses show the potential of fusing the learned deep features into radiomic features for boosting the classification power. Study V focuses on the early assessment of lung tumor response to the applied treatments by proposing a novel feature set that can be interpreted physiologically. This feature set was employed to quantify the changes in the tumor characteristics from longitudinal PET-CT scans in order to predict the overall survival status of the patients two years after the last session of treatments. The discriminative power of the introduced imaging biomarkers was compared against the conventional radiomics, and the quantitative evaluations verified the superiority of the proposed feature set. Whereas Study V focuses on a binary survival prediction task, Study VI addresses the prediction of survival rate in patients diagnosed with lung and head-neck cancer by investigating the potential of spherical convolutional neural networks and comparing their performance against other types of features, including radiomics. While comparable results were achieved in intra- dataset analyses, the proposed spherical-based features show more predictive power in inter-dataset analyses. In summary, the six studies incorporate different imaging modalities and a wide range of image processing and machine-learning techniques in the methods developed for the quantitative assessment of tumor characteristics and contribute to the essential procedures of cancer diagnosis and prognosis

Learning Algorithms for Fat Quantification and Tumor Characterization

Obesity is one of the most prevalent health conditions. About 30% of the world\u27s and over 70% of the United States\u27 adult populations are either overweight or obese, causing an increased risk for cardiovascular diseases, diabetes, and certain types of cancer. Among all cancers, lung cancer is the leading cause of death, whereas pancreatic cancer has the poorest prognosis among all major cancers. Early diagnosis of these cancers can save lives. This dissertation contributes towards the development of computer-aided diagnosis tools in order to aid clinicians in establishing the quantitative relationship between obesity and cancers. With respect to obesity and metabolism, in the first part of the dissertation, we specifically focus on the segmentation and quantification of white and brown adipose tissue. For cancer diagnosis, we perform analysis on two important cases: lung cancer and Intraductal Papillary Mucinous Neoplasm (IPMN), a precursor to pancreatic cancer. This dissertation proposes an automatic body region detection method trained with only a single example. Then a new fat quantification approach is proposed which is based on geometric and appearance characteristics. For the segmentation of brown fat, a PET-guided CT co-segmentation method is presented. With different variants of Convolutional Neural Networks (CNN), supervised learning strategies are proposed for the automatic diagnosis of lung nodules and IPMN. In order to address the unavailability of a large number of labeled examples required for training, unsupervised learning approaches for cancer diagnosis without explicit labeling are proposed. We evaluate our proposed approaches (both supervised and unsupervised) on two different tumor diagnosis challenges: lung and pancreas with 1018 CT and 171 MRI scans respectively. The proposed segmentation, quantification and diagnosis approaches explore the important adiposity-cancer association and help pave the way towards improved diagnostic decision making in routine clinical practice

Computational methods for the analysis of functional 4D-CT chest images.

Medical imaging is an important emerging technology that has been intensively used in the last few decades for disease diagnosis and monitoring as well as for the assessment of treatment effectiveness. Medical images provide a very large amount of valuable information that is too huge to be exploited by radiologists and physicians. Therefore, the design of computer-aided diagnostic (CAD) system, which can be used as an assistive tool for the medical community, is of a great importance. This dissertation deals with the development of a complete CAD system for lung cancer patients, which remains the leading cause of cancer-related death in the USA. In 2014, there were approximately 224,210 new cases of lung cancer and 159,260 related deaths. The process begins with the detection of lung cancer which is detected through the diagnosis of lung nodules (a manifestation of lung cancer). These nodules are approximately spherical regions of primarily high density tissue that are visible in computed tomography (CT) images of the lung. The treatment of these lung cancer nodules is complex, nearly 70% of lung cancer patients require radiation therapy as part of their treatment. Radiation-induced lung injury is a limiting toxicity that may decrease cure rates and increase morbidity and mortality treatment. By finding ways to accurately detect, at early stage, and hence prevent lung injury, it will have significant positive consequences for lung cancer patients. The ultimate goal of this dissertation is to develop a clinically usable CAD system that can improve the sensitivity and specificity of early detection of radiation-induced lung injury based on the hypotheses that radiated lung tissues may get affected and suffer decrease of their functionality as a side effect of radiation therapy treatment. These hypotheses have been validated by demonstrating that automatic segmentation of the lung regions and registration of consecutive respiratory phases to estimate their elasticity, ventilation, and texture features to provide discriminatory descriptors that can be used for early detection of radiation-induced lung injury. The proposed methodologies will lead to novel indexes for distinguishing normal/healthy and injured lung tissues in clinical decision-making. To achieve this goal, a CAD system for accurate detection of radiation-induced lung injury that requires three basic components has been developed. These components are the lung fields segmentation, lung registration, and features extraction and tissue classification. This dissertation starts with an exploration of the available medical imaging modalities to present the importance of medical imaging in today’s clinical applications. Secondly, the methodologies, challenges, and limitations of recent CAD systems for lung cancer detection are covered. This is followed by introducing an accurate segmentation methodology of the lung parenchyma with the focus of pathological lungs to extract the volume of interest (VOI) to be analyzed for potential existence of lung injuries stemmed from the radiation therapy. After the segmentation of the VOI, a lung registration framework is introduced to perform a crucial and important step that ensures the co-alignment of the intra-patient scans. This step eliminates the effects of orientation differences, motion, breathing, heart beats, and differences in scanning parameters to be able to accurately extract the functionality features for the lung fields. The developed registration framework also helps in the evaluation and gated control of the radiotherapy through the motion estimation analysis before and after the therapy dose. Finally, the radiation-induced lung injury is introduced, which combines the previous two medical image processing and analysis steps with the features estimation and classification step. This framework estimates and combines both texture and functional features. The texture features are modeled using the novel 7th-order Markov Gibbs random field (MGRF) model that has the ability to accurately models the texture of healthy and injured lung tissues through simultaneously accounting for both vertical and horizontal relative dependencies between voxel-wise signals. While the functionality features calculations are based on the calculated deformation fields, obtained from the 4D-CT lung registration, that maps lung voxels between successive CT scans in the respiratory cycle. These functionality features describe the ventilation, the air flow rate, of the lung tissues using the Jacobian of the deformation field and the tissues’ elasticity using the strain components calculated from the gradient of the deformation field. Finally, these features are combined in the classification model to detect the injured parts of the lung at an early stage and enables an earlier intervention

3D Lung Nodule Classification in Computed Tomography Images

Lung cancer is the leading cause of cancer death worldwide. One of the reasons is the absence of symptoms at an early stage, which means that it is only discovered at a later stage, where the treatment is more difficult [1]. Furthermore, when making a diagnosis, frequently done by reading computed tomographies (CT's), it is regularly allied with errors. One of the reasons is the variation of the opinion of the doctors regarding the diagnosis of the same nodule [2,3].The use of CADx, Computer-Aided Diagnosis, systems can be a great help for this problem by assisting doctors in diagnosis with a second opinion. Although its efficiency has already been proven [4], it often ends up not being used because doctors can not understand the "how and why" of CADx diagnostic results, and ultimately do not trust the system [5]. To increase the radiologists' confidence in the CADx system it is proposed that along with the results of malignancy prediction, there are also results with evidence that explains those malignancy results.There are some visible features in lung nodules that are correlated with malignancy. Since humans are able to visually identify these characteristics and correlate them with nodule malignancy, one way to present those evidence is to make predictions of those characteristics. To have these predictions it is proposed to use deep learning approaches. Convolutional neural networks had shown to outperform the state of the art results in medical image analysis [6]. To predict the characteristics and malignancy in CADx system, the architecture HSCNN, a deep hierarchical semantic convolutional neural network, proposed by Shen et al. [7], will be used.The Lung Image Database Consortium image collection (LIDC-IDRI) public dataset is frequently used as input for lung cancer CADx systems. The LIDC-IDRI consists of thoracic CT scans, presenting a lot of data's quantity and variability. In most of the nodules, this dataset has doctor's evaluations for 9 different characteristics. A recurrent problem in those evaluations is the subjectivity of the doctors' interpretation in what each characteristic is. In some characteristics, it can result in a great divergence in evaluations regarding the same nodule, which makes the inclusion of those evaluations as an input in CADx systems not useful as it could be. To reduce this subjectivity, it is proposed the creation of a metric that makes the characteristics classification more objective. For this, it is planned bibliographic and LIDC-IDRI dataset reviews. With that, taking into account this new metric, validated after by doctors from Hospital de São João, will be made a reclassification in LIDC-IDRI dataset. This way it could be possible to use as input all the relevant characteristics. The principal objective of this dissertation is to develop a lung nodule CADx system methodology which promotes the confidence of specialists in its use. This will be made classifying lung nodules according to relevant characteristics to diagnosis and malignancy. The reclassified LIDC-IDRI dataset will be used as an input for CADx system and the architecture used for predicting the characteristics and malignancy results will be the HSCNN. To measure the classification evaluation will be used sensitivity, sensibility, and area under the Receiver Operating Characteristic (ROC), curve. The proposed solution may be used for improving a CADx system, LNDetector, currently in development by the Center for Biomedical Engineering Research (C-BER) group from INESC-TEC in which this work will be developed.[1] - S. Sone M. Hasegawa and S. Takashima. Growth rate of small lung cancels detected on mass ct screening. Tire British Journal of Radiology, pages 1252-1259[2] - D. J. Bell S. E. Marley P. Guo H. Mann M. L. Scott L. H. Schwartz D. C. Ghiorghiu B. Zhao, Y. Tan. Exploring intra-and inter-reader variability in uni-dimensional, bi-dimensional, and volumetric measurements of solid tumors on ct scans reconstructed at different slice intervals. European journal of radiology 82, page 959-968, 2013[3] - H.T Winer-Muram. The solitary pulmonary nodule 1. Radiology, 239, pages 39-49, 2006.[4] - R. Yan J. Lee L. C. Chu C. T. Lin A. Hussien J. Rathmell B. Thomas C. Chen et al. P. Huang, S. Park. Added value of computer-aided ct image features for early lung cancer diagnosis with small pulmonary nodules: A matched case-control study. Radiology 286, page 286-295, 2017[5] - W Jorritsma, Fokie Cnossen, and Peter Van Ooijen. Improving the radiologist-cad interaction: Designing for appropriate trust. Clinical Radiology, 70, 10 2014.[6] - Tom Brosch, Youngjin Yoo, David Li, Anthony Traboulsee, and Roger Tam. Modeling the variability in brain morphology and lesion distribution in multiple sclerosis by deep learning. Volume 17, 09 2014.[7] - Simon Aberle Deni A. T. Bui Alex Hsu Willliam Shen, Shiwen X. Han. An interpretable deep hierarchical semantic convolutional neural network for lung nodule malignancy classification. june 201

Autoencoder-based Image Recommendation for Lung Cancer Characterization

Neste projeto, temos como objetivo desenvolver um sistema de IA que recomende um conjunto de casos relativos (passados) para orientar a tomada de decisão do médico. Objetivo: A ambição é desenvolver um modelo de aprendizado baseado em IA para caracterização de câncer de pulmão, a fim de auxiliar na rotina clínica. Considerando a complexidade dos fenômenos biológicos que ocorrem durante o desenvolvimento do câncer, as relações entre eles e as manifestações visuais capturadas pela tomografia computadorizada (CT) têm sido exploradas nos últimos anos. No entanto, devido à falta de robustez dos métodos atuais de aprendizado profundo, essas correlações são frequentemente consideradas espúrias e se perdem quando confrontadas com dados coletados a partir de distribuições alteradas: diferentes instituições, características demográficas ou até mesmo estágios de desenvolvimento do câncer.In this project, we aim to develop an AI system that recommends a set of relative (past) cases to guide the decision-making of the clinician. Objective: The ambition is to develop an AI-based learning model for lung cancer characterization in order to assist in clinical routine. Considering the complexity of the biological phenomenat hat occur during cancer development, relationships between these and visual manifestations captured by CT have been explored in recent years; however, given the lack of robustness of current deep learning methods, these correlations are often found spurious and get lost when facing data collected from shifted distributions: different institutions, demographics or even stages of cancer development