Segmentation and classification of lung nodules from Thoracic CT scans : methods based on dictionary learning and deep convolutional neural networks.

Lung cancer is a leading cause of cancer death in the world. Key to survival of patients is early diagnosis. Studies have demonstrated that screening high risk patients with Low-dose Computed Tomography (CT) is invaluable for reducing morbidity and mortality. Computer Aided Diagnosis (CADx) systems can assist radiologists and care providers in reading and analyzing lung CT images to segment, classify, and keep track of nodules for signs of cancer. In this thesis, we propose a CADx system for this purpose. To predict lung nodule malignancy, we propose a new deep learning framework that combines Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) to learn best in-plane and inter-slice visual features for diagnostic nodule classification. Since a nodule\u27s volumetric growth and shape variation over a period of time may reveal information regarding the malignancy of nodule, separately, a dictionary learning based approach is proposed to segment the nodule\u27s shape at two time points from two scans, one year apart. The output of a CNN classifier trained to learn visual appearance of malignant nodules is then combined with the derived measures of shape change and volumetric growth in assigning a probability of malignancy to the nodule. Due to the limited number of available CT scans of benign and malignant nodules in the image database from the National Lung Screening Trial (NLST), we chose to initially train a deep neural network on the larger LUNA16 Challenge database which was built for the purpose of eliminating false positives from detected nodules in thoracic CT scans. Discriminative features that were learned in this application were transferred to predict malignancy. The algorithm for segmenting nodule shapes in serial CT scans utilizes a sparse combination of training shapes (SCoTS). This algorithm captures a sparse representation of a shape in input data through a linear span of previously delineated shapes in a training repository. The model updates shape prior over level set iterations and captures variabilities in shapes by a sparse combination of the training data. The level set evolution is therefore driven by a data term as well as a term capturing valid prior shapes. During evolution, the shape prior influence is adjusted based on shape reconstruction, with the assigned weight determined from the degree of sparsity of the representation. The discriminative nature of sparse representation, affords us the opportunity to compare nodules\u27 variations in consecutive time points and to predict malignancy. Experimental validations of the proposed segmentation algorithm have been demonstrated on 542 3-D lung nodule data from the LIDC-IDRI database which includes radiologist delineated nodule boundaries. The effectiveness of the proposed deep learning and dictionary learning architectures for malignancy prediction have been demonstrated on CT data from 370 biopsied subjects collected from the NLST database. Each subject in this database had at least two serial CT scans at two separate time points one year apart. The proposed RNN CAD system achieved an ROC Area Under the Curve (AUC) of 0.87, when validated on CT data from nodules at second sequential time point and 0.83 based on dictionary learning method; however, when nodule shape change and appearance were combined, the classifier performance improved to AUC=0.89

Aprendendo a suprimir não-máximos para aperfeiçoar a detecção de nódulos pulmonares em imagens de CT

Orientador: Alexandre Xavier FalcãoDissertação (mestrado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: O câncer de pulmão é o tipo mais comum de câncer em homens e o terceiro mais comum em mulheres. Devido ao mau prognóstico, o câncer de pulmão é responsável pela maior taxa de mortalidade, atingindo 1,8 milhão de mortes por ano. O diagnóstico e o tratamento nos estágios iniciais podem aumentar as chances de sobrevivência. A tomografia computadorizada (TC) é a modalidade de imagem preferida para detectar e diagnosticar câncer de pulmão, pois fornece imagens 3D do tórax em alta resolução, facilitando a detecção de pequenos nódulos. No entanto, a natureza 3D das imagens dificulta sua análise visual. Como conseqüência, o número de falsos positivos ainda é alto e, mesmo contando com a opinião de vários especialistas, o diagnóstico é frequentemente sujeito a alguma falta de consenso. Os sistemas de Diagnóstico Assistida por Computador (CAD) foram desenvolvidos para solucionar o problema, auxiliando especialistas na tarefa de detecção e classificação mais rápidas e precisas de anormalidades. As técnicas usadas nos sistemas CAD podem ser divididas em dois grupos: sistemas CAD que exploram features de imagem baseados em conhecimento e sistemas CAD que aprendem os features de imagens anotadas, principalmente baseadas em aprendizado profundo por meio de redes neurais convolucionais (CNNs). Na última década, muitos métodos computacionais (sistemas CAD) foram desenvolvidos para auxiliar os médicos na detecção de nódulos pulmonares. Tais métodos são baseados principalmente em CNNs, que alcançaram resultados promissores na detecção precoce de nódulos pulmonares. No entanto, esses métodos geram várias regiões candidatas por nódulo, de modo que um algoritmo de não-máxima supressão (NMS) é necessário para selecionar uma única região por nódulo, eliminando as redundantes. O GossipNet é uma rede neural 1D para NMS, que pode aprender os parâmetros do NMS em vez de confiar nos parâmetros artesanais. No entanto, o GossipNet não tira proveito dos features de imagem para aprender NMS. Neste trabalho, propomos um sistema CAD automatizado para detecção de nódulos pulmonares, que consiste em quatro módulos: pré-processamento, a definição de uma região de interesse (por exemplo, por segmentação pulmonar), detecção de nódulos e a eliminação de candidatos redundantes. Para a segmentação pulmonar, usamos uma abordagem recente baseada em sequências de transformações florestais de imagem (IFTs) denominada ALTIS, fornecendo uma segmentação mais precisa dos pulmões em comparação com o método usado no desafio LUNA16. Para a detecção de nódulos e a eliminação de candidatos redundantes, usamos o 3D Faster R-CNN com ResNet18 para a detecção de regiões candidatas com nódulos e apresentamos FeatureNMS --- uma rede neural que fornece features de imagem adicionais à entrada do GossipNet, que resultam de uma transformação sobre as intensidades de voxel de cada região candidata na imagem da TC. Para validação, usamos o conjunto de dados de desafio LUNA16Abstract: Lung cancer is the most common type of cancer in men and the third most common one in women. Due to poor prognosis, lung cancer is responsible for the largest mortality rate, reaching 1.8 million deaths per year. Diagnosis and treatment at the early stages can increase the chances of survival. Computerized Tomography (CT) is the imaging modality of preference to detect and diagnose lung cancer since it provides high-resolution 3D images of the thorax, facilitating the detection of small nodules. However, the 3D nature of the images makes their visual analysis difficult. As a consequence, the number of false positives is still high and, even by counting on the opinion of multiple specialists, the diagnosis is often subjected to some lack of consensus. Computer-Aided Detection (CAD) systems have been developed to address the problem, assisting to specialists in the task of quicker and more accurate detection and classification of abnormalities. The techniques used in CAD systems may be divided into two groups: CAD systems that explore knowledge-based image features and CAD systems that learn the features from annotated images, mostly based on deep learning through Convolutional Neural Networks (CNNs). In the last decade, many computational methods (CAD systems) have been developed to assist physicians in lung nodule detection. Such methods are mostly based on CNNs, which have achieved promising results in early detection of lung nodules. However, these methods generate several candidate regions per nodule, such that a Non-Maximum Suppression (NMS) algorithm is required to select a single region per nodule while eliminating the redundant ones. GossipNet is a 1D Neural Network (NN) for NMS, which can learn the NMS parameters rather than relying on handcrafted ones. However, GossipNet does not take advantage of image features to learn NMS. In this work, we propose an automated CAD system for lung nodule detection which consists of four modules: pre-processing, the definition of a region of interest (e.g., by lung segmentation), nodule detection, and the elimination of redundant candidates. For lung segmentation, we use a recent approach based on sequences of Image Foresting Transforms (IFTs) named ALTIS providing a more accurate segmentation of the lungs compared to the method used in the LUNA16 challenge. For nodule detection and the elimination of redundant candidates, we use 3D Faster R-CNN with ResNet18 for the detection of candidate regions with nodules and present \emph{FeatureNMS} --- a neural network that provides additional image features to the input of GossipNet, which result from a transformation over the voxel intensities of each candidate region in the CT image. For validation, we use the LUNA16 challenge datasetMestradoCiência da ComputaçãoMestre em Ciência da Computação171063/2017-12014/12236-1CNPQFAPES

Cancer diagnosis using deep learning: A bibliographic review

In this paper, we first describe the basics of the field of cancer diagnosis, which includes steps of cancer diagnosis followed by the typical classification methods used by doctors, providing a historical idea of cancer classification techniques to the readers. These methods include Asymmetry, Border, Color and Diameter (ABCD) method, seven-point detection method, Menzies method, and pattern analysis. They are used regularly by doctors for cancer diagnosis, although they are not considered very efficient for obtaining better performance. Moreover, considering all types of audience, the basic evaluation criteria are also discussed. The criteria include the receiver operating characteristic curve (ROC curve), Area under the ROC curve (AUC), F1 score, accuracy, specificity, sensitivity, precision, dice-coefficient, average accuracy, and Jaccard index. Previously used methods are considered inefficient, asking for better and smarter methods for cancer diagnosis. Artificial intelligence and cancer diagnosis are gaining attention as a way to define better diagnostic tools. In particular, deep neural networks can be successfully used for intelligent image analysis. The basic framework of how this machine learning works on medical imaging is provided in this study, i.e., pre-processing, image segmentation and post-processing. The second part of this manuscript describes the different deep learning techniques, such as convolutional neural networks (CNNs), generative adversarial models (GANs), deep autoencoders (DANs), restricted Boltzmann’s machine (RBM), stacked autoencoders (SAE), convolutional autoencoders (CAE), recurrent neural networks (RNNs), long short-term memory (LTSM), multi-scale convolutional neural network (M-CNN), multi-instance learning convolutional neural network (MIL-CNN). For each technique, we provide Python codes, to allow interested readers to experiment with the cited algorithms on their own diagnostic problems. The third part of this manuscript compiles the successfully applied deep learning models for different types of cancers. Considering the length of the manuscript, we restrict ourselves to the discussion of breast cancer, lung cancer, brain cancer, and skin cancer. The purpose of this bibliographic review is to provide researchers opting to work in implementing deep learning and artificial neural networks for cancer diagnosis a knowledge from scratch of the state-of-the-art achievements

Computer-aided detection of lung nodules: A review

We present an in-depth review and analysis of salient methods for computer-aided detection of lung nodules. We evaluate the current methods for detecting lung nodules using literature searches with selection criteria based on validation dataset types, nodule sizes, numbers of cases, types of nodules, extracted features in traditional feature-based classifiers, sensitivity, and false positives (FP)/scans. Our review shows that current detection systems are often optimized for particular datasets and can detect only one or two types of nodules. We conclude that, in addition to achieving high sensitivity and reduced FP/scans, strategies for detecting lung nodules must detect a variety of nodules with high precision to improve the performances of the radiologists. To the best of our knowledge, ours is the first review of the effectiveness of feature extraction using traditional feature-based classifiers. Moreover, we discuss deep-learning methods in detail and conclude that features must be appropriately selected to improve the overall accuracy of the system. We present an analysis of current schemes and highlight constraints and future research areas

Lung Nodules Classification Using Convolutional Neural Network with Transfer Learning

Healthcare industry plays a vital role in improving daily life. Machine learning and deep neural networks have contributed a lot to benefit various industries nowadays. Agriculture, healthcare, machinery, aviation, management, and even education have all benefited from the development and implementation of machine learning. Deep neural networks provide insight and assistance in improving daily activities. Convolutional neural network (CNN), one of the deep neural network methods, has had a significant impact in the field of computer vision. CNN has long been known for its ability to improve detection and classification in images. With the implementation of deep learning, more deep knowledge can be gathered and help healthcare workers to know more about a patient’s disease. Deep neural networks and machine learning are increasingly being used in healthcare. The benefit they provide in terms of improved detection and classification has a positive impact on healthcare. CNNs are widely used in the detection and classification of imaging tasks like CT and MRI scans. Although CNN has advantages in this industry, the algorithm must be trained with a large number of data sets in order to achieve high accuracy and performance. Large medical datasets are always unavailable due to a variety of factors such as ethical concerns, a scarcity of expert explanatory notes and labelled data, and a general scarcity of disease images. In this paper, lung nodules classification using CNN with transfer learning is proposed to help in classifying benign and malignant lung nodules from CT scan images. The objectives of this study are to pre-process lung nodules data, develop a CNN with transfer learning algorithm, and analyse the effectiveness of CNN with transfer learning compared to standard of other methods. According to the findings of this study, CNN with transfer learning outperformed standard CNN without transfer learning

A proposed methodology for detecting the malignant potential of pulmonary nodules in sarcoma using computed tomographic imaging and artificial intelligence-based models

The presence of lung metastases in patients with primary malignancies is an important criterion for treatment management and prognostication. Computed tomography (CT) of the chest is the preferred method to detect lung metastasis. However, CT has limited efficacy in differentiating metastatic nodules from benign nodules (e.g., granulomas due to tuberculosis) especially at early stages (<5 mm). There is also a significant subjectivity associated in making this distinction, leading to frequent CT follow-ups and additional radiation exposure along with financial and emotional burden to the patients and family. Even 18F-fluoro-deoxyglucose positron emission technology-computed tomography (18F-FDG PET-CT) is not always confirmatory for this clinical problem. While pathological biopsy is the gold standard to demonstrate malignancy, invasive sampling of small lung nodules is often not clinically feasible. Currently, there is no non-invasive imaging technique that can reliably characterize lung metastases. The lung is one of the favored sites of metastasis in sarcomas. Hence, patients with sarcomas, especially from tuberculosis prevalent developing countries, can provide an ideal platform to develop a model to differentiate lung metastases from benign nodules. To overcome the lack of optimal specificity of CT scan in detecting pulmonary metastasis, a novel artificial intelligence (AI)-based protocol is proposed utilizing a combination of radiological and clinical biomarkers to identify lung nodules and characterize it as benign or metastasis. This protocol includes a retrospective cohort of nearly 2,000–2,250 sample nodules (from at least 450 patients) for training and testing and an ambispective cohort of nearly 500 nodules (from 100 patients; 50 patients each from the retrospective and prospective cohort) for validation. Ground-truth annotation of lung nodules will be performed using an in-house-built segmentation tool. Ground-truth labeling of lung nodules (metastatic/benign) will be performed based on histopathological results or baseline and/or follow-up radiological findings along with clinical outcome of the patient. Optimal methods for data handling and statistical analysis are included to develop a robust protocol for early detection and classification of pulmonary metastasis at baseline and at follow-up and identification of associated potential clinical and radiological markers

Domain specific cues improve robustness of deep learning based segmentation of ct volumes

Machine Learning has considerably improved medical image analysis in the past years. Although data-driven approaches are intrinsically adaptive and thus, generic, they often do not perform the same way on data from different imaging modalities. In particular Computed tomography (CT) data poses many challenges to medical image segmentation based on convolutional neural networks (CNNs), mostly due to the broad dynamic range of intensities and the varying number of recorded slices of CT volumes. In this paper, we address these issues with a framework that combines domain-specific data preprocessing and augmentation with state-of-the-art CNN architectures. The focus is not limited to optimise the score, but also to stabilise the prediction performance since this is a mandatory requirement for use in automated and semi-automated workflows in the clinical environment. The framework is validated with an architecture comparison to show CNN architecture-independent effects of our framework functionality. We compare a modified U-Net and a modified Mixed-Scale Dense Network (MS-D Net) to compare dilated convolutions for parallel multi-scale processing to the U-Net approach based on traditional scaling operations. Finally, we propose an ensemble model combining the strengths of different individual methods. The framework performs well on a range of tasks such as liver and kidney segmentation, without significant differences in prediction performance on strongly differing volume sizes and varying slice thickness. Thus our framework is an essential step towards performing robust segmentation of unknown real-world samples

Artificial Intelligence in Image-Based Screening, Diagnostics, and Clinical Care of Cardiopulmonary Diseases

Cardiothoracic and pulmonary diseases are a significant cause of mortality and morbidity worldwide. The COVID-19 pandemic has highlighted the lack of access to clinical care, the overburdened medical system, and the potential of artificial intelligence (AI) in improving medicine. There are a variety of diseases affecting the cardiopulmonary system including lung cancers, heart disease, tuberculosis (TB), etc., in addition to COVID-19-related diseases. Screening, diagnosis, and management of cardiopulmonary diseases has become difficult owing to the limited availability of diagnostic tools and experts, particularly in resource-limited regions. Early screening, accurate diagnosis and staging of these diseases could play a crucial role in treatment and care, and potentially aid in reducing mortality. Radiographic imaging methods such as computed tomography (CT), chest X-rays (CXRs), and echo ultrasound (US) are widely used in screening and diagnosis. Research on using image-based AI and machine learning (ML) methods can help in rapid assessment, serve as surrogates for expert assessment, and reduce variability in human performance. In this Special Issue, “Artificial Intelligence in Image-Based Screening, Diagnostics, and Clinical Care of Cardiopulmonary Diseases”, we have highlighted exemplary primary research studies and literature reviews focusing on novel AI/ML methods and their application in image-based screening, diagnosis, and clinical management of cardiopulmonary diseases. We hope that these articles will help establish the advancements in AI