A Survey on Deep Learning in Medical Image Analysis

Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks and provide concise overviews of studies per application area. Open challenges and directions for future research are discussed.Comment: Revised survey includes expanded discussion section and reworked introductory section on common deep architectures. Added missed papers from before Feb 1st 201

Towards generalizable machine learning models for computer-aided diagnosis in medicine

Hidden stratification represents a phenomenon in which a training dataset contains unlabeled (hidden) subsets of cases that may affect machine learning model performance. Machine learning models that ignore the hidden stratification phenomenon--despite promising overall performance measured as accuracy and sensitivity--often fail at predicting the low prevalence cases, but those cases remain important. In the medical domain, patients with diseases are often less common than healthy patients, and a misdiagnosis of a patient with a disease can have significant clinical impacts. Therefore, to build a robust and trustworthy CAD system and a reliable treatment effect prediction model, we cannot only pursue machine learning models with high overall accuracy, but we also need to discover any hidden stratification in the data and evaluate the proposing machine learning models with respect to both overall performance and the performance on certain subsets (groups) of the data, such as the ‘worst group’. In this study, I investigated three approaches for data stratification: a novel algorithmic deep learning (DL) approach that learns similarities among cases and two schema completion approaches that utilize domain expert knowledge. I further proposed an innovative way to integrate the discovered latent groups into the loss functions of DL models to allow for better model generalizability under the domain shift scenario caused by the data heterogeneity. My results on lung nodule Computed Tomography (CT) images and breast cancer histopathology images demonstrate that learning homogeneous groups within heterogeneous data significantly improves the performance of the computer-aided diagnosis (CAD) system, particularly for low-prevalence or worst-performing cases. This study emphasizes the importance of discovering and learning the latent stratification within the data, as it is a critical step towards building ML models that are generalizable and reliable. Ultimately, this discovery can have a profound impact on clinical decision-making, particularly for low-prevalence cases

Cancer diagnosis using deep learning: A bibliographic review

In this paper, we first describe the basics of the field of cancer diagnosis, which includes steps of cancer diagnosis followed by the typical classification methods used by doctors, providing a historical idea of cancer classification techniques to the readers. These methods include Asymmetry, Border, Color and Diameter (ABCD) method, seven-point detection method, Menzies method, and pattern analysis. They are used regularly by doctors for cancer diagnosis, although they are not considered very efficient for obtaining better performance. Moreover, considering all types of audience, the basic evaluation criteria are also discussed. The criteria include the receiver operating characteristic curve (ROC curve), Area under the ROC curve (AUC), F1 score, accuracy, specificity, sensitivity, precision, dice-coefficient, average accuracy, and Jaccard index. Previously used methods are considered inefficient, asking for better and smarter methods for cancer diagnosis. Artificial intelligence and cancer diagnosis are gaining attention as a way to define better diagnostic tools. In particular, deep neural networks can be successfully used for intelligent image analysis. The basic framework of how this machine learning works on medical imaging is provided in this study, i.e., pre-processing, image segmentation and post-processing. The second part of this manuscript describes the different deep learning techniques, such as convolutional neural networks (CNNs), generative adversarial models (GANs), deep autoencoders (DANs), restricted Boltzmann’s machine (RBM), stacked autoencoders (SAE), convolutional autoencoders (CAE), recurrent neural networks (RNNs), long short-term memory (LTSM), multi-scale convolutional neural network (M-CNN), multi-instance learning convolutional neural network (MIL-CNN). For each technique, we provide Python codes, to allow interested readers to experiment with the cited algorithms on their own diagnostic problems. The third part of this manuscript compiles the successfully applied deep learning models for different types of cancers. Considering the length of the manuscript, we restrict ourselves to the discussion of breast cancer, lung cancer, brain cancer, and skin cancer. The purpose of this bibliographic review is to provide researchers opting to work in implementing deep learning and artificial neural networks for cancer diagnosis a knowledge from scratch of the state-of-the-art achievements

Informative sample generation using class aware generative adversarial networks for classification of chest Xrays

Training robust deep learning (DL) systems for disease detection from medical images is challenging due to limited images covering different disease types and severity. The problem is especially acute, where there is a severe class imbalance. We propose an active learning (AL) framework to select most informative samples for training our model using a Bayesian neural network. Informative samples are then used within a novel class aware generative adversarial network (CAGAN) to generate realistic chest xray images for data augmentation by transferring characteristics from one class label to another. Experiments show our proposed AL framework is able to achieve state-of-the-art performance by using about $35\%$ of the full dataset, thus saving significant time and effort over conventional methods

Deep Learning for Lung Cancer Detection: An Analysis of the Effects of Imperfect Data and Model Biases

Lung cancer is the cancer with the highest mortality as it is usually diagnosed in later stages when treatment options are limited. The most promising solution to reducing the burden associated with lung cancer is screening so that signs of cancer may be detected while still in the early stages. The National Lung Screening Trial (NLST) has shown that the use of low-dose Computed Tomography (CT) for screening instead of chest radiography led to a reduction of 20% in lung cancer mortality in high-risk patients. The introduction of screening programmes will produce a large volume of thoracic CT scans that will need to be processed and assessed by expert radiologists. In this thesis, the aim is to leverage machine learning, and specifically deep learning techniques, for the detection of lung cancer. While the detection of pulmonary nodules can be considered as a mostly solved problem, the characterisation of the nodules still remains a challenging task.Initially, this thesis explores how Convolutional Neural Network (CNN) architectures perform on pulmonary nodule characterisation tasks, such as spiculation and malignancy classification, by leveraging the publicly available and widely used Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI). The analysis delves deeper into the learnt latent representations to provide valuable insights about CNNs. The findings indicate the presence of biases with a strong inter-connection of size and malignancy. This correlation, however, while not spurious, is not the only cause for the malignant nature of a pulmonary nodule.To uncover the reasons behind the presence of such biases, the thesis then branches out in two directions in an attempt to understand whether the short-comings occur from the data or the models. The first direction focuses on LIDC-IDRI and sheds new light on the problematic design choices in prior works, which are aggregating multiple annotations to extract nodule labels. The second direction introduces a synthetic dataset with fully controllable modes of variation to explore the features that CNN architectures learn under different loss functions and learning schemes, such as contrastive learning.Having identified that many issues relating to biases in computational methods for lung nodule analysis arise from the data and not the model, the last part of this thesis turns to the NLST dataset, which contains biopsy-confirmed ground truth labels. However, the lack of consistency in the design of lung cancer datasets and primarily the absence of nodule-level annotations hampers the direct transfer of methods developed for LIDC-IDRI. To mitigate these issues, multiple instance learning and weak annotations are explored, in order to perform patient-level cancer classification.Overall, this thesis focuses on representation learning for pulmonary nodule characterisation and highlights its limitations, which stem from imperfect data and inconsistencies in the dataset generation process