Towards automatic pulmonary nodule management in lung cancer screening with deep learning

The introduction of lung cancer screening programs will produce an unprecedented amount of chest CT scans in the near future, which radiologists will have to read in order to decide on a patient follow-up strategy. According to the current guidelines, the workup of screen-detected nodules strongly relies on nodule size and nodule type. In this paper, we present a deep learning system based on multi-stream multi-scale convolutional networks, which automatically classifies all nodule types relevant for nodule workup. The system processes raw CT data containing a nodule without the need for any additional information such as nodule segmentation or nodule size and learns a representation of 3D data by analyzing an arbitrary number of 2D views of a given nodule. The deep learning system was trained with data from the Italian MILD screening trial and validated on an independent set of data from the Danish DLCST screening trial. We analyze the advantage of processing nodules at multiple scales with a multi-stream convolutional network architecture, and we show that the proposed deep learning system achieves performance at classifying nodule type that surpasses the one of classical machine learning approaches and is within the inter-observer variability among four experienced human observers.Comment: Published on Scientific Report

Deep Learning of Unified Region, Edge, and Contour Models for Automated Image Segmentation

Image segmentation is a fundamental and challenging problem in computer vision with applications spanning multiple areas, such as medical imaging, remote sensing, and autonomous vehicles. Recently, convolutional neural networks (CNNs) have gained traction in the design of automated segmentation pipelines. Although CNN-based models are adept at learning abstract features from raw image data, their performance is dependent on the availability and size of suitable training datasets. Additionally, these models are often unable to capture the details of object boundaries and generalize poorly to unseen classes. In this thesis, we devise novel methodologies that address these issues and establish robust representation learning frameworks for fully-automatic semantic segmentation in medical imaging and mainstream computer vision. In particular, our contributions include (1) state-of-the-art 2D and 3D image segmentation networks for computer vision and medical image analysis, (2) an end-to-end trainable image segmentation framework that unifies CNNs and active contour models with learnable parameters for fast and robust object delineation, (3) a novel approach for disentangling edge and texture processing in segmentation networks, and (4) a novel few-shot learning model in both supervised settings and semi-supervised settings where synergies between latent and image spaces are leveraged to learn to segment images given limited training data.Comment: PhD dissertation, UCLA, 202

From Fully-Supervised, Single-Task to Scarcely-Supervised, Multi-Task Deep Learning for Medical Image Analysis

Image analysis based on machine learning has gained prominence with the advent of deep learning, particularly in medical imaging. To be effective in addressing challenging image analysis tasks, however, conventional deep neural networks require large corpora of annotated training data, which are unfortunately scarce in the medical domain, thus often rendering fully-supervised learning strategies ineffective.This thesis devises for use in a variety of medical image analysis applications a series of novel deep learning methods, ranging from fully-supervised, single-task learning to scarcely-supervised, multi-task learning that makes efficient use of annotated training data. Specifically, its main contributions include (1) fully-supervised, single-task learning for the segmentation of pulmonary lobes from chest CT scans and the analysis of scoliosis from spine X-ray images; (2) supervised, single-task, domain-generalized pulmonary segmentation in chest X-ray images and retinal vasculature segmentation in fundoscopic images; (3) largely-unsupervised, multiple-task learning via deep generative modeling for the joint synthesis and classification of medical image data; and (4) partly-supervised, multiple-task learning for the combined segmentation and classification of chest and spine X-ray images

Pulmonary fissure integrity and collateral ventilation in COPD patients

Purpose: To investigate whether the integrity (completeness) of pulmonary fissures affects pulmonary function in patients with chronic obstructive pulmonary disease (COPD). Materials and Methods: A dataset consisting of 573 CT exams acquired on different subjects was collected from a COPD study. According to the global initiative for chronic obstructive lung disease (GOLD) criteria, these subjects (examinations) were classified into five different subgroups, namely non-COPD (222 subjects), GOLD-I (83 subjects), GOLD-II (141 subjects), GOLD-III (63 subjects), and GOLD-IV (64 subjects), in terms of disease severity. An available computer tool was used to aid in an objective and efficient quantification of fissure integrity. The correlations between fissure integrity, and pulmonary functions (e.g., FEV1, and FEV1/FVC) and COPD severity were assessed using Pearson and Spearman's correlation coefficients, respectively. Results: For the five sub-groups ranging from non-COPD to GOLD-IV, the average integrities of the right oblique fissure (ROF) were 81.8%, 82.4%, 81.8%, 82.8%, and 80.2%, respectively; the average integrities of the right horizontal fissure (RHF) were 62.6%, 61.8%, 62.1%, 62.2%, and 62.3%, respectively; the average integrities of the left oblique fissure (LOF) were 82.0%, 83.2%, 81.7%, 82.0%, and 78.4%, respectively; and the average integrities of all fissures in the entire lung were 78.0%, 78.6%, 78.1%, 78.5%, and 76.4%, respectively. Their Pearson correlation coefficients with FEV1 and FE1/FVC range from 0.027 to 0.248 with p values larger than 0.05. Their Spearman correlation coefficients with COPD severity except GOLD-IV range from -0.013 to -0.073 with p values larger than 0.08. Conclusion: There is no significant difference in fissure integrity for patients with different levels of disease severity, suggesting that the development of COPD does not change the completeness of pulmonary fissures and incomplete fissures alone may not contribute to the collateral ventilation. © 2014 Pu et al

Comparing algorithms for automated vessel segmentation in computed tomography scans of the lung: the VESSEL12 study

The VESSEL12 (VESsel SEgmentation in the Lung) challenge objectively compares the performance of different algorithms to identify vessels in thoracic computed tomography (CT) scans. Vessel segmentation is fundamental in computer aided processing of data generated by 3D imaging modalities. As manual vessel segmentation is prohibitively time consuming, any real world application requires some form of automation. Several approaches exist for automated vessel segmentation, but judging their relative merits is difficult due to a lack of standardized evaluation. We present an annotated reference dataset containing 20 CT scans and propose nine categories to perform a comprehensive evaluation of vessel segmentation algorithms from both academia and industry. Twenty algorithms participated in the VESSEL12 challenge, held at International Symposium on Biomedical Imaging (ISBI) 2012. All results have been published at the VESSEL12 website http://vessel12.grand-challenge.org. The challenge remains ongoing and open to new participants. Our three contributions are: (1) an annotated reference dataset available online for evaluation of new algorithms; (2) a quantitative scoring system for objective comparison of algorithms; and (3) performance analysis of the strengths and weaknesses of the various vessel segmentation methods in the presence of various lung diseases.Rudyanto, RD.; Kerkstra, S.; Van Rikxoort, EM.; Fetita, C.; Brillet, P.; Lefevre, C.; Xue, W.... (2014). Comparing algorithms for automated vessel segmentation in computed tomography scans of the lung: the VESSEL12 study. Medical Image Analysis. 18(7):1217-1232. doi:10.1016/j.media.2014.07.003S1217123218