7 research outputs found
Multimodal Spatial Attention Module for Targeting Multimodal PET-CT Lung Tumor Segmentation
Multimodal positron emission tomography-computed tomography (PET-CT) is used
routinely in the assessment of cancer. PET-CT combines the high sensitivity for
tumor detection with PET and anatomical information from CT. Tumor segmentation
is a critical element of PET-CT but at present, there is not an accurate
automated segmentation method. Segmentation tends to be done manually by
different imaging experts and it is labor-intensive and prone to errors and
inconsistency. Previous automated segmentation methods largely focused on
fusing information that is extracted separately from the PET and CT modalities,
with the underlying assumption that each modality contains complementary
information. However, these methods do not fully exploit the high PET tumor
sensitivity that can guide the segmentation. We introduce a multimodal spatial
attention module (MSAM) that automatically learns to emphasize regions (spatial
areas) related to tumors and suppress normal regions with physiologic
high-uptake. The resulting spatial attention maps are subsequently employed to
target a convolutional neural network (CNN) for segmentation of areas with
higher tumor likelihood. Our MSAM can be applied to common backbone
architectures and trained end-to-end. Our experimental results on two clinical
PET-CT datasets of non-small cell lung cancer (NSCLC) and soft tissue sarcoma
(STS) validate the effectiveness of the MSAM in these different cancer types.
We show that our MSAM, with a conventional U-Net backbone, surpasses the
state-of-the-art lung tumor segmentation approach by a margin of 7.6% in Dice
similarity coefficient (DSC)
ISA-Net: Improved spatial attention network for PET-CT tumor segmentation
Achieving accurate and automated tumor segmentation plays an important role
in both clinical practice and radiomics research. Segmentation in medicine is
now often performed manually by experts, which is a laborious, expensive and
error-prone task. Manual annotation relies heavily on the experience and
knowledge of these experts. In addition, there is much intra- and interobserver
variation. Therefore, it is of great significance to develop a method that can
automatically segment tumor target regions. In this paper, we propose a deep
learning segmentation method based on multimodal positron emission
tomography-computed tomography (PET-CT), which combines the high sensitivity of
PET and the precise anatomical information of CT. We design an improved spatial
attention network(ISA-Net) to increase the accuracy of PET or CT in detecting
tumors, which uses multi-scale convolution operation to extract feature
information and can highlight the tumor region location information and
suppress the non-tumor region location information. In addition, our network
uses dual-channel inputs in the coding stage and fuses them in the decoding
stage, which can take advantage of the differences and complementarities
between PET and CT. We validated the proposed ISA-Net method on two clinical
datasets, a soft tissue sarcoma(STS) and a head and neck tumor(HECKTOR)
dataset, and compared with other attention methods for tumor segmentation. The
DSC score of 0.8378 on STS dataset and 0.8076 on HECKTOR dataset show that
ISA-Net method achieves better segmentation performance and has better
generalization. Conclusions: The method proposed in this paper is based on
multi-modal medical image tumor segmentation, which can effectively utilize the
difference and complementarity of different modes. The method can also be
applied to other multi-modal data or single-modal data by proper adjustment
FSS-2019-nCov:A deep learning architecture for semi-supervised few-shot segmentation of COVID-19 infection
The newly discovered coronavirus (COVID-19) pneumonia is providing major challenges to research in terms of diagnosis and disease quantification. Deep-learning (DL) techniques allow extremely precise image segmentation; yet, they necessitate huge volumes of manually labeled data to be trained in a supervised manner. Few-Shot Learning (FSL) paradigms tackle this issue by learning a novel category from a small number of annotated instances. We present an innovative semi-supervised few-shot segmentation (FSS) approach for efficient segmentation of 2019-nCov infection (FSS-2019-nCov) from only a few amounts of annotated lung CT scans. The key challenge of this study is to provide accurate segmentation of COVID-19 infection from a limited number of annotated instances. For that purpose, we propose a novel dual-path deep-learning architecture for FSS. Every path contains encoder–decoder (E-D) architecture to extract high-level information while maintaining the channel information of COVID-19 CT slices. The E-D architecture primarily consists of three main modules: a feature encoder module, a context enrichment (CE) module, and a feature decoder module. We utilize the pre-trained ResNet34 as an encoder backbone for feature extraction. The CE module is designated by a newly introduced proposed Smoothed Atrous Convolution (SAC) block and Multi-scale Pyramid Pooling (MPP) block. The conditioner path takes the pairs of CT images and their labels as input and produces a relevant knowledge representation that is transferred to the segmentation path to be used to segment the new images. To enable effective collaboration between both paths, we propose an adaptive recombination and recalibration (RR) module that permits intensive knowledge exchange between paths with a trivial increase in computational complexity. The model is extended to multi-class labeling for various types of lung infections. This contribution overcomes the limitation of the lack of large numbers of COVID-19 CT scans. It also provides a general framework for lung disease diagnosis in limited data situations
Multi-Modality Automatic Lung Tumor Segmentation Method Using Deep Learning and Radiomics
Delineation of the tumor volume is the initial and fundamental step in the radiotherapy planning process. The current clinical practice of manual delineation is time-consuming and suffers from observer variability. This work seeks to develop an effective automatic framework to produce clinically usable lung tumor segmentations. First, to facilitate the development and validation of our methodology, an expansive database of planning CTs, diagnostic PETs, and manual tumor segmentations was curated, and an image registration and preprocessing pipeline was established. Then a deep learning neural network was constructed and optimized to utilize dual-modality PET and CT images for lung tumor segmentation. The feasibility of incorporating radiomics and other mechanisms such as a tumor volume-based stratification scheme for training/validation/testing were investigated to improve the segmentation performance. The proposed methodology was evaluated both quantitatively with similarity metrics and clinically with physician reviews. In addition, external validation with an independent database was also conducted. Our work addressed some of the major limitations that restricted clinical applicability of the existing approaches and produced automatic segmentations that were consistent with the manually contoured ground truth and were highly clinically-acceptable according to both the quantitative and clinical evaluations. Both novel approaches of implementing a tumor volume-based training/validation/ testing stratification strategy as well as incorporating voxel-wise radiomics feature images were shown to improve the segmentation performance. The results showed that the proposed method was effective and robust, producing automatic lung tumor segmentations that could potentially improve both the quality and consistency of manual tumor delineation
Deep Learning-based Radiomics Framework for Multi-Modality PET-CT Images
Multimodal positron emission tomography - computed tomography (PET-CT) imaging is widely regarded as the imaging modality of choice for cancer management. This is because PET-CT combines the high sensitivity of PET in detecting regions of abnormal functions and the specificity of CT in depicting the underlying anatomy of where the abnormal functions are occurring.
Radiomics is an emerging research field that enables the extraction and analysis of quantitative features from medical images, providing valuable insights into the underlying pathophysiology that cannot be discerned by the naked eyes. This information is capable of assisting decision-making in clinical practice, leading to better personalised treatment planning, patient outcome prediction, and therapy response assessment.
The aim of this thesis is to propose a new deep learning-based radiomics framework for multimodal PET-CT images. The proposed framework comprises of three methods: 1) a tumour segmentation method via a self-supervision enabled false positive and false negative reduction network; 2) a constrained hierarchical multi-modality feature learning is constructed to predict the patient outcome with multimodal PET-CT images; 3) an automatic neural architecture search method to automatically find the optimal network architecture for both patient outcome prediction and tumour segmentation.
Extensive experiments have been conducted on three datasets, including one public soft-tissue sarcomas dataset, one public challenge dataset, and one in-house lung cancer data. The results demonstrated that the proposed methods obtained better performance in all tasks when compared to the state-of-the-art methods