4 research outputs found
Multi-dimension unified Swin Transformer for 3D Lesion Segmentation in Multiple Anatomical Locations
In oncology research, accurate 3D segmentation of lesions from CT scans is
essential for the modeling of lesion growth kinetics. However, following the
RECIST criteria, radiologists routinely only delineate each lesion on the axial
slice showing the largest transverse area, and delineate a small number of
lesions in 3D for research purposes. As a result, we have plenty of unlabeled
3D volumes and labeled 2D images, and scarce labeled 3D volumes, which makes
training a deep-learning 3D segmentation model a challenging task. In this
work, we propose a novel model, denoted a multi-dimension unified Swin
transformer (MDU-ST), for 3D lesion segmentation. The MDU-ST consists of a
Shifted-window transformer (Swin-transformer) encoder and a convolutional
neural network (CNN) decoder, allowing it to adapt to 2D and 3D inputs and
learn the corresponding semantic information in the same encoder. Based on this
model, we introduce a three-stage framework: 1) leveraging large amount of
unlabeled 3D lesion volumes through self-supervised pretext tasks to learn the
underlying pattern of lesion anatomy in the Swin-transformer encoder; 2)
fine-tune the Swin-transformer encoder to perform 2D lesion segmentation with
2D RECIST slices to learn slice-level segmentation information; 3) further
fine-tune the Swin-transformer encoder to perform 3D lesion segmentation with
labeled 3D volumes. The network's performance is evaluated by the Dice
similarity coefficient (DSC) and Hausdorff distance (HD) using an internal 3D
lesion dataset with 593 lesions extracted from multiple anatomical locations.
The proposed MDU-ST demonstrates significant improvement over the competing
models. The proposed method can be used to conduct automated 3D lesion
segmentation to assist radiomics and tumor growth modeling studies. This paper
has been accepted by the IEEE International Symposium on Biomedical Imaging
(ISBI) 2023
The role of deep learning in structural and functional lung imaging
Background: Structural and functional lung imaging are critical components of pulmonary patient care. Image analysis methods, such as image segmentation, applied to structural and functional lung images, have significant benefits for patients with lung pathologies, including the computation of clinical biomarkers. Traditionally, machine learning (ML) approaches, such as clustering, and computational modelling techniques, such as CT-ventilation imaging, have been used for segmentation and synthesis, respectively. Deep learning (DL) has shown promise in medical image analysis tasks, often outperforming alternative methods.
Purpose: To address the hypothesis that DL can outperform conventional ML and classical image analysis methods for the segmentation and synthesis of structural and functional lung imaging via:
i. development and comparison of 3D convolutional neural networks (CNNs) for the segmentation of ventilated lung using hyperpolarised (HP) gas MRI.
ii. development of a generalisable, multi-centre CNN for segmentation of the lung cavity using 1H-MRI.
iii. the proposal of a framework for estimating the lung cavity in the spatial domain of HP gas MRI.
iv. development of a workflow to synthesise HP gas MRI from multi-inflation, non-contrast CT.
v. the proposal of a framework for the synthesis of fully-volumetric HP gas MRI ventilation from a large, diverse dataset of non-contrast, multi-inflation 1H-MRI scans.
Methods:
i. A 3D CNN-based method for the segmentation of ventilated lung using HP gas MRI was developed and CNN parameters, such as architecture, loss function and pre-processing were optimised.
ii. A 3D CNN trained on a multi-acquisition dataset and validated on data from external centres was compared with a 2D alternative for the segmentation of the lung cavity using 1H-MRI.
iii. A dual-channel, multi-modal segmentation framework was compared to single-channel approaches for estimation of the lung cavity in the domain of HP gas MRI.
iv. A hybrid data-driven and model-based approach for the synthesis of HP gas MRI ventilation from CT was compared to approaches utilising DL or computational modelling alone.
v. A physics-constrained, multi-channel framework for the synthesis of fully-volumetric ventilation surrogates from 1H-MRI was validated using five-fold cross-validation and an external test data set.
Results:
i. The 3D CNN, developed via parameterisation experiments, accurately segmented ventilation scans and outperformed conventional ML methods.
ii. The 3D CNN produced more accurate segmentations than its 2D analogues for the segmentation of the lung cavity, exhibiting minimal variation in performance between centres, vendors and acquisitions.
iii. Dual-channel, multi-modal approaches generate significant improvements compared to methods which use a single imaging modality for the estimation of the lung cavity.
iv. The hybrid approach produced synthetic ventilation scans which correlate with HP gas MRI.
v. The physics-constrained, 3D multi-channel synthesis framework outperformed approaches which did not integrate computational modelling, demonstrating generalisability to external data.
Conclusion: DL approaches demonstrate the ability to segment and synthesise lung MRI across a range of modalities and pulmonary pathologies. These methods outperform computational modelling and classical ML approaches, reducing the time required to adequately edit segmentations and improving the modelling of synthetic ventilation, which may facilitate the clinical translation of DL in structural and functional lung imaging