38 research outputs found
Relational Learning between Multiple Pulmonary Nodules via Deep Set Attention Transformers
Diagnosis and treatment of multiple pulmonary nodules are clinically
important but challenging. Prior studies on nodule characterization use
solitary-nodule approaches on multiple nodular patients, which ignores the
relations between nodules. In this study, we propose a multiple instance
learning (MIL) approach and empirically prove the benefit to learn the
relations between multiple nodules. By treating the multiple nodules from a
same patient as a whole, critical relational information between
solitary-nodule voxels is extracted. To our knowledge, it is the first study to
learn the relations between multiple pulmonary nodules. Inspired by recent
advances in natural language processing (NLP) domain, we introduce a
self-attention transformer equipped with 3D CNN, named {NoduleSAT}, to replace
typical pooling-based aggregation in multiple instance learning. Extensive
experiments on lung nodule false positive reduction on LUNA16 database, and
malignancy classification on LIDC-IDRI database, validate the effectiveness of
the proposed method.Comment: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI
2020
Improving diagnosis and prognosis of lung cancer using vision transformers: A scoping review
Vision transformer-based methods are advancing the field of medical
artificial intelligence and cancer imaging, including lung cancer applications.
Recently, many researchers have developed vision transformer-based AI methods
for lung cancer diagnosis and prognosis. This scoping review aims to identify
the recent developments on vision transformer-based AI methods for lung cancer
imaging applications. It provides key insights into how vision transformers
complemented the performance of AI and deep learning methods for lung cancer.
Furthermore, the review also identifies the datasets that contributed to
advancing the field. Of the 314 retrieved studies, this review included 34
studies published from 2020 to 2022. The most commonly addressed task in these
studies was the classification of lung cancer types, such as lung squamous cell
carcinoma versus lung adenocarcinoma, and identifying benign versus malignant
pulmonary nodules. Other applications included survival prediction of lung
cancer patients and segmentation of lungs. The studies lacked clear strategies
for clinical transformation. SWIN transformer was a popular choice of the
researchers; however, many other architectures were also reported where vision
transformer was combined with convolutional neural networks or UNet model. It
can be concluded that vision transformer-based models are increasingly in
popularity for developing AI methods for lung cancer applications. However,
their computational complexity and clinical relevance are important factors to
be considered for future research work. This review provides valuable insights
for researchers in the field of AI and healthcare to advance the
state-of-the-art in lung cancer diagnosis and prognosis. We provide an
interactive dashboard on lung-cancer.onrender.com/.Comment: submitted to BMC Medical Imaging journa
Hierarchical Classification of Pulmonary Lesions: A Large-Scale Radio-Pathomics Study
Diagnosis of pulmonary lesions from computed tomography (CT) is important but
challenging for clinical decision making in lung cancer related diseases. Deep
learning has achieved great success in computer aided diagnosis (CADx) area for
lung cancer, whereas it suffers from label ambiguity due to the difficulty in
the radiological diagnosis. Considering that invasive pathological analysis
serves as the clinical golden standard of lung cancer diagnosis, in this study,
we solve the label ambiguity issue via a large-scale radio-pathomics dataset
containing 5,134 radiological CT images with pathologically confirmed labels,
including cancers (e.g., invasive/non-invasive adenocarcinoma, squamous
carcinoma) and non-cancer diseases (e.g., tuberculosis, hamartoma). This
retrospective dataset, named Pulmonary-RadPath, enables development and
validation of accurate deep learning systems to predict invasive pathological
labels with a non-invasive procedure, i.e., radiological CT scans. A
three-level hierarchical classification system for pulmonary lesions is
developed, which covers most diseases in cancer-related diagnosis. We explore
several techniques for hierarchical classification on this dataset, and propose
a Leaky Dense Hierarchy approach with proven effectiveness in experiments. Our
study significantly outperforms prior arts in terms of data scales (6x larger),
disease comprehensiveness and hierarchies. The promising results suggest the
potentials to facilitate precision medicine.Comment: MICCAI 2020 (Early Accepted
MIA-Prognosis: A Deep Learning Framework to Predict Therapy Response
Predicting clinical outcome is remarkably important but challenging. Research
efforts have been paid on seeking significant biomarkers associated with the
therapy response or/and patient survival. However, these biomarkers are
generally costly and invasive, and possibly dissatifactory for novel therapy.
On the other hand, multi-modal, heterogeneous, unaligned temporal data is
continuously generated in clinical practice. This paper aims at a unified deep
learning approach to predict patient prognosis and therapy response, with
easily accessible data, e.g., radiographics, laboratory and clinical
information. Prior arts focus on modeling single data modality, or ignore the
temporal changes. Importantly, the clinical time series is asynchronous in
practice, i.e., recorded with irregular intervals. In this study, we formalize
the prognosis modeling as a multi-modal asynchronous time series classification
task, and propose a MIA-Prognosis framework with Measurement, Intervention and
Assessment (MIA) information to predict therapy response, where a Simple
Temporal Attention (SimTA) module is developed to process the asynchronous time
series. Experiments on synthetic dataset validate the superiory of SimTA over
standard RNN-based approaches. Furthermore, we experiment the proposed method
on an in-house, retrospective dataset of real-world non-small cell lung cancer
patients under anti-PD-1 immunotherapy. The proposed method achieves promising
performance on predicting the immunotherapy response. Notably, our predictive
model could further stratify low-risk and high-risk patients in terms of
long-term survival.Comment: MICCAI 2020 (Early Accepted; Student Travel Award
Is attention all you need in medical image analysis? A review
Medical imaging is a key component in clinical diagnosis, treatment planning
and clinical trial design, accounting for almost 90% of all healthcare data.
CNNs achieved performance gains in medical image analysis (MIA) over the last
years. CNNs can efficiently model local pixel interactions and be trained on
small-scale MI data. The main disadvantage of typical CNN models is that they
ignore global pixel relationships within images, which limits their
generalisation ability to understand out-of-distribution data with different
'global' information. The recent progress of Artificial Intelligence gave rise
to Transformers, which can learn global relationships from data. However, full
Transformer models need to be trained on large-scale data and involve
tremendous computational complexity. Attention and Transformer compartments
(Transf/Attention) which can well maintain properties for modelling global
relationships, have been proposed as lighter alternatives of full Transformers.
Recently, there is an increasing trend to co-pollinate complementary
local-global properties from CNN and Transf/Attention architectures, which led
to a new era of hybrid models. The past years have witnessed substantial growth
in hybrid CNN-Transf/Attention models across diverse MIA problems. In this
systematic review, we survey existing hybrid CNN-Transf/Attention models,
review and unravel key architectural designs, analyse breakthroughs, and
evaluate current and future opportunities as well as challenges. We also
introduced a comprehensive analysis framework on generalisation opportunities
of scientific and clinical impact, based on which new data-driven domain
generalisation and adaptation methods can be stimulated
KD_ConvNeXt: knowledge distillation-based image classification of lung tumor surgical specimen sections
Introduction: Lung cancer is currently among the most prevalent and lethal cancers in the world in terms of incidence and fatality rates. In clinical practice, identifying the specific subtypes of lung cancer is essential in diagnosing and treating lung lesions.Methods: This paper aims to collect histopathological section images of lung tumor surgical specimens to construct a clinical dataset for researching and addressing the classification problem of specific subtypes of lung tumors. Our method proposes a teacher-student network architecture based on a knowledge distillation mechanism for the specific subtype classification of lung tumor histopathological section images to assist clinical applications, namely KD_ConvNeXt. The proposed approach enables the student network (ConvNeXt) to extract knowledge from the intermediate feature layers of the teacher network (Swin Transformer), improving the feature extraction and fitting capabilities of ConvNeXt. Meanwhile, Swin Transformer provides soft labels containing information about the distribution of images in various categories, making the model focused more on the information carried by types with smaller sample sizes while training.Results: This work has designed many experiments on a clinical lung tumor image dataset, and the KD_ConvNeXt achieved a superior classification accuracy of 85.64% and an F1-score of 0.7717 compared with other advanced image classification method
Data efficient deep learning for medical image analysis: A survey
The rapid evolution of deep learning has significantly advanced the field of
medical image analysis. However, despite these achievements, the further
enhancement of deep learning models for medical image analysis faces a
significant challenge due to the scarcity of large, well-annotated datasets. To
address this issue, recent years have witnessed a growing emphasis on the
development of data-efficient deep learning methods. This paper conducts a
thorough review of data-efficient deep learning methods for medical image
analysis. To this end, we categorize these methods based on the level of
supervision they rely on, encompassing categories such as no supervision,
inexact supervision, incomplete supervision, inaccurate supervision, and only
limited supervision. We further divide these categories into finer
subcategories. For example, we categorize inexact supervision into multiple
instance learning and learning with weak annotations. Similarly, we categorize
incomplete supervision into semi-supervised learning, active learning, and
domain-adaptive learning and so on. Furthermore, we systematically summarize
commonly used datasets for data efficient deep learning in medical image
analysis and investigate future research directions to conclude this survey.Comment: Under Revie