12,315 research outputs found
Foundational Models in Medical Imaging: A Comprehensive Survey and Future Vision
Foundation models, large-scale, pre-trained deep-learning models adapted to a
wide range of downstream tasks have gained significant interest lately in
various deep-learning problems undergoing a paradigm shift with the rise of
these models. Trained on large-scale dataset to bridge the gap between
different modalities, foundation models facilitate contextual reasoning,
generalization, and prompt capabilities at test time. The predictions of these
models can be adjusted for new tasks by augmenting the model input with
task-specific hints called prompts without requiring extensive labeled data and
retraining. Capitalizing on the advances in computer vision, medical imaging
has also marked a growing interest in these models. To assist researchers in
navigating this direction, this survey intends to provide a comprehensive
overview of foundation models in the domain of medical imaging. Specifically,
we initiate our exploration by providing an exposition of the fundamental
concepts forming the basis of foundation models. Subsequently, we offer a
methodical taxonomy of foundation models within the medical domain, proposing a
classification system primarily structured around training strategies, while
also incorporating additional facets such as application domains, imaging
modalities, specific organs of interest, and the algorithms integral to these
models. Furthermore, we emphasize the practical use case of some selected
approaches and then discuss the opportunities, applications, and future
directions of these large-scale pre-trained models, for analyzing medical
images. In the same vein, we address the prevailing challenges and research
pathways associated with foundational models in medical imaging. These
encompass the areas of interpretability, data management, computational
requirements, and the nuanced issue of contextual comprehension.Comment: The paper is currently in the process of being prepared for
submission to MI
Contrastive Attention for Automatic Chest X-ray Report Generation
Recently, chest X-ray report generation, which aims to automatically generate
descriptions of given chest X-ray images, has received growing research
interests. The key challenge of chest X-ray report generation is to accurately
capture and describe the abnormal regions. In most cases, the normal regions
dominate the entire chest X-ray image, and the corresponding descriptions of
these normal regions dominate the final report. Due to such data bias,
learning-based models may fail to attend to abnormal regions. In this work, to
effectively capture and describe abnormal regions, we propose the Contrastive
Attention (CA) model. Instead of solely focusing on the current input image,
the CA model compares the current input image with normal images to distill the
contrastive information. The acquired contrastive information can better
represent the visual features of abnormal regions. According to the experiments
on the public IU-X-ray and MIMIC-CXR datasets, incorporating our CA into
several existing models can boost their performance across most metrics. In
addition, according to the analysis, the CA model can help existing models
better attend to the abnormal regions and provide more accurate descriptions
which are crucial for an interpretable diagnosis. Specifically, we achieve the
state-of-the-art results on the two public datasets.Comment: Appear in Findings of ACL 2021 (The Joint Conference of the 59th
Annual Meeting of the Association for Computational Linguistics and the 11th
International Joint Conference on Natural Language Processing (ACL-IJCNLP
2021)
Competence-based Multimodal Curriculum Learning for Medical Report Generation
Medical report generation task, which targets to produce long and coherent
descriptions of medical images, has attracted growing research interests
recently. Different from the general image captioning tasks, medical report
generation is more challenging for data-driven neural models. This is mainly
due to 1) the serious data bias and 2) the limited medical data. To alleviate
the data bias and make best use of available data, we propose a
Competence-based Multimodal Curriculum Learning framework (CMCL). Specifically,
CMCL simulates the learning process of radiologists and optimizes the model in
a step by step manner. Firstly, CMCL estimates the difficulty of each training
instance and evaluates the competence of current model; Secondly, CMCL selects
the most suitable batch of training instances considering current model
competence. By iterating above two steps, CMCL can gradually improve the
model's performance. The experiments on the public IU-Xray and MIMIC-CXR
datasets show that CMCL can be incorporated into existing models to improve
their performance.Comment: Accepted by ACL 2021 (Oral
Deep reproductive feature generation framework for the diagnosis of COVID-19 and viral pneumonia using chest X-ray images
The rapid and accurate detection of COVID-19 cases is critical for timely
treatment and preventing the spread of the disease. In this study, a two-stage
feature extraction framework using eight state-of-the-art pre-trained deep
Convolutional Neural Networks (CNNs) and an autoencoder is proposed to
determine the health conditions of patients (COVID-19, Normal, Viral Pneumonia)
based on chest X-rays. The X-ray scans are divided into four equally sized
sections and analyzed by deep pre-trained CNNs. Subsequently, an autoencoder
with three hidden layers is trained to extract reproductive features from the
concatenated ouput of CNNs. To evaluate the performance of the proposed
framework, three different classifiers, which are single-layer perceptron
(SLP), multi-layer perceptron (MLP), and support vector machine (SVM) are used.
Furthermore, the deep CNN architectures are used to create benchmark models and
trained on the same dataset for comparision. The proposed framework outperforms
other frameworks wih pre-trained feature extractors in binary classification
and shows competitive results in three-class classification. The proposed
methodology is task-independent and suitable for addressing various problems.
The results show that the discriminative features are a subset of the
reproductive features, suggesting that extracting task-independent features is
superior to the extraction only task-based features. The flexibility and
task-independence of the reproductive features make the conceptive information
approach more favorable. The proposed methodology is novel and shows promising
results for analyzing medical image data
Data-Centric Foundation Models in Computational Healthcare: A Survey
The advent of foundation models (FMs) as an emerging suite of AI techniques
has struck a wave of opportunities in computational healthcare. The interactive
nature of these models, guided by pre-training data and human instructions, has
ignited a data-centric AI paradigm that emphasizes better data
characterization, quality, and scale. In healthcare AI, obtaining and
processing high-quality clinical data records has been a longstanding
challenge, ranging from data quantity, annotation, patient privacy, and ethics.
In this survey, we investigate a wide range of data-centric approaches in the
FM era (from model pre-training to inference) towards improving the healthcare
workflow. We discuss key perspectives in AI security, assessment, and alignment
with human values. Finally, we offer a promising outlook of FM-based analytics
to enhance the performance of patient outcome and clinical workflow in the
evolving landscape of healthcare and medicine. We provide an up-to-date list of
healthcare-related foundation models and datasets at
https://github.com/Yunkun-Zhang/Data-Centric-FM-Healthcare
Generative models improve fairness of medical classifiers under distribution shifts
A ubiquitous challenge in machine learning is the problem of domain
generalisation. This can exacerbate bias against groups or labels that are
underrepresented in the datasets used for model development. Model bias can
lead to unintended harms, especially in safety-critical applications like
healthcare. Furthermore, the challenge is compounded by the difficulty of
obtaining labelled data due to high cost or lack of readily available domain
expertise. In our work, we show that learning realistic augmentations
automatically from data is possible in a label-efficient manner using
generative models. In particular, we leverage the higher abundance of
unlabelled data to capture the underlying data distribution of different
conditions and subgroups for an imaging modality. By conditioning generative
models on appropriate labels, we can steer the distribution of synthetic
examples according to specific requirements. We demonstrate that these learned
augmentations can surpass heuristic ones by making models more robust and
statistically fair in- and out-of-distribution. To evaluate the generality of
our approach, we study 3 distinct medical imaging contexts of varying
difficulty: (i) histopathology images from a publicly available generalisation
benchmark, (ii) chest X-rays from publicly available clinical datasets, and
(iii) dermatology images characterised by complex shifts and imaging
conditions. Complementing real training samples with synthetic ones improves
the robustness of models in all three medical tasks and increases fairness by
improving the accuracy of diagnosis within underrepresented groups. This
approach leads to stark improvements OOD across modalities: 7.7% prediction
accuracy improvement in histopathology, 5.2% in chest radiology with 44.6%
lower fairness gap and a striking 63.5% improvement in high-risk sensitivity
for dermatology with a 7.5x reduction in fairness gap
Recommended from our members
Toward fairness in artificial intelligence for medical image analysis: Identification and mitigation of potential biases in the roadmap from data collection to model deployment
Purpose: To recognize and address various sources of bias essential for algorithmic fairness and trustworthiness and to contribute to a just and equitable deployment of AI in medical imaging, there is an increasing interest in developing medical imaging-based machine learning methods, also known as medical imaging artificial intelligence (AI), for the detection, diagnosis, prognosis, and risk assessment of disease with the goal of clinical implementation. These tools are intended to help improve traditional human decision-making in medical imaging. However, biases introduced in the steps toward clinical deployment may impede their intended function, potentially exacerbating inequities. Specifically, medical imaging AI can propagate or amplify biases introduced in the many steps from model inception to deployment, resulting in a systematic difference in the treatment of different groups. Approach: Our multi-institutional team included medical physicists, medical imaging artificial intelligence/machine learning (AI/ML) researchers, experts in AI/ML bias, statisticians, physicians, and scientists from regulatory bodies. We identified sources of bias in AI/ML, mitigation strategies for these biases, and developed recommendations for best practices in medical imaging AI/ML development. Results: Five main steps along the roadmap of medical imaging AI/ML were identified: (1) data collection, (2) data preparation and annotation, (3) model development, (4) model evaluation, and (5) model deployment. Within these steps, or bias categories, we identified 29 sources of potential bias, many of which can impact multiple steps, as well as mitigation strategies. Conclusions: Our findings provide a valuable resource to researchers, clinicians, and the public at large.</p
- …