1,595 research outputs found
Multi-scale Transformer Network with Edge-aware Pre-training for Cross-Modality MR Image Synthesis
Cross-modality magnetic resonance (MR) image synthesis can be used to
generate missing modalities from given ones. Existing (supervised learning)
methods often require a large number of paired multi-modal data to train an
effective synthesis model. However, it is often challenging to obtain
sufficient paired data for supervised training. In reality, we often have a
small number of paired data while a large number of unpaired data. To take
advantage of both paired and unpaired data, in this paper, we propose a
Multi-scale Transformer Network (MT-Net) with edge-aware pre-training for
cross-modality MR image synthesis. Specifically, an Edge-preserving Masked
AutoEncoder (Edge-MAE) is first pre-trained in a self-supervised manner to
simultaneously perform 1) image imputation for randomly masked patches in each
image and 2) whole edge map estimation, which effectively learns both
contextual and structural information. Besides, a novel patch-wise loss is
proposed to enhance the performance of Edge-MAE by treating different masked
patches differently according to the difficulties of their respective
imputations. Based on this proposed pre-training, in the subsequent fine-tuning
stage, a Dual-scale Selective Fusion (DSF) module is designed (in our MT-Net)
to synthesize missing-modality images by integrating multi-scale features
extracted from the encoder of the pre-trained Edge-MAE. Further, this
pre-trained encoder is also employed to extract high-level features from the
synthesized image and corresponding ground-truth image, which are required to
be similar (consistent) in the training. Experimental results show that our
MT-Net achieves comparable performance to the competing methods even using
of all available paired data. Our code will be publicly available at
https://github.com/lyhkevin/MT-Net.Comment: 13 pages, 15 figure
A Novel Confidence Induced Class Activation Mapping for MRI Brain Tumor Segmentation
Magnetic resonance imaging (MRI) is a commonly used technique for brain tumor
segmentation, which is critical for evaluating patients and planning treatment.
To make the labeling process less laborious and dependent on expertise,
weakly-supervised semantic segmentation (WSSS) methods using class activation
mapping (CAM) have been proposed. However, current CAM-based WSSS methods
generate the object localization map using internal neural network information,
such as gradient or trainable parameters, which can lead to suboptimal
solutions. To address these issues, we propose the confidence-induced CAM
(Cfd-CAM), which calculates the weight of each feature map by using the
confidence of the target class. Our experiments on two brain tumor datasets
show that Cfd-CAM outperforms existing state-of-the-art methods under the same
level of supervision. Overall, our proposed Cfd-CAM approach improves the
accuracy of brain tumor segmentation and may provide valuable insights for
developing better WSSS methods for other medical imaging tasks
Recent Progress in Transformer-based Medical Image Analysis
The transformer is primarily used in the field of natural language
processing. Recently, it has been adopted and shows promise in the computer
vision (CV) field. Medical image analysis (MIA), as a critical branch of CV,
also greatly benefits from this state-of-the-art technique. In this review, we
first recap the core component of the transformer, the attention mechanism, and
the detailed structures of the transformer. After that, we depict the recent
progress of the transformer in the field of MIA. We organize the applications
in a sequence of different tasks, including classification, segmentation,
captioning, registration, detection, enhancement, localization, and synthesis.
The mainstream classification and segmentation tasks are further divided into
eleven medical image modalities. A large number of experiments studied in this
review illustrate that the transformer-based method outperforms existing
methods through comparisons with multiple evaluation metrics. Finally, we
discuss the open challenges and future opportunities in this field. This
task-modality review with the latest contents, detailed information, and
comprehensive comparison may greatly benefit the broad MIA community.Comment: Computers in Biology and Medicine Accepte
- …