35,846 research outputs found
CTransNet: Convolutional Neural Network Combined with Transformer for Medical Image Segmentation
The Transformer has been widely used for many tasks in NLP before, but there is still much room to explore the application of the Transformer to the image domain. In this paper, we propose a simple and efficient hybrid Transformer framework, CTransNet, which combines self-attention and CNN to improve medical image segmentation performance. Capturing long-range dependencies at different scales. To this end, this paper proposes an effective self-attention mechanism incorporating relative position information encoding, which can reduce the time complexity of self-attention from O(n2) to O(n), and a new self-attention decoder that can recover fine-grained features in encoder from skip connection. This paper aims to address the current dilemma of Transformer applications: i.e., the need to learn induction bias from large amounts of training data. The hybrid layer in CTransNet allows the Transformer to be initialized as a CNN without pre-training. We have evaluated the performance of CTransNet on several medical segmentation datasets. CTransNet shows superior segmentation performance, robustness, and great promise for generalization to other medical image segmentation tasks
Recommended from our members
A Hybrid Energy Model for Region Based Curve Evolution - Application to CTA Coronary Segmentation
Background and Objective: State-of-the-art medical imaging techniques have enabled non-invasive imaging of the internal organs. However, high volumes of imaging data make manual interpretation and delineation of abnormalities cumbersome for clinicians. These challenges have driven intensive research into efficient medical image segmentation. In this work, we propose a hybrid region-based energy formulation for effective segmentation in computed tomography angiography (CTA) imagery.
Methods: The proposed hybrid energy couples an intensity-based local term with an efficient discontinuity-based global model of the image for optimal segmentation. The segmentation is achieved using a level set formulation due to the computational robustness. After validating the statistical significance of the hybrid energy, we applied the proposed model to solve an important clinical problem of 3D coronary segmentation. An improved seed detection method is used to initialize the level set evolution. Moreover, we employed an auto-correction feature that captures the emerging peripheries during the curve evolution for completeness of the coronary tree.
Results: We evaluated the segmentation accuracy of the proposed energy model against the existing techniques in two stages. Qualitative and quantitative results demonstrate the effectiveness of the proposed framework with a consistent mean sensitivity and specificity measures of 80% across the CTA data. Moreover, a high degree of agreement with respect to the inter-observer differences justifies the generalization of the proposed method.
Conclusions: The proposed method is effective to segment the coronary tree from the CTA volume based on hybrid image based energy, which can improve the clinicians ability to detect arterial abnormalities
SW-UNet: a U-Net fusing sliding window transformer block with CNN for segmentation of lung nodules
Medical images are information carriers that visually reflect and record the anatomical structure of the human body, and play an important role in clinical diagnosis, teaching and research, etc. Modern medicine has become increasingly inseparable from the intelligent processing of medical images. In recent years, there have been more and more attempts to apply deep learning theory to medical image segmentation tasks, and it is imperative to explore a simple and efficient deep learning algorithm for medical image segmentation. In this paper, we investigate the segmentation of lung nodule images. We address the above-mentioned problems of medical image segmentation algorithms and conduct research on medical image fusion algorithms based on a hybrid channel-space attention mechanism and medical image segmentation algorithms with a hybrid architecture of Convolutional Neural Networks (CNN) and Visual Transformer. To the problem that medical image segmentation algorithms are difficult to capture long-range feature dependencies, this paper proposes a medical image segmentation model SW-UNet based on a hybrid CNN and Vision Transformer (ViT) framework. Self-attention mechanism and sliding window design of Visual Transformer are used to capture global feature associations and break the perceptual field limitation of convolutional operations due to inductive bias. At the same time, a widened self-attentive vector is used to streamline the number of modules and compress the model size so as to fit the characteristics of a small amount of medical data, which makes the model easy to be overfitted. Experiments on the LUNA16 lung nodule image dataset validate the algorithm and show that the proposed network can achieve efficient medical image segmentation on a lightweight scale. In addition, to validate the migratability of the model, we performed additional validation on other tumor datasets with desirable results. Our research addresses the crucial need for improved medical image segmentation algorithms. By introducing the SW-UNet model, which combines CNN and ViT, we successfully capture long-range feature dependencies and break the perceptual field limitations of traditional convolutional operations. This approach not only enhances the efficiency of medical image segmentation but also maintains model scalability and adaptability to small medical datasets. The positive outcomes on various tumor datasets emphasize the potential migratability and broad applicability of our proposed model in the field of medical image analysis
ScribFormer: Transformer Makes CNN Work Better for Scribble-based Medical Image Segmentation
Most recent scribble-supervised segmentation methods commonly adopt a CNN
framework with an encoder-decoder architecture. Despite its multiple benefits,
this framework generally can only capture small-range feature dependency for
the convolutional layer with the local receptive field, which makes it
difficult to learn global shape information from the limited information
provided by scribble annotations. To address this issue, this paper proposes a
new CNN-Transformer hybrid solution for scribble-supervised medical image
segmentation called ScribFormer. The proposed ScribFormer model has a
triple-branch structure, i.e., the hybrid of a CNN branch, a Transformer
branch, and an attention-guided class activation map (ACAM) branch.
Specifically, the CNN branch collaborates with the Transformer branch to fuse
the local features learned from CNN with the global representations obtained
from Transformer, which can effectively overcome limitations of existing
scribble-supervised segmentation methods. Furthermore, the ACAM branch assists
in unifying the shallow convolution features and the deep convolution features
to improve model's performance further. Extensive experiments on two public
datasets and one private dataset show that our ScribFormer has superior
performance over the state-of-the-art scribble-supervised segmentation methods,
and achieves even better results than the fully-supervised segmentation
methods. The code is released at https://github.com/HUANGLIZI/ScribFormer.Comment: Accepted by IEEE Transactions on Medical Imaging (TMI
ScribFormer: Transformer Makes CNN Work Better for Scribble-based Medical Image Segmentation
Most recent scribble-supervised segmentation methods commonly adopt a CNN framework with an encoder-decoder architecture. Despite its multiple benefits, this framework generally can only capture small-range feature dependency for the convolutional layer with the local receptive field, which makes it difficult to learn global shape information from the limited information provided by scribble annotations. To address this issue, this paper proposes a new CNN-Transformer hybrid solution for scribble-supervised medical image segmentation called ScribFormer. The proposed ScribFormer model has a triple-branch structure, i.e., the hybrid of a CNN branch, a Transformer branch, and an attention-guided class activation map (ACAM) branch. Specifically, the CNN branch collaborates with the Transformer branch to fuse the local features learned from CNN with the global representations obtained from Transformer, which can effectively overcome limitations of existing scribble-supervised segmentation methods. Furthermore, the ACAM branch assists in unifying the shallow convolution features and the deep convolution features to improve model’s performance further. Extensive experiments on two public datasets and one private dataset show that our ScribFormer has superior performance over the state-of-the-art scribble-supervised segmentation methods, and achieves even better results than the fully-supervised segmentation methods. The code is released at https://github.com/HUANGLIZI/ScribFormer
Atlas-Based Prostate Segmentation Using an Hybrid Registration
Purpose: This paper presents the preliminary results of a semi-automatic
method for prostate segmentation of Magnetic Resonance Images (MRI) which aims
to be incorporated in a navigation system for prostate brachytherapy. Methods:
The method is based on the registration of an anatomical atlas computed from a
population of 18 MRI exams onto a patient image. An hybrid registration
framework which couples an intensity-based registration with a robust
point-matching algorithm is used for both atlas building and atlas
registration. Results: The method has been validated on the same dataset that
the one used to construct the atlas using the "leave-one-out method". Results
gives a mean error of 3.39 mm and a standard deviation of 1.95 mm with respect
to expert segmentations. Conclusions: We think that this segmentation tool may
be a very valuable help to the clinician for routine quantitative image
exploitation.Comment: International Journal of Computer Assisted Radiology and Surgery
(2008) 000-99
- …