293 research outputs found
Generation of annotated multimodal ground truth datasets for abdominal medical image registration
Sparsity of annotated data is a major limitation in medical image processing
tasks such as registration. Registered multimodal image data are essential for
the diagnosis of medical conditions and the success of interventional medical
procedures. To overcome the shortage of data, we present a method that allows
the generation of annotated multimodal 4D datasets. We use a CycleGAN network
architecture to generate multimodal synthetic data from the 4D extended
cardiac-torso (XCAT) phantom and real patient data. Organ masks are provided by
the XCAT phantom, therefore the generated dataset can serve as ground truth for
image segmentation and registration. Realistic simulation of respiration and
heartbeat is possible within the XCAT framework. To underline the usability as
a registration ground truth, a proof of principle registration is performed.
Compared to real patient data, the synthetic data showed good agreement
regarding the image voxel intensity distribution and the noise characteristics.
The generated T1-weighted magnetic resonance imaging (MRI), computed tomography
(CT), and cone beam CT (CBCT) images are inherently co-registered. Thus, the
synthetic dataset allowed us to optimize registration parameters of a
multimodal non-rigid registration, utilizing liver organ masks for evaluation.
Our proposed framework provides not only annotated but also multimodal
synthetic data which can serve as a ground truth for various tasks in medical
imaging processing. We demonstrated the applicability of synthetic data for the
development of multimodal medical image registration algorithms.Comment: 12 pages, 5 figures. This work has been published in the
International Journal of Computer Assisted Radiology and Surgery volum
Towards Cross-modality Medical Image Segmentation with Online Mutual Knowledge Distillation
The success of deep convolutional neural networks is partially attributed to
the massive amount of annotated training data. However, in practice, medical
data annotations are usually expensive and time-consuming to be obtained.
Considering multi-modality data with the same anatomic structures are widely
available in clinic routine, in this paper, we aim to exploit the prior
knowledge (e.g., shape priors) learned from one modality (aka., assistant
modality) to improve the segmentation performance on another modality (aka.,
target modality) to make up annotation scarcity. To alleviate the learning
difficulties caused by modality-specific appearance discrepancy, we first
present an Image Alignment Module (IAM) to narrow the appearance gap between
assistant and target modality data.We then propose a novel Mutual Knowledge
Distillation (MKD) scheme to thoroughly exploit the modality-shared knowledge
to facilitate the target-modality segmentation. To be specific, we formulate
our framework as an integration of two individual segmentors. Each segmentor
not only explicitly extracts one modality knowledge from corresponding
annotations, but also implicitly explores another modality knowledge from its
counterpart in mutual-guided manner. The ensemble of two segmentors would
further integrate the knowledge from both modalities and generate reliable
segmentation results on target modality. Experimental results on the public
multi-class cardiac segmentation data, i.e., MMWHS 2017, show that our method
achieves large improvements on CT segmentation by utilizing additional MRI data
and outperforms other state-of-the-art multi-modality learning methods.Comment: Accepted by AAAI 202
Multi-modality cardiac image computing: a survey
Multi-modality cardiac imaging plays a key role in the management of patients with cardiovascular diseases. It allows a combination of complementary anatomical, morphological and functional information, increases diagnosis accuracy, and improves the efficacy of cardiovascular interventions and clinical outcomes. Fully-automated processing and quantitative analysis of multi-modality cardiac images could have a direct impact on clinical research and evidence-based patient management. However, these require overcoming significant challenges including inter-modality misalignment and finding optimal methods to integrate information from different modalities.
This paper aims to provide a comprehensive review of multi-modality imaging in cardiology, the computing methods, the validation strategies, the related clinical workflows and future perspectives. For the computing methodologies, we have a favored focus on the three tasks, i.e., registration, fusion and segmentation, which generally involve multi-modality imaging data, either combining information from different modalities or transferring information across modalities. The review highlights that multi-modality cardiac imaging data has the potential of wide applicability in the clinic, such as trans-aortic valve implantation guidance, myocardial viability assessment, and catheter ablation therapy and its patient selection. Nevertheless, many challenges remain unsolved, such as missing modality, modality selection, combination of imaging and non-imaging data, and uniform analysis and representation of different modalities. There is also work to do in defining how the well-developed techniques fit in clinical workflows and how much additional and relevant information they introduce. These problems are likely to continue to be an active field of research and the questions to be answered in the future
Deep learning in medical image registration
Image registration is a fundamental task in multiple medical image analysis applications. With the advent of deep learning, there have been significant advances in algorithmic performance for various computer vision tasks in recent years, including medical image registration. The last couple of years have seen a dramatic increase in the development of deep learning-based medical image registration algorithms. Consequently, a comprehensive review of the current state-of-the-art algorithms in the field is timely, and necessary. This review is aimed at understanding the clinical applications and challenges that drove this innovation, analysing the functionality and limitations of existing approaches, and at providing insights to open challenges and as yet unmet clinical needs that could shape future research directions. To this end, the main contributions of this paper are: (a) discussion of all deep learning-based medical image registration papers published since 2013 with significant methodological and/or functional contributions to the field; (b) analysis of the development and evolution of deep learning-based image registration methods, summarising the current trends and challenges in the domain; and (c) overview of unmet clinical needs and potential directions for future research in deep learning-based medical image registration
KNOWLEDGE FUSION IN ALGORITHMS FOR MEDICAL IMAGE ANALYSIS
Medical imaging is one of the primary modalities used for clinical diagnosis and treatment planning. Building up a reliable automatic system to assist clinicians read the enormous amount of images benefits the efficiency and accuracy in general clinical trail. Recently deep learning techniques have been widely applied on medical images, but for applications in real clinical scenario, the accuracy, robustness, interpretability of those algorithms requires further validation.
In this dissertation, we introduce different strategies of knowledge fusion for improving current approaches in various tasks in medical image analysis. (i) To improve the robustness of segmentation algorithm, we propose to learn the shape prior for organ segmentation and apply it for automatic quality assessment. (ii) To detect pancreatic lesion with patient-level label only, we propose to extract shape and texture information from CT scans and combine them with a fusion network. (iii) In image registration, semantic information is important yet hard to obtain. We propose two methods for introducing semantic knowledge without the need of segmentation label. The first one designs a joint framework for registration synthesis and segmentation to share knowledge between different tasks. The second one introduces unsupervised semantic embedding to improve regular registration framework. (iv) To reduce the false positives in tumor detection task, we propose a hybrid feature engineering system extracting features of the tumor candidates from various perspectives and merging them in the decision stage
- …