11,912 research outputs found
Learning Deep Similarity Metric for 3D MR-TRUS Registration
Purpose: The fusion of transrectal ultrasound (TRUS) and magnetic resonance
(MR) images for guiding targeted prostate biopsy has significantly improved the
biopsy yield of aggressive cancers. A key component of MR-TRUS fusion is image
registration. However, it is very challenging to obtain a robust automatic
MR-TRUS registration due to the large appearance difference between the two
imaging modalities. The work presented in this paper aims to tackle this
problem by addressing two challenges: (i) the definition of a suitable
similarity metric and (ii) the determination of a suitable optimization
strategy.
Methods: This work proposes the use of a deep convolutional neural network to
learn a similarity metric for MR-TRUS registration. We also use a composite
optimization strategy that explores the solution space in order to search for a
suitable initialization for the second-order optimization of the learned
metric. Further, a multi-pass approach is used in order to smooth the metric
for optimization.
Results: The learned similarity metric outperforms the classical mutual
information and also the state-of-the-art MIND feature based methods. The
results indicate that the overall registration framework has a large capture
range. The proposed deep similarity metric based approach obtained a mean TRE
of 3.86mm (with an initial TRE of 16mm) for this challenging problem.
Conclusion: A similarity metric that is learned using a deep neural network
can be used to assess the quality of any given image registration and can be
used in conjunction with the aforementioned optimization framework to perform
automatic registration that is robust to poor initialization.Comment: To appear on IJCAR
Keypoint Transfer for Fast Whole-Body Segmentation
We introduce an approach for image segmentation based on sparse
correspondences between keypoints in testing and training images. Keypoints
represent automatically identified distinctive image locations, where each
keypoint correspondence suggests a transformation between images. We use these
correspondences to transfer label maps of entire organs from the training
images to the test image. The keypoint transfer algorithm includes three steps:
(i) keypoint matching, (ii) voting-based keypoint labeling, and (iii)
keypoint-based probabilistic transfer of organ segmentations. We report
segmentation results for abdominal organs in whole-body CT and MRI, as well as
in contrast-enhanced CT and MRI. Our method offers a speed-up of about three
orders of magnitude in comparison to common multi-atlas segmentation, while
achieving an accuracy that compares favorably. Moreover, keypoint transfer does
not require the registration to an atlas or a training phase. Finally, the
method allows for the segmentation of scans with highly variable field-of-view.Comment: Accepted for publication at IEEE Transactions on Medical Imagin
Medical Image Segmentation Based on Multi-Modal Convolutional Neural Network: Study on Image Fusion Schemes
Image analysis using more than one modality (i.e. multi-modal) has been
increasingly applied in the field of biomedical imaging. One of the challenges
in performing the multimodal analysis is that there exist multiple schemes for
fusing the information from different modalities, where such schemes are
application-dependent and lack a unified framework to guide their designs. In
this work we firstly propose a conceptual architecture for the image fusion
schemes in supervised biomedical image analysis: fusing at the feature level,
fusing at the classifier level, and fusing at the decision-making level.
Further, motivated by the recent success in applying deep learning for natural
image analysis, we implement the three image fusion schemes above based on the
Convolutional Neural Network (CNN) with varied structures, and combined into a
single framework. The proposed image segmentation framework is capable of
analyzing the multi-modality images using different fusing schemes
simultaneously. The framework is applied to detect the presence of soft tissue
sarcoma from the combination of Magnetic Resonance Imaging (MRI), Computed
Tomography (CT) and Positron Emission Tomography (PET) images. It is found from
the results that while all the fusion schemes outperform the single-modality
schemes, fusing at the feature level can generally achieve the best performance
in terms of both accuracy and computational cost, but also suffers from the
decreased robustness in the presence of large errors in any image modalities.Comment: Zhe Guo and Xiang Li contribute equally to this wor
HyperDense-Net: A hyper-densely connected CNN for multi-modal image segmentation
Recently, dense connections have attracted substantial attention in computer
vision because they facilitate gradient flow and implicit deep supervision
during training. Particularly, DenseNet, which connects each layer to every
other layer in a feed-forward fashion, has shown impressive performances in
natural image classification tasks. We propose HyperDenseNet, a 3D fully
convolutional neural network that extends the definition of dense connectivity
to multi-modal segmentation problems. Each imaging modality has a path, and
dense connections occur not only between the pairs of layers within the same
path, but also between those across different paths. This contrasts with the
existing multi-modal CNN approaches, in which modeling several modalities
relies entirely on a single joint layer (or level of abstraction) for fusion,
typically either at the input or at the output of the network. Therefore, the
proposed network has total freedom to learn more complex combinations between
the modalities, within and in-between all the levels of abstraction, which
increases significantly the learning representation. We report extensive
evaluations over two different and highly competitive multi-modal brain tissue
segmentation challenges, iSEG 2017 and MRBrainS 2013, with the former focusing
on 6-month infant data and the latter on adult images. HyperDenseNet yielded
significant improvements over many state-of-the-art segmentation networks,
ranking at the top on both benchmarks. We further provide a comprehensive
experimental analysis of features re-use, which confirms the importance of
hyper-dense connections in multi-modal representation learning. Our code is
publicly available at https://www.github.com/josedolz/HyperDenseNet.Comment: Paper accepted at IEEE TMI in October 2018. Last version of this
paper updates the reference to the IEEE TMI paper which compares the
submissions to the iSEG 2017 MICCAI Challeng
Multi-branch Convolutional Neural Network for Multiple Sclerosis Lesion Segmentation
In this paper, we present an automated approach for segmenting multiple
sclerosis (MS) lesions from multi-modal brain magnetic resonance images. Our
method is based on a deep end-to-end 2D convolutional neural network (CNN) for
slice-based segmentation of 3D volumetric data. The proposed CNN includes a
multi-branch downsampling path, which enables the network to encode information
from multiple modalities separately. Multi-scale feature fusion blocks are
proposed to combine feature maps from different modalities at different stages
of the network. Then, multi-scale feature upsampling blocks are introduced to
upsize combined feature maps to leverage information from lesion shape and
location. We trained and tested the proposed model using orthogonal plane
orientations of each 3D modality to exploit the contextual information in all
directions. The proposed pipeline is evaluated on two different datasets: a
private dataset including 37 MS patients and a publicly available dataset known
as the ISBI 2015 longitudinal MS lesion segmentation challenge dataset,
consisting of 14 MS patients. Considering the ISBI challenge, at the time of
submission, our method was amongst the top performing solutions. On the private
dataset, using the same array of performance metrics as in the ISBI challenge,
the proposed approach shows high improvements in MS lesion segmentation
compared with other publicly available tools.Comment: This paper has been accepted for publication in NeuroImag
Registration and Fusion of Multi-Spectral Images Using a Novel Edge Descriptor
In this paper we introduce a fully end-to-end approach for multi-spectral
image registration and fusion. Our method for fusion combines images from
different spectral channels into a single fused image by different approaches
for low and high frequency signals. A prerequisite of fusion is a stage of
geometric alignment between the spectral bands, commonly referred to as
registration. Unfortunately, common methods for image registration of a single
spectral channel do not yield reasonable results on images from different
modalities. For that end, we introduce a new algorithm for multi-spectral image
registration, based on a novel edge descriptor of feature points. Our method
achieves an accurate alignment of a level that allows us to further fuse the
images. As our experiments show, we produce a high quality of multi-spectral
image registration and fusion under many challenging scenarios
- …