428 research outputs found
Incorporating Prior Knowledge on Class Probabilities into Local Similarity Measures for Intermodality Image Registration
We present a methodology for incorporating prior knowledge on class probabilities into the registration process. By using knowledge from the imaging modality, pre-segmentations, and/or probabilistic atlases, we construct vectors of class probabilities for each image voxel. By defining new image similarity measures for distribution-valued images, we show how the class probability images can be nonrigidly registered in a variational framework. An experiment on nonrigid registration of MR and CT full-body scans illustrates that the proposed technique outperforms standard mutual information (MI) and normalized mutual information (NMI) based registration techniques when measured in terms of target registration error (TRE) of manually labeled fiducials
Fast Predictive Multimodal Image Registration
We introduce a deep encoder-decoder architecture for image deformation
prediction from multimodal images. Specifically, we design an image-patch-based
deep network that jointly (i) learns an image similarity measure and (ii) the
relationship between image patches and deformation parameters. While our method
can be applied to general image registration formulations, we focus on the
Large Deformation Diffeomorphic Metric Mapping (LDDMM) registration model. By
predicting the initial momentum of the shooting formulation of LDDMM, we
preserve its mathematical properties and drastically reduce the computation
time, compared to optimization-based approaches. Furthermore, we create a
Bayesian probabilistic version of the network that allows evaluation of
registration uncertainty via sampling of the network at test time. We evaluate
our method on a 3D brain MRI dataset using both T1- and T2-weighted images. Our
experiments show that our method generates accurate predictions and that
learning the similarity measure leads to more consistent registrations than
relying on generic multimodal image similarity measures, such as mutual
information. Our approach is an order of magnitude faster than
optimization-based LDDMM.Comment: Accepted as a conference paper for ISBI 201
Retrospective study on phantom for the application of medical image registration in the operating room scenario
This paper presents a phantom study to asses the feasibility of the medical image registration algorithms in the operating room (OR) scenario. The main issues of the registration algorithms in an OR application are, on one hand, the lack of the initial guess of the registration transformation - the images to be registered may be completely independentand, on the other hand, the multimodality of the data. Other requirements to be addressed by the OR registration algorithms are: real-time execution and the necessity of the validation of the results. This work analyzes how, under these requirements, the current state of the art algorithms in medical image registration may be used and shows which direction should be taken when designing a OR navigation system that includes registration as a component
Disentangled Autoencoder for Cross-Stain Feature Extraction in Pathology Image Analysis
A novel deep autoencoder architecture is proposed for the analysis of histopathology images. Its purpose is to produce a disentangled latent representation in which the structure and colour information are confined to different subspaces so that stain-independent models may be learned. For this, we introduce two constraints on the representation which are implemented as a classifier and an adversarial discriminator. We show how they can be used for learning a latent representation across haematoxylin-eosin and a number of immune stains. Finally, we demonstrate the utility of the proposed representation in the context of matching image patches for registration applications and for learning a bag of visual words for whole slide image summarization
Fast Predictive Image Registration
We present a method to predict image deformations based on patch-wise image
appearance. Specifically, we design a patch-based deep encoder-decoder network
which learns the pixel/voxel-wise mapping between image appearance and
registration parameters. Our approach can predict general deformation
parameterizations, however, we focus on the large deformation diffeomorphic
metric mapping (LDDMM) registration model. By predicting the LDDMM
momentum-parameterization we retain the desirable theoretical properties of
LDDMM, while reducing computation time by orders of magnitude: combined with
patch pruning, we achieve a 1500x/66x speed up compared to GPU-based
optimization for 2D/3D image registration. Our approach has better prediction
accuracy than predicting deformation or velocity fields and results in
diffeomorphic transformations. Additionally, we create a Bayesian probabilistic
version of our network, which allows evaluation of deformation field
uncertainty through Monte Carlo sampling using dropout at test time. We show
that deformation uncertainty highlights areas of ambiguous deformations. We
test our method on the OASIS brain image dataset in 2D and 3D
Multi-modality cardiac image computing: a survey
Multi-modality cardiac imaging plays a key role in the management of patients with cardiovascular diseases. It allows a combination of complementary anatomical, morphological and functional information, increases diagnosis accuracy, and improves the efficacy of cardiovascular interventions and clinical outcomes. Fully-automated processing and quantitative analysis of multi-modality cardiac images could have a direct impact on clinical research and evidence-based patient management. However, these require overcoming significant challenges including inter-modality misalignment and finding optimal methods to integrate information from different modalities.
This paper aims to provide a comprehensive review of multi-modality imaging in cardiology, the computing methods, the validation strategies, the related clinical workflows and future perspectives. For the computing methodologies, we have a favored focus on the three tasks, i.e., registration, fusion and segmentation, which generally involve multi-modality imaging data, either combining information from different modalities or transferring information across modalities. The review highlights that multi-modality cardiac imaging data has the potential of wide applicability in the clinic, such as trans-aortic valve implantation guidance, myocardial viability assessment, and catheter ablation therapy and its patient selection. Nevertheless, many challenges remain unsolved, such as missing modality, modality selection, combination of imaging and non-imaging data, and uniform analysis and representation of different modalities. There is also work to do in defining how the well-developed techniques fit in clinical workflows and how much additional and relevant information they introduce. These problems are likely to continue to be an active field of research and the questions to be answered in the future
Quicksilver: Fast Predictive Image Registration - a Deep Learning Approach
This paper introduces Quicksilver, a fast deformable image registration
method. Quicksilver registration for image-pairs works by patch-wise prediction
of a deformation model based directly on image appearance. A deep
encoder-decoder network is used as the prediction model. While the prediction
strategy is general, we focus on predictions for the Large Deformation
Diffeomorphic Metric Mapping (LDDMM) model. Specifically, we predict the
momentum-parameterization of LDDMM, which facilitates a patch-wise prediction
strategy while maintaining the theoretical properties of LDDMM, such as
guaranteed diffeomorphic mappings for sufficiently strong regularization. We
also provide a probabilistic version of our prediction network which can be
sampled during the testing time to calculate uncertainties in the predicted
deformations. Finally, we introduce a new correction network which greatly
increases the prediction accuracy of an already existing prediction network. We
show experimental results for uni-modal atlas-to-image as well as uni- / multi-
modal image-to-image registrations. These experiments demonstrate that our
method accurately predicts registrations obtained by numerical optimization, is
very fast, achieves state-of-the-art registration results on four standard
validation datasets, and can jointly learn an image similarity measure.
Quicksilver is freely available as an open-source software.Comment: Add new discussion
Motion estimation and correction for simultaneous PET/MR using SIRF and CIL
SIRF is a powerful PET/MR image reconstruction research tool for processing data and developing new algorithms. In this research, new developments to SIRF are presented, with focus on motion estimation and correction. SIRF's recent inclusion of the adjoint of the resampling operator allows gradient propagation through resampling, enabling the MCIR technique. Another enhancement enabled registering and resampling of complex images, suitable for MRI. Furthermore, SIRF's integration with the optimization library CIL enables the use of novel algorithms. Finally, SPM is now supported, in addition to NiftyReg, for registration. Results of MR and PET MCIR reconstructions are presented, using FISTA and PDHG, respectively. These demonstrate the advantages of incorporating motion correction and variational and structural priors. This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 2'
Image registration via stochastic gradient markov chain monte carlo
We develop a fully Bayesian framework for non-rigid registration of three-dimensional medical images, with a focus on uncertainty quantification. Probabilistic registration of large images along with calibrated uncertainty estimates is difficult for both computational and modelling reasons. To address the computational issues, we explore connections between the Markov chain Monte Carlo by backprop and the variational inference by backprop frameworks in order to efficiently draw thousands of samples from the posterior distribution. Regarding the modelling issues, we carefully design a Bayesian model for registration to overcome the existing barriers when using a dense, high-dimensional, and diffeomorphic parameterisation of the transformation. This results in improved calibration of uncertainty estimates
- …