241 research outputs found
Meta-Registration: Learning Test-Time Optimization for Single-Pair Image Registration
Neural networks have been proposed for medical image registration by
learning, with a substantial amount of training data, the optimal
transformations between image pairs. These trained networks can further be
optimized on a single pair of test images - known as test-time optimization.
This work formulates image registration as a meta-learning algorithm. Such
networks can be trained by aligning the training image pairs while
simultaneously improving test-time optimization efficacy; tasks which were
previously considered two independent training and optimization processes. The
proposed meta-registration is hypothesized to maximize the efficiency and
effectiveness of the test-time optimization in the "outer" meta-optimization of
the networks. For image guidance applications that often are time-critical yet
limited in training data, the potentially gained speed and accuracy are
compared with classical registration algorithms, registration networks without
meta-learning, and single-pair optimization without test-time optimization
data. Experiments are presented in this paper using clinical transrectal
ultrasound image data from 108 prostate cancer patients. These experiments
demonstrate the effectiveness of a meta-registration protocol, which yields
significantly improved performance relative to existing learning-based methods.
Furthermore, the meta-registration achieves comparable results to classical
iterative methods in a fraction of the time, owing to its rapid test-time
optimization process.Comment: Accepted to ASMUS 2022 Workshop at MICCA
Meta-Learning Initializations for Interactive Medical Image Registration
We present a meta-learning framework for interactive medical image
registration. Our proposed framework comprises three components: a
learning-based medical image registration algorithm, a form of user interaction
that refines registration at inference, and a meta-learning protocol that
learns a rapidly adaptable network initialization. This paper describes a
specific algorithm that implements the registration, interaction and
meta-learning protocol for our exemplar clinical application: registration of
magnetic resonance (MR) imaging to interactively acquired, sparsely-sampled
transrectal ultrasound (TRUS) images. Our approach obtains comparable
registration error (4.26 mm) to the best-performing non-interactive
learning-based 3D-to-3D method (3.97 mm) while requiring only a fraction of the
data, and occurring in real-time during acquisition. Applying sparsely sampled
data to non-interactive methods yields higher registration errors (6.26 mm),
demonstrating the effectiveness of interactive MR-TRUS registration, which may
be applied intraoperatively given the real-time nature of the adaptation
process.Comment: 11 pages, 10 figures. Paper accepted to IEEE Transactions on Medical
Imaging (October 26 2022
Multimodality Biomedical Image Registration using Free Point Transformer Networks
We describe a point-set registration algorithm based on a novel free point
transformer (FPT) network, designed for points extracted from multimodal
biomedical images for registration tasks, such as those frequently encountered
in ultrasound-guided interventional procedures. FPT is constructed with a
global feature extractor which accepts unordered source and target point-sets
of variable size. The extracted features are conditioned by a shared multilayer
perceptron point transformer module to predict a displacement vector for each
source point, transforming it into the target space. The point transformer
module assumes no vicinity or smoothness in predicting spatial transformation
and, together with the global feature extractor, is trained in a data-driven
fashion with an unsupervised loss function. In a multimodal registration task
using prostate MR and sparsely acquired ultrasound images, FPT yields
comparable or improved results over other rigid and non-rigid registration
methods. This demonstrates the versatility of FPT to learn registration
directly from real, clinical training data and to generalize to a challenging
task, such as the interventional application presented.Comment: 10 pages, 4 figures. Accepted for publication at International
Conference on Medical Image Computing and Computer Assisted Intervention
(MICCAI) workshop on Advances in Simplifying Medical UltraSound (ASMUS) 202
Multi-phase synthetic contrast enhancement in interventional computed tomography for guiding renal cryotherapy
PURPOSE: Minimally invasive treatments for renal carcinoma offer a low rate of complications and quick recovery. One drawback of the use of computed tomography (CT) for needle guidance is the use of iodinated contrast agents, which require an increased X-ray dose and can potentially cause adverse reactions. The purpose of this work is to generalise the problem of synthetic contrast enhancement to allow the generation of multiple phases on non-contrast CT data from a real-world, clinical dataset without training multiple convolutional neural networks. METHODS: A framework for switching between contrast phases by conditioning the network on the phase information is proposed and compared with separately trained networks. We then examine how the degree of supervision affects the generated contrast by evaluating three established architectures: U-Net (fully supervised), Pix2Pix (adversarial with supervision), and CycleGAN (fully adversarial). RESULTS: We demonstrate that there is no performance loss when testing the proposed method against separately trained networks. Of the training paradigms investigated, the fully adversarial CycleGAN performs the worst, while the fully supervised U-Net generates more realistic voxel intensities and performed better than Pix2Pix in generating contrast images for use in a downstream segmentation task. Lastly, two models are shown to generalise to intra-procedural data not seen during the training process, also enhancing features such as needles and ice balls relevant to interventional radiological procedures. CONCLUSION: The proposed contrast switching framework is a feasible option for generating multiple contrast phases without the overhead of training multiple neural networks, while also being robust towards unseen data and enhancing contrast in features relevant to clinical practice
Intraoperative Organ Motion Models with an Ensemble of Conditional Generative Adversarial Networks
In this paper, we describe how a patient-specific, ultrasound-probe-induced
prostate motion model can be directly generated from a single preoperative MR
image. Our motion model allows for sampling from the conditional distribution
of dense displacement fields, is encoded by a generative neural network
conditioned on a medical image, and accepts random noise as additional input.
The generative network is trained by a minimax optimisation with a second
discriminative neural network, tasked to distinguish generated samples from
training motion data. In this work, we propose that 1) jointly optimising a
third conditioning neural network that pre-processes the input image, can
effectively extract patient-specific features for conditioning; and 2)
combining multiple generative models trained separately with heuristically
pre-disjointed training data sets can adequately mitigate the problem of mode
collapse. Trained with diagnostic T2-weighted MR images from 143 real patients
and 73,216 3D dense displacement fields from finite element simulations of
intraoperative prostate motion due to transrectal ultrasound probe pressure,
the proposed models produced physically-plausible patient-specific motion of
prostate glands. The ability to capture biomechanically simulated motion was
evaluated using two errors representing generalisability and specificity of the
model. The median values, calculated from a 10-fold cross-validation, were
2.8+/-0.3 mm and 1.7+/-0.1 mm, respectively. We conclude that the introduced
approach demonstrates the feasibility of applying state-of-the-art machine
learning algorithms to generate organ motion models from patient images, and
shows significant promise for future research.Comment: Accepted to MICCAI 201
Adversarial Deformation Regularization for Training Image Registration Neural Networks
We describe an adversarial learning approach to constrain convolutional
neural network training for image registration, replacing heuristic smoothness
measures of displacement fields often used in these tasks. Using
minimally-invasive prostate cancer intervention as an example application, we
demonstrate the feasibility of utilizing biomechanical simulations to
regularize a weakly-supervised anatomical-label-driven registration network for
aligning pre-procedural magnetic resonance (MR) and 3D intra-procedural
transrectal ultrasound (TRUS) images. A discriminator network is optimized to
distinguish the registration-predicted displacement fields from the motion data
simulated by finite element analysis. During training, the registration network
simultaneously aims to maximize similarity between anatomical labels that
drives image alignment and to minimize an adversarial generator loss that
measures divergence between the predicted- and simulated deformation. The
end-to-end trained network enables efficient and fully-automated registration
that only requires an MR and TRUS image pair as input, without anatomical
labels or simulated data during inference. 108 pairs of labelled MR and TRUS
images from 76 prostate cancer patients and 71,500 nonlinear finite-element
simulations from 143 different patients were used for this study. We show that,
with only gland segmentation as training labels, the proposed method can help
predict physically plausible deformation without any other smoothness penalty.
Based on cross-validation experiments using 834 pairs of independent validation
landmarks, the proposed adversarial-regularized registration achieved a target
registration error of 6.3 mm that is significantly lower than those from
several other regularization methods.Comment: Accepted to MICCAI 201
Learning Generalized Non-Rigid Multimodal Biomedical Image Registration from Generic Point Set Data
Free Point Transformer (FPT) has been proposed as a data-driven, non-rigid
point set registration approach using deep neural networks. As FPT does not
assume constraints based on point vicinity or correspondence, it may be trained
simply and in a flexible manner by minimizing an unsupervised loss based on the
Chamfer Distance. This makes FPT amenable to real-world medical imaging
applications where ground-truth deformations may be infeasible to obtain, or in
scenarios where only a varying degree of completeness in the point sets to be
aligned is available. To test the limit of the correspondence finding ability
of FPT and its dependency on training data sets, this work explores the
generalizability of the FPT from well-curated non-medical data sets to medical
imaging data sets. First, we train FPT on the ModelNet40 dataset to demonstrate
its effectiveness and the superior registration performance of FPT over
iterative and learning-based point set registration methods. Second, we
demonstrate superior performance in rigid and non-rigid registration and
robustness to missing data. Last, we highlight the interesting generalizability
of the ModelNet-trained FPT by registering reconstructed freehand ultrasound
scans of the spine and generic spine models without additional training,
whereby the average difference to the ground truth curvatures is 1.3 degrees,
across 13 patients.Comment: Accepted to ASMUS 2022 Workshop at MICCA
Assisted Probe Positioning for Ultrasound Guided Radiotherapy Using Image Sequence Classification
Effective transperineal ultrasound image guidance in prostate external beam
radiotherapy requires consistent alignment between probe and prostate at each
session during patient set-up. Probe placement and ultrasound image
inter-pretation are manual tasks contingent upon operator skill, leading to
interoperator uncertainties that degrade radiotherapy precision. We demonstrate
a method for ensuring accurate probe placement through joint classification of
images and probe position data. Using a multi-input multi-task algorithm,
spatial coordinate data from an optically tracked ultrasound probe is combined
with an image clas-sifier using a recurrent neural network to generate two sets
of predictions in real-time. The first set identifies relevant prostate anatomy
visible in the field of view using the classes: outside prostate, prostate
periphery, prostate centre. The second set recommends a probe angular
adjustment to achieve alignment between the probe and prostate centre with the
classes: move left, move right, stop. The algo-rithm was trained and tested on
9,743 clinical images from 61 treatment sessions across 32 patients. We
evaluated classification accuracy against class labels de-rived from three
experienced observers at 2/3 and 3/3 agreement thresholds. For images with
unanimous consensus between observers, anatomical classification accuracy was
97.2% and probe adjustment accuracy was 94.9%. The algorithm identified optimal
probe alignment within a mean (standard deviation) range of 3.7
(1.2) from angle labels with full observer consensus, comparable to
the 2.8 (2.6) mean interobserver range. We propose such an
algorithm could assist ra-diotherapy practitioners with limited experience of
ultrasound image interpreta-tion by providing effective real-time feedback
during patient set-up.Comment: Accepted to MICCAI 202
Label-driven weakly-supervised learning for multimodal deformable image registration
Spatially aligning medical images from different modalities remains a
challenging task, especially for intraoperative applications that require fast
and robust algorithms. We propose a weakly-supervised, label-driven formulation
for learning 3D voxel correspondence from higher-level label correspondence,
thereby bypassing classical intensity-based image similarity measures. During
training, a convolutional neural network is optimised by outputting a dense
displacement field (DDF) that warps a set of available anatomical labels from
the moving image to match their corresponding counterparts in the fixed image.
These label pairs, including solid organs, ducts, vessels, point landmarks and
other ad hoc structures, are only required at training time and can be
spatially aligned by minimising a cross-entropy function of the warped moving
label and the fixed label. During inference, the trained network takes a new
image pair to predict an optimal DDF, resulting in a fully-automatic,
label-free, real-time and deformable registration. For interventional
applications where large global transformation prevails, we also propose a
neural network architecture to jointly optimise the global- and local
displacements. Experiment results are presented based on cross-validating
registrations of 111 pairs of T2-weighted magnetic resonance images and 3D
transrectal ultrasound images from prostate cancer patients with a total of
over 4000 anatomical labels, yielding a median target registration error of 4.2
mm on landmark centroids and a median Dice of 0.88 on prostate glands.Comment: Accepted to ISBI 201
Strategising template-guided needle placement for MR-targeted prostate biopsy
Clinically significant prostate cancer has a better chance to be sampled
during ultrasound-guided biopsy procedures, if suspected lesions found in
pre-operative magnetic resonance (MR) images are used as targets. However, the
diagnostic accuracy of the biopsy procedure is limited by the
operator-dependent skills and experience in sampling the targets, a sequential
decision making process that involves navigating an ultrasound probe and
placing a series of sampling needles for potentially multiple targets. This
work aims to learn a reinforcement learning (RL) policy that optimises the
actions of continuous positioning of 2D ultrasound views and biopsy needles
with respect to a guiding template, such that the MR targets can be sampled
efficiently and sufficiently. We first formulate the task as a Markov decision
process (MDP) and construct an environment that allows the targeting actions to
be performed virtually for individual patients, based on their anatomy and
lesions derived from MR images. A patient-specific policy can thus be
optimised, before each biopsy procedure, by rewarding positive sampling in the
MDP environment. Experiment results from fifty four prostate cancer patients
show that the proposed RL-learned policies obtained a mean hit rate of 93% and
an average cancer core length of 11 mm, which compared favourably to two
alternative baseline strategies designed by humans, without hand-engineered
rewards that directly maximise these clinically relevant metrics. Perhaps more
interestingly, it is found that the RL agents learned strategies that were
adaptive to the lesion size, where spread of the needles was prioritised for
smaller lesions. Such a strategy has not been previously reported or commonly
adopted in clinical practice, but led to an overall superior targeting
performance when compared with intuitively designed strategies.Comment: Paper submitted and accepted to CaPTion (Cancer Prevention through
early detecTion) @ MICCAI 2022 Worksho
- …